close
Skip to content

[Feature][Zeta] Support slf4j mdc tracing job id output into logs#7491

Merged
Carl-Zhou-CN merged 1 commit into
apache:devfrom
hailin0:dev-mdc-zeta-logs
Sep 2, 2024
Merged

[Feature][Zeta] Support slf4j mdc tracing job id output into logs#7491
Carl-Zhou-CN merged 1 commit into
apache:devfrom
hailin0:dev-mdc-zeta-logs

Conversation

@hailin0

@hailin0 hailin0 commented Aug 24, 2024

Copy link
Copy Markdown
Member

Purpose of this pull request

[Feature][Zeta] Support slf4j mdc tracing job id output into logs

reference
https://www.slf4j.org/manual.html#mdc
https://logging.apache.org/log4j/2.x/manual/thread-context.html#configuration
https://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout

subtask of #2725

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added

Check list

@github-actions github-actions Bot added core SeaTunnel core module Zeta e2e api labels Aug 24, 2024
@hailin0

hailin0 commented Aug 24, 2024

Copy link
Copy Markdown
Member Author

img_v3_02e2_5a13324d-dd6b-4d3c-94d5-d1f0fcb443fg
img_v3_02e2_e03e4732-5174-42ac-a91f-72cab838f89g
img_v3_02e2_a24373b2-777f-433d-99b7-130eb11c2f0g
img_v3_02e2_46670f2e-388e-40ec-9217-aa26034df05g

@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 2 times, most recently from b2e749e to b8fcb17 Compare August 28, 2024 06:46
@github-actions github-actions Bot added the CI&CD label Aug 28, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 2 times, most recently from c9c175d to 9c546ec Compare August 30, 2024 15:12
@github-actions github-actions Bot removed the CI&CD label Aug 30, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 3 times, most recently from 0c06e63 to a2860eb Compare August 30, 2024 16:20
@github-actions github-actions Bot added the CI&CD label Aug 30, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch from a2860eb to 188047b Compare August 30, 2024 16:24
@github-actions github-actions Bot removed the CI&CD label Aug 30, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 3 times, most recently from 5c93167 to ed5e863 Compare August 30, 2024 17:33
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

img_v3_02e9_d9119e5c-9e1f-4b6d-9f87-b9d2aa80403g

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

img_v3_02e9_7f329aa6-f531-444e-80dd-63397ffa35ag

@hailin0 hailin0 marked this pull request as ready for review September 1, 2024 14:59

# Log4j2 Config
JAVA_OPTS="${JAVA_OPTS} -Dlog4j2.contextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector"
JAVA_OPTS="${JAVA_OPTS} -Dlog4j2.isThreadContextMapInheritable=true"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Carl-Zhou-CN Carl-Zhou-CN left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@Carl-Zhou-CN Carl-Zhou-CN merged commit e224bda into apache:dev Sep 2, 2024
@arshadmohammad

arshadmohammad commented Sep 2, 2024

Copy link
Copy Markdown
Contributor

Good feature, Thanks @hailin0 for working on this.
related to this feature, is there anyway to get all the logs corresponding to job id, using API or command ?

@hailin0

hailin0 commented Sep 2, 2024

Copy link
Copy Markdown
Member Author

Good feature, Thanks @hailin0 for working on this. related to this feature, is there anyway to get all the logs corresponding to job id, using API or command ?

@arshadmohammad
The logs of distributed programs(job) are scattered on different machines. What are your thoughts?

@arshadmohammad

arshadmohammad commented Sep 2, 2024

Copy link
Copy Markdown
Contributor

I was looking for functionality similar to mapreduce and yarn, all mapreduce jobs have separate logs.

@hailin0

hailin0 commented Sep 2, 2024

Copy link
Copy Markdown
Member Author

I will do some verification later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants