[Improve][Zeta] Handle user privacy when submitting a task print config logs #7247
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hawk9821
changed the title
Member
Hisoka-X
reviewed
Jul 29, 2024
|
|
||
| public static final ConfigRenderOptions CONFIG_RENDER_OPTIONS = | ||
| ConfigRenderOptions.concise().setFormatted(true); | ||
| public static List<String> KEYWORDS = |
Member
There was a problem hiding this comment.
I am not sure whether we need this feature as different key names need to be maintained
Contributor
Author
There was a problem hiding this comment.
I think it is necessary to print the configuration log for easy troubleshooting. We can start by maintaining generic keywords such as "password" and "secret".
Member
There was a problem hiding this comment.
Contributor
Author
There was a problem hiding this comment.
configure privacy keywords in seatunnel's profile，i will modified.
Hisoka-X
reviewed
Aug 8, 2024
Comment on lines
41
to
46
| <dependency> | ||
| <groupId>org.apache.seatunnel</groupId> | ||
| <artifactId>seatunnel-engine-common</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
|
|
Member
There was a problem hiding this comment.
-1. we should not include this dependency. The starter for all engine, not only for zeta.
| } | ||
|
|
||
| public static Map<String, Object> privacyHandler(Map<String, Object> configMap) { | ||
| List<String> logPrivacyKeyword = seaTunnelConfig.getEngineConfig().getLogPrivacyKeyword(); |
Member
There was a problem hiding this comment.
Please just use
as value for now.
config/seatunnel.yaml
Outdated
| queue-type: blockingqueue | ||
| print-execution-info-interval: 60 | ||
| print-job-metrics-info-interval: 60 | ||
| log-privacy-keyword: username,password,passwd,token,jwt,secret |
Member
There was a problem hiding this comment.
At this stage, there is no need to introduce this parameter, because the configuration printing config is effective for all engines, but this configuration file is only used for zeta.
hawk9821
force-pushed
the
logPrivacy
branch
2 times, most recently
from
b3bb458 to
20a28d2
Compare
Hisoka-X
reviewed
Aug 21, 2024
| return config; | ||
| } | ||
|
|
||
| public static Map<String, Object> privacyHandler(Map<String, Object> configMap) { |
Member
There was a problem hiding this comment.
Suggested change
| public static Map<String, Object> privacyHandler(Map<String, Object> configMap) { | |
| public static Map<String, Object> configDesensitization(Map<String, Object> configMap) { |
Hisoka-X
changed the title
hawk9821
force-pushed
the
logPrivacy
branch
3 times, most recently
from
38713f6 to
7e63669
Compare
…ion logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs [improve] Handle user privacy when submitting a task print configuration logs
hailin0
approved these changes
Aug 24, 2024
hawk9821
added a commit
to hawk9821/seatunnel
that referenced
this pull request
Aug 29, 2024
hawk9821
added a commit
to hawk9821/seatunnel
that referenced
this pull request
Aug 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose of this pull request
Contribution Checklist
#6987
#6980
privacy keywords should be desensitized instead of deleting log prints directly
Does this PR introduce any user-facing change?
How was this patch tested?
tests case: ConfigShadeTest.testUsePrivacyHandler
Check list
New License Guide
release-note.