-
Notifications
You must be signed in to change notification settings - Fork 2.1k
[Improve][Connector-V2] MaxComputeSink support create partition in savemode #8474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
e7bf956 to
af56153
Compare
davidzollo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
| super.recreateSchema(); | ||
| if (StringUtils.isNotEmpty(readonlyConfig.get(PARTITION_SPEC))) { | ||
| ((MaxComputeCatalog) catalog) | ||
| .createPartition( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the code here may be copied and pasted from the previous method, so it should be written like this
((MaxComputeCatalog) catalog)
.truncatePartition(
tablePath, new PartitionSpec(readonlyConfig.get(PARTITION_SPEC)));
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @W-dragan , after recreate schema, the old partition will be deleted too, so why we need truncate it not create it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My understanding of 'recreate schema' is to delete a table or partition, and then recreate the table/partition. In the context of maxcompute, it is equivalent to overwrite=true.
Otherwise, the method you wrote in MaxCompute Catalog will not be called anywhere. Or is this method written in advance for future expansion?
Line 256 in df14201
| public void truncateTable(TablePath tablePath, boolean ignoreIfNotExists) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it will be invoked in
seatunnel/seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/DefaultSaveModeHandler.java
Line 217 in df14201
| catalog.truncateTable(tablePath, true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Understood, there is a deviation in my understanding. Thank you for your guidance




Purpose of this pull request
MaxComputeSink support create partition in savemode.
Does this PR introduce any user-facing change?
no
How was this patch tested?
tested in local
Check list
New License Guide
release-note.