-
Notifications
You must be signed in to change notification settings - Fork 2.1k
[Improve][Connector-jdbc] add comments when schema not include all columns #9559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
d56d687 to
308def2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR ensures that column comments are retained when merging catalog tables if the schema doesn’t include all columns.
- Introduces a transformation in
mergeCatalogTableto wrap query columns withPhysicalColumnand carry over existing comments from the path schema when types match. - Updates unit tests to assert that comments are preserved for each merged column.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcCatalogUtils.java | Added streaming logic to map and preserve column comments during schema merge |
| seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcCatalogUtilsTest.java | Updated test cases to include expected comment values in merged columns |
Comments suppressed due to low confidence (1)
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcCatalogUtils.java:364
- [nitpick] The variable name
columnsWithCommentis descriptive but could be clearer—consider renaming tocolumnsWithPreservedCommentsormergedColumnsto better convey its role.
.columns(columnsWithComment)
| tableOfPath.getTableId().getDatabaseName(), | ||
| tableOfPath.getTableId().getSchemaName(), | ||
| tableOfPath.getTableId().getTableName()); | ||
| List<Column> columnsWithComment = |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| columnsOfQuery | ||
| .get(column.getName()) | ||
| .getDataType() |
Copilot
AI
Jul 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The comparison looks up columnsOfQuery.get(column.getName()), but column is already from that source. You can simplify by calling column.getDataType().getSqlType() directly to avoid the redundant map lookup.
| columnsOfQuery | |
| .get(column.getName()) | |
| .getDataType() | |
| column.getDataType() |
davidzollo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 if CI passed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
代码审查意见(PR #9559: [improve][Connector-jdbc] add comments when schema not include all columns)
优点:
- 该 PR 有效解决了 schema 未包含所有列时,字段注释丢失的问题,提升了 JDBC connector 的易用性和一致性。
- 代码实现思路清晰,逻辑简洁,易于维护。
- 单元测试已同步完善,覆盖了注释保留的场景。
建议与潜在问题:
- 代码简化建议:
- 在类型判断和注释拷贝时,
columnsOfQuery.get(column.getName()).getDataType()实际上可以直接用column.getDataType(),避免重复 map 查询,提升可读性。
- 在类型判断和注释拷贝时,
- 空指针安全:
- 请确保所有 map 查询(如
columnsOfPath.get(column.getName()))都做了 null 检查,避免因 schema 差异导致 NPE。
- 请确保所有 map 查询(如
- 注释说明:
- 建议在代码中补充注释,说明“仅当类型一致时才拷贝注释”的设计原因,便于后续维护者理解。
- 测试覆盖:
- 建议补充负向测试,如类型不一致或 path schema 缺失时注释不应被拷贝,提升健壮性。
结论:
本次改动对 JDBC connector 体验有明显提升。若能进一步优化代码细节和测试覆盖,将更加健壮。感谢贡献!
Hisoka-X
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @wildpea
…lumns (apache#9559) Co-authored-by: 龙妮 <[email protected]>
…lumns (apache#9559) Co-authored-by: 龙妮 <[email protected]>
Purpose of this pull request
In mergeCatalogTable, if not schemaIncludeAllColumns, column comment information will be lost.
This modification aims to preserve comments in such scenarios
Does this PR introduce any user-facing change?
How was this patch tested?
JdbcCatalogUtilsTest::testColumnNotIncludeMerge
tableOfQuery add comment to each column, and expect reserved in mergeTable
Check list
New License Guide