[Feature][MySQL CDC] MySQL cdc support start by time #9735
Conversation
| @Override | ||
| public Offset timestamp(long timestamp) { | ||
| throw new UnsupportedOperationException("not supported create new Offset by timestamp."); | ||
| // mysql binlog timestamp is second, so we need to divide 1000 |
There was a problem hiding this comment.
milliseconds?
in BinlogOffset timestamp is second
There was a problem hiding this comment.
so why we should / 1000?
https://github.com/apache/seatunnel/pull/9735/files/f780aa68a6e524e1401734506fbb5d867176c293
There was a problem hiding this comment.
so why we should
/ 1000? https://github.com/apache/seatunnel/pull/9735/files/f780aa68a6e524e1401734506fbb5d867176c293
the timestamp is milliseconds, but in BinlogOffset timestamp is second. so /1000 make milliseconds to second
|
|
||
| public static BinlogOffset findBinlogOffsetBytimestamp( | ||
| JdbcConnection jdbc, BinaryLogClient client, long timestamp) { | ||
| final String showBinaryLogStmt = "SHOW BINARY LOGS"; |
There was a problem hiding this comment.
cc @hawk9821 . After this PR merged we should update this for 8.4?
There was a problem hiding this comment.
@dyp12 We should support 8.4 too. Please refer e338743#diff-a059ddeb198852981f568a6087e543c358b90c5116cd918bcdf07c32e4008324R77
There was a problem hiding this comment.
@dyp12 We should support 8.4 too. Please refer e338743#diff-a059ddeb198852981f568a6087e543c358b90c5116cd918bcdf07c32e4008324R77
fixed it
| public static long getBinlogTimestamp(BinaryLogClient client, String binlogFile) | ||
| throws IOException, InterruptedException { | ||
|
|
||
| ArrayBlockingQueue<Long> binlogTimestamps = new ArrayBlockingQueue<>(1); |
There was a problem hiding this comment.
why we still use queue?
sorry,i fixed it
| super.handleEvent(partition, offsetContext, event); | ||
| } | ||
|
|
||
| private void updateOffsetPosition( |
There was a problem hiding this comment.
Could you explain why we must update MySqlOffsetContext when skip data?
There was a problem hiding this comment.
Could you explain why we must update
MySqlOffsetContextwhen skip data?
update MySqlOffsetContext,then checkpoint will save latest offset，when restore，only begin from save offset，not from earliest. in MySqlStreamingChangeEventSource.handleEvent also update it
|
Please rebase dev because we merged #9720 |
i fix it |
3 participants
Purpose of this pull request
close #9144
Does this PR introduce any user-facing change?
How was this patch tested?
Check list
New License Guide