-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Closed
Labels
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
I am trying to running seatunnel with standalone mysql database and cluster postgresql sink database.
And it gives following ERROR:
It seems that it is trying to update id's value with id condition. how to prevent updating primary columns?
SeaTunnel Version
2.3.10
SeaTunnel Config
env {
job.name = "mysql-cdc-test"
parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 5000
}
source {
MySQL-CDC {
server-id = 5652
username = "system"
password = "xxxxxxxxx"
table-names = ["seatunnel.role"]
base-url = "jdbc:mysql://10.xx.xx.xx:3306"
}
}
transform {
}
sink {
jdbc {
url = "jdbc:postgresql://xxxxxxxxx:11345/tmodel"
driver = "org.postgresql.Driver"
user = "xxxxx"
password = "xxxxxxxx"
generate_sink_sql = true
data_save_mode = DROP_DATA
database = "tmodel"
table = "data.${table_name}"
primary_keys = ["${primary_key}"]
enable_upsert = false
support_upsert_by_query_primary_key_exist = false
}
}
Running Command
`seatunnel.sh --config mysql-cdc-test.conf `Error Exception
Caused by: java.sql.BatchUpdateException: Batch entry 0 UPDATE "tmodel"."data"."role" SET "id" = ('15'::int4), "type" = ('1'::int4), "role_name" = ('NORMAL_ROLE2'), "description" = ('Normal User'), "create_time" = ('2025-03-24 08:45:55.804+08'), "update_time" = ('2025-05-08 13:31:13.768+08') WHERE "id" = ('15'::int4) was aborted: ERROR: Distributed column "id" can't be updated in current version Call getNextException to see other errors in the batch.
at org.postgresql.jdbc.BatchResultHandler.handleCompletion(BatchResultHandler.java:186)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:591)
at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:889)
at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:913)
at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1739)
at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement.executeBatch(FieldNamedPreparedStatement.java:540)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.InsertOrUpdateBatchStatementExecutor.executeBatch(InsertOrUpdateBatchStatementExecutor.java:108)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.BufferReducedBatchStatementExecutor.executeBatch(BufferReducedBatchStatementExecutor.java:89)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.attemptFlush(JdbcOutputFormat.java:172)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.flush(JdbcOutputFormat.java:136)
... 9 more
Caused by: org.postgresql.util.PSQLException: ERROR: Distributed column "id" can't be updated in current version
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2733)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2420)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:580)
... 19 more
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:220)
... 2 more
2025-05-08 13:31:41,309 INFO [s.c.s.s.c.ClientExecuteCommand] [SeaTunnel-CompletableFuture-Thread-0] - run shutdown hook because get close signal
Zeta or Flink or Spark Version
zeta with seatunnel v2.3.10
Java or Scala Version
java version "1.8.0_391"
Java(TM) SE Runtime Environment (build 1.8.0_391-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.391-b13, mixed mode)
Screenshots
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct