Skip to content

[Bug] [Connector-V2] sink trying to update primary key in distribute sink database #9293

@papadave66

Description

@papadave66

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

I am trying to running seatunnel with standalone mysql database and cluster postgresql sink database.
And it gives following ERROR:

It seems that it is trying to update id's value with id condition. how to prevent updating primary columns?

SeaTunnel Version

2.3.10

SeaTunnel Config

env {
  job.name = "mysql-cdc-test"
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 5000
}

source {
  MySQL-CDC {
    server-id = 5652
    username = "system"
    password = "xxxxxxxxx"
    table-names = ["seatunnel.role"]
    base-url = "jdbc:mysql://10.xx.xx.xx:3306"
  }
}

transform {
}

sink {
  jdbc {
    url = "jdbc:postgresql://xxxxxxxxx:11345/tmodel"
    driver = "org.postgresql.Driver"
    user = "xxxxx"
    password = "xxxxxxxx"
    generate_sink_sql = true
    data_save_mode = DROP_DATA
    database = "tmodel"
    table = "data.${table_name}"
    primary_keys = ["${primary_key}"]
    enable_upsert = false
    support_upsert_by_query_primary_key_exist = false
  }
}

Running Command

`seatunnel.sh --config mysql-cdc-test.conf `

Error Exception

Caused by: java.sql.BatchUpdateException: Batch entry 0 UPDATE "tmodel"."data"."role" SET "id" = ('15'::int4), "type" = ('1'::int4), "role_name" = ('NORMAL_ROLE2'), "description" = ('Normal User'), "create_time" = ('2025-03-24 08:45:55.804+08'), "update_time" = ('2025-05-08 13:31:13.768+08') WHERE "id" = ('15'::int4) was aborted: ERROR: Distributed column "id" can't be updated in current version  Call getNextException to see other errors in the batch.
        at org.postgresql.jdbc.BatchResultHandler.handleCompletion(BatchResultHandler.java:186)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:591)
        at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:889)
        at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:913)
        at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1739)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement.executeBatch(FieldNamedPreparedStatement.java:540)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.InsertOrUpdateBatchStatementExecutor.executeBatch(InsertOrUpdateBatchStatementExecutor.java:108)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.BufferReducedBatchStatementExecutor.executeBatch(BufferReducedBatchStatementExecutor.java:89)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.attemptFlush(JdbcOutputFormat.java:172)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.flush(JdbcOutputFormat.java:136)
        ... 9 more
Caused by: org.postgresql.util.PSQLException: ERROR: Distributed column "id" can't be updated in current version
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2733)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2420)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:580)
        ... 19 more

        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:220)
        ... 2 more
2025-05-08 13:31:41,309 INFO  [s.c.s.s.c.ClientExecuteCommand] [SeaTunnel-CompletableFuture-Thread-0] - run shutdown hook because get close signal

Zeta or Flink or Spark Version

zeta with seatunnel v2.3.10

Java or Scala Version

java version "1.8.0_391"
Java(TM) SE Runtime Environment (build 1.8.0_391-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.391-b13, mixed mode)

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions