Skip to content

Conversation

@CosmosNi
Copy link
Contributor

@CosmosNi CosmosNi commented Aug 28, 2025

  1. VECTOR_REDUCE - Vector dimensionality reduction with multiple algorithms
  2. VECTOR_NORMALIZE - Normalize vector to unit length

Supported Reduction Methods

  • TRUNCATE - Simple truncation: keep first N dimensions
  • RANDOM_PROJECTION - Gaussian random projection
  • SPARSE_RANDOM_PROJECTION - Sparse random projection for memory efficiency

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@Hisoka-X
Copy link
Member

waiting #9765 be merged first.

# Conflicts:
#	docs/en/transform-v2/sql-functions.md
#	docs/zh/transform-v2/sql-functions.md
#	seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLFunction.java
#	seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/VectorFunction.java
Comment on lines 215 to 216
// -------------------------vector functions----------------------------
public static final String VECTOR_REDUCE = "VECTOR_REDUCE";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// -------------------------vector functions----------------------------
public static final String VECTOR_REDUCE = "VECTOR_REDUCE";
public static final String VECTOR_REDUCE = "VECTOR_REDUCE";


// Helper methods

private static Float[] extractFloatArray(Object vectorData) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's reuse

@CosmosNi CosmosNi requested a review from Hisoka-X September 1, 2025 10:54
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @CosmosNi

@Hisoka-X Hisoka-X changed the title [Feature][Transform V2] Add Vector Dimension Reduction Transform [Feature][Transform V2] Add vector dimension reduction transform Sep 1, 2025
@corgy-w corgy-w merged commit c115613 into apache:dev Sep 2, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants