Skip to content

Conversation

@xiaochen-zhou
Copy link
Contributor

@xiaochen-zhou xiaochen-zhou commented Aug 16, 2025

Purpose of this pull request

Support jsonpath extracting multiple fields in a configuration block

transform {
  JsonPath {
    plugin_input = "fake"
    plugin_output = "fake1"
    columns = [
     {
        "src_field" = "data"
        "path" = ["$.data.c_string", "$.data.c_boolean", "$.data.c_integer", "$.data.c_float", "$.data.c_double", "$.data.c_decimal", "$.data.c_date", "$.data.c_datetime", "$.data.c_array", "$.data.c_map_array"]
        "dest_field" = ["c1_string", "c1_boolean", "c1_integer", "c1_float", "c1_double", "c1_decimal", "c1_date", "c1_datetime", "c1_array", "c1_map_array"]
        "dest_type" = ["string", "boolean", "int", "float", "double", "decimal(4,2)", "date", "time", "array<string>", "array<map<string, string>>"]
     }
    ]
  }
}

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Add new tests

Check list

@xiaochen-zhou xiaochen-zhou force-pushed the optimze-simple-jsonpath branch from b8de226 to 1dd9bc6 Compare August 17, 2025 12:44
columns = [
{
"src_field" = "data"
"path" = "$.data.c_string;$.data.c_boolean;$.data.c_integer;$.data.c_float;$.data.c_double;$.data.c_decimal;$.data.c_date;$.data.c_datetime;$.data.c_array;$.data.c_map_array"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking can we use array type value?

"path" = ["$.data.c_string", $.data.c_boolean]

@Hisoka-X
Copy link
Member

waiting test case passes.

@xiaochen-zhou
Copy link
Contributor Author

waiting test case passes.

done.

@corgy-w corgy-w merged commit c57bdd1 into apache:dev Aug 26, 2025
5 checks passed
@xiaochen-zhou xiaochen-zhou deleted the optimze-simple-jsonpath branch October 9, 2025 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants