close
Skip to content

[Fix][connector-elasticsearch] support elasticsearch nest type && spark with Array<map>#8492

Merged
hailin0 merged 12 commits into
apache:devfrom
CosmosNi:fix-spark-es-nest
Jan 21, 2025
Merged

[Fix][connector-elasticsearch] support elasticsearch nest type && spark with Array<map>#8492
hailin0 merged 12 commits into
apache:devfrom
CosmosNi:fix-spark-es-nest

Conversation

@CosmosNi

@CosmosNi CosmosNi commented Jan 9, 2025

Copy link
Copy Markdown
Collaborator

support elasticsearch nest type && spark with Array

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@corgy-w corgy-w left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a good contribution,There is a small ut that needs to be added

Comment thread seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelRow.java Outdated
@corgy-w

corgy-w commented Jan 12, 2025

Copy link
Copy Markdown
Contributor

LGTM,Please @Hisoka-X review it

Comment thread seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelRow.java Outdated
@CosmosNi

Copy link
Copy Markdown
Collaborator Author

@Hisoka-X PTAL

Comment on lines +120 to +130
if (((ArrayType<?, ?>) dataType).getElementType() instanceof MapType) {
Object arrayMap =
Array.newInstance(ArrayBasedMapData.class, ((Map[]) field).length);
for (int i = 0; i < ((Map[]) field).length; i++) {
Map<?, ?> value = (Map<?, ?>) ((Map[]) field)[i];
MapType<?, ?> type =
(MapType<?, ?>) ((ArrayType<?, ?>) dataType).getElementType();
Array.set(arrayMap, i, convertMap(value, type));
}
return ArrayData.toArrayData(arrayMap);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good

@Hisoka-X

Copy link
Copy Markdown
Member

Overall LGTM.

@hailin0 hailin0 merged commit 92d2a4a into apache:dev Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants