close
Skip to content

[Feature][Connectors-V2][File]support assign encoding for file source/sink#6489

Merged
hailin0 merged 2 commits into
apache:devfrom
ShaunWuu:dev-240304-fileEncoding
Mar 14, 2024
Merged

[Feature][Connectors-V2][File]support assign encoding for file source/sink#6489
hailin0 merged 2 commits into
apache:devfrom
ShaunWuu:dev-240304-fileEncoding

Conversation

@ShaunWuu

Copy link
Copy Markdown
Contributor

Purpose of this pull request

This pull request support assign encoding for file source/sink

Does this PR introduce any user-facing change?

Yes.
A config to assign encoding for file source/sink. The name of the config is "encoding".
This pr is within the unreleased branch dev.

One of the example:

LocalFile {
    path = "/tmp/hive/warehouse/test2"
    file_format_type = "text"
    encoding = "gbk"
}

How was this patch tested?

Unit tests are as the followings:

  • org.apache.seatunnel.connectors.seatunnel.file.writer.ReadStrategyEncodingTest

E2E test cases are as the followings:

  • org.apache.seatunnel.e2e.connector.file.local.LocalFileIT

Check list

@hailin0

hailin0 commented Mar 12, 2024

Copy link
Copy Markdown
Member

cc @TyrantLucifer

@ShaunWuu ShaunWuu force-pushed the dev-240304-fileEncoding branch from d5adc8c to 424ca42 Compare March 12, 2024 10:27
@ShaunWuu

Copy link
Copy Markdown
Contributor Author

force-pushed the dev-240304-fileEncoding branch from d5adc8c to 424ca42

This force push deleted a redundant line of code.

@lightzhao

lightzhao commented Mar 12, 2024

Copy link
Copy Markdown
Contributor

#5973 cc @EricJoy2048

@Hisoka-X Hisoka-X left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM

@hailin0 hailin0 merged commit d159fbe into apache:dev Mar 14, 2024
chaorongzhi pushed a commit to chaorongzhi/seatunnel that referenced this pull request Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants