close
Skip to content

[Improve] Add disable 2pc in SelectDB cloud sink#6266

Merged
ruanwenjun merged 2 commits into
apache:devfrom
Hisoka-X:selectdb-batch-load
Feb 23, 2024
Merged

[Improve] Add disable 2pc in SelectDB cloud sink#6266
ruanwenjun merged 2 commits into
apache:devfrom
Hisoka-X:selectdb-batch-load

Conversation

@Hisoka-X

Copy link
Copy Markdown
Member

Purpose of this pull request

This PR add config sink.enable-2pc to disable 2pc feature in SelectDB cloud, default value is true.
Currently, SelectDB cloud writes temporary files that rely on cache, and then submits the files to the database in the second phase. However, if the amount of data is large, the file may have expired when the second phase files are submitted. So this PR supports commit file every time when it is generated. It only takes effect when the 2pc feature is turned off.

Does this PR introduce any user-facing change?

no

How was this patch tested?

tested in local with selectdb cloud.

Check list

@Hisoka-X Hisoka-X added the feature New feature label Jan 23, 2024
@ruanwenjun ruanwenjun merged commit aa0b211 into apache:dev Feb 23, 2024
chaorongzhi pushed a commit to chaorongzhi/seatunnel that referenced this pull request Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants