close
Skip to content

[Feature][Paimon] Customize the hadoop user #8888

Merged
hailin0 merged 1 commit into
apache:devfrom
hawk9821:paimon_hadoop_user
Mar 6, 2025
Merged

[Feature][Paimon] Customize the hadoop user #8888
hailin0 merged 1 commit into
apache:devfrom
hawk9821:paimon_hadoop_user

Conversation

@hawk9821

@hawk9821 hawk9821 commented Mar 3, 2025

Copy link
Copy Markdown
Contributor

Purpose of this pull request

Customize the hadoop user when read-write Paimon on Hadoop . for examples :

paimon.hadoop.conf = {
      hadoop_user_name = "hdfs"
      fs.defaultFS = "hdfs://nameservice1"
      dfs.nameservices = "nameservice1"
      dfs.ha.namenodes.nameservice1 = "nn1,nn2"
      dfs.namenode.rpc-address.nameservice1.nn1 = "hadoop03:8020"
      dfs.namenode.rpc-address.nameservice1.nn2 = "hadoop04:8020"
      dfs.client.failover.proxy.provider.nameservice1 = "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"
      dfs.client.use.datanode.hostname = "true"
    }

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@hawk9821 hawk9821 changed the title [Feature][Paimon] Customize the hadoop user when read-write Paimon on Hadoop [Feature][Paimon] Customize the hadoop user Mar 3, 2025
@Hisoka-X

Hisoka-X commented Mar 4, 2025

Copy link
Copy Markdown
Member

Does hadoop has own key to set read user? It's weird to maintain hadoop series key by ourself. Or reuse REMOTE_USER

@hawk9821

hawk9821 commented Mar 4, 2025

Copy link
Copy Markdown
Contributor Author

Does hadoop has own key to set read user?
the key is HADOOP_USER_NAME

image

@hailin0 hailin0 merged commit 2657626 into apache:dev Mar 6, 2025
xiaochen-zhou pushed a commit to TongchengOpenSource/seatunnel that referenced this pull request Mar 21, 2025
@hawk9821 hawk9821 deleted the paimon_hadoop_user branch March 27, 2025 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants