[Feature][Connector-V2] Support multi-table sink feature for HBase

[BruceWong96]

Purpose of this pull request

[Feature][HBase] Support multi-table sink feature #5652

Does this PR introduce any user-facing change?

How was this patch tested?

new e2e test

Check list

• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document. For more details you can refer to connector-v2
  2. Update plugin-mapping.properties and add new connector information in it
  3. Update the pom file of seatunnel-dist
• Update the release-note.

[BruceWong96]

@Hisoka-X
@hailin0
Hi, PTAL.
Thanks.

[Hisoka-X]

Thanks @BruceWong96 for contribute this feature! Could you follow the guide to open github action on your fork repository? https://github.com/apache/seatunnel/pull/7169/checks?check_run_id=27312810188

[BruceWong96]

Thanks @BruceWong96 for contribute this feature! Could you follow the guide to open github action on your fork repository? https://github.com/apache/seatunnel/pull/7169/checks?check_run_id=27312810188

OK

[BruceWong96]

@Hisoka-X
Up to now, Do I need to address JDK 11 compatibility issues ?
Looking forward to your reply !

[Hisoka-X]

@Hisoka-X Up to now, Do I need to address JDK 11 compatibility issues ? Looking forward to your reply !

Yes. Before we merge PR, we should make sure all test case passed.

[hailin0]

How does it support writing to multiple tables?

[hailin0]

Please add test cases to write to tables with different structures

[BruceWong96]

How does it support writing to multiple tables?

Maybe I have a problem with my understanding. Is it many-to-many that we're going to end up with ?

[hailin0]

@BruceWong96

You decide how to design this function.

e.g. multiple tables as input upstream:

source {
  FakeSource {
    tables_configs = [
      {
          schema {
            table = "table-11111"
            fields {
                id = string
                age = int
            }
          }
      },
      {
          schema {
            table = "table-2222"
            fields {
               f1 = boolean
               f2 = boolean
               f3 = tinyint
            }
          }
      }
    ]
  }
}

[BruceWong96]

@BruceWong96

You decide how to design this function.

e.g. multiple tables as input upstream:

source {
  FakeSource {
    tables_configs = [
      {
          schema {
            table = "table-11111"
            fields {
                id = string
                age = int
            }
          }
      },
      {
          schema {
            table = "table-2222"
            fields {
               f1 = boolean
               f2 = boolean
               f3 = tinyint
            }
          }
      }
    ]
  }
}

env {
  parallelism = 1
  job.mode = "BATCH"
}


source {
  FakeSource {
    tables_configs = [
      {
          schema {
            table = "table-11111"
            fields {
                id = string
                age = int
            }
          }
      },
      {
          schema {
            table = "table-2222"
            fields {
               f1 = boolean
               f2 = boolean
               f3 = tinyint
            }
          }
      }
    ]
  }
}

sink {
  Hbase {
    zookeeper_quorum = "hadoop001:2181,hadoop002:2181,hadoop003:2181"
    tables_configs = [
      {
        source_table_name = "table-11111"
        table = "hbase_table_1"
        rowkey_column = ["id"]
        family_name {
          id = "cf1"
          age = "cf1"
        }
      },
      {
        source_table_name = "table-2222"
        table = "hbase_table_2"
        rowkey_column = ["f1"]
        family_name {
          f1 = "cf2"
          f2 = "cf2"
          f3 = "cf2"
        }
      }
    ]
  }
}

Please check whether this configuration meets the requirements. If so, I will develop according to this configuration.
Thanks.

[hailin0]

Please update docs

[hailin0]

Please add test cases to write to tables with different structures

[hailin0]

@BruceWong96 I found a duplicate PR， here #6969, This is the correct way to code, you can use it as a reference for learning

@TaoZex @BruceWong96 Are you willing to collaborate with each other?

[BruceWong96]

@BruceWong96 I found a duplicate PR， here #6969, This is the correct way to code, you can use it as a reference for learning

@TaoZex @BruceWong96 Are you willing to collaborate with each other?

OK , I'm going to learn about it. So you can close this PR. Thanks.

[TaoZex]

@BruceWong96 I found a duplicate PR， here #6969, This is the correct way to code, you can use it as a reference for learning
@TaoZex @BruceWong96 Are you willing to collaborate with each other?

OK , I'm going to learn about it. So you can close this PR. Thanks.

I close that pr, you can use it as a reference to try to implement it. Feel free to contact me if you have any questions.

[BruceWong96]

@BruceWong96 I found a duplicate PR， here #6969, This is the correct way to code, you can use it as a reference for learning
@TaoZex @BruceWong96 Are you willing to collaborate with each other?

OK , I'm going to learn about it. So you can close this PR. Thanks.

I close that pr, you can use it as a reference to try to implement it. Feel free to contact me if you have any questions.

Thanks a lot . I will commit the code at a later time.

[hailin0]

hi @BruceWong96
Is there any progress on this task?

[BruceWong96]

hi @BruceWong96 Is there any progress on this task?

Yes, I am doing some local tests, sorry for the delay, I will update the code soon.

Thanks.

[BruceWong96]

@hailin0 PTAL.
Thanks.

[BruceWong96]

@Hisoka-X PTAL.
Thanks.

[BruceWong96]

@hailin0 PTAL.
Thanks.

[Hisoka-X]

LGTM if ci passes. Thanks @BruceWong96 !