Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • I Incubator Kyuubi
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
    • Requirements
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • 小 白蛋
  • Incubator Kyuubi
  • Repository
Switch branch/tag
  • incubator-kyuubi
  • externals
  • kyuubi-spark-sql-engine
  • pom.xml
Find file BlameHistoryPermalink
  • timothy65535's avatar
    [KYUUBI #894] Enhance kyuubi-zookeeper module · 5526898f
    timothy65535 authored Aug 11, 2021
    <!--
    Thanks for sending a pull request!
    
    Here are some tips for you:
      1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
      2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
      3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
    -->
    
    ### _Why are the changes needed?_
    <!--
    Please clarify why the changes are needed. For instance,
      1. If you add a feature, you can talk about the use case of it.
      2. If you fix a bug, you can clarify why it is a bug.
    -->
    
    ### 1. Describe the feature
    
    Kyuubi use `TestingServer` as the EmbeddedZookeeper service in codebase, and require many args to init the service. These `electionPort`, `quorumPort`, `serverId` parameters only work in cluster mode, these ports are not bound to the host.
    
    From hadoop and bookkeeper project, they use `ZooKeeperServer` as a built-in zk service, especially the bookeeper project.
    - https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java
    - https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java
    
    ### 2. Describe the solution
    
    Improve
    - Remove `electionPort`, `quorumPort`, `serverId` which are nerver be used, hadoop and bookkeer do the same
    - Replace `TestingServer` with `ZooKeeperServer`
    - Remove `curator-test` dependency
    
    ### 3. `deleteDataDirectoryOnClose`
    
    The name style for `deleteDataDirectoryOnClose` option, refer to hadoop `core-default.xml`
    https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/core-default.xml
    ```
    hadoop.security.dns.log-slow-lookups.enabled
    hadoop.security.dns.log-slow-lookups.threshold.ms
    dfs.client.block.write.replace-datanode-on-failure.policy
    ```
    
    ### _How was this patch tested?_
    - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
    
    - [ ] Add screenshots for manual tests if appropriate
    
    - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests
    
    ) locally before make a pull request
    
    Closes #907 from timothy65535/ky-894.
    
    Closes #894
    
    7d7fde9f [timothy65535] remove useless config
    ce06a0fd [timothy65535] limit travis build only master branch
    ff825c3f [timothy65535] revert deleteDataDirectoryOnClose config
    a336544a [timothy65535] remove javassist
    3da0bd55 [timothy65535] remove javassist
    0abb44df [timothy65535] [KYUUBI #894] Enhance kyuubi-zookeeper module
    Authored-by: default avatartimothy65535 <timothy65535@163.com>
    Signed-off-by: default avatarulysses-you <ulyssesyou18@gmail.com>
    5526898f

免费DevSecOps平台,让您的项目体验完整的DevSecOps流程,让项目更安全