• Binjie Yang's avatar
    [KYUUBI #768] [K8s] [Tool] tools shuffle-files-cleaner · 5e53748b
    Binjie Yang authored
    ### _Why are the changes needed?_
    Add Kyuubi tools cache-file-cleaner on kubernetes.
    This tools help to start daemonSet on Kubernetes to clean shuffle file and spark cache file.
    By config some envs, cache-file-cleaner will help to check some old file(which depending on configuration) and delete them for free dist.
    And after first cleaning, if the free capacity lower than threshold, cache file cleaner will start deep clean to help free dist.
    With the cache-file-cleaner, add Dockerfile, entrypoint and daemonSet.yaml to help user to use this tool.
    Using threadPool to help delete shuffle files.
    
    ### _How was this patch tested?_
    - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
    
    - [ ] Add screenshots for manual tests if appropriate
    
    - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests
    
    ) locally before make a pull request
    
    Closes #768 from zwangsheng/shuffle-cleaner.
    
    Closes #768
    
    3e3d5c65 [Binjie Yang] tini
    2f7eab6e [Binjie Yang] dist
    d19b9d0c [Binjie Yang] dist
    3c8b7828 [Binjie Yang] dist
    21bf5b4d [Binjie Yang] big change
    4d803504 [Binjie Yang] rename to spark-block-cleaner & fix bug
    ba6515d0 [Binjie Yang] do some change
    1084feaa [Binjie Yang] delete empty dir
    ada682da [Binjie Yang] big change
    aabbf9ba [Binjie Yang] bugfix
    a504448a [Binjie Yang] big change
    0e0d5811 [Binjie Yang] modify
    262fe314 [Binjie Yang] pom
    ad9c533c [Binjie Yang] dist
    0364f6e9 [Binjie Yang] pom
    6fbb5cae [Binjie Yang] dist
    797bcc39 [Binjie Yang] shuffle cleaner
    Authored-by: default avatarBinjie Yang <2213335496@qq.com>
    Signed-off-by: default avatarulysses-you <ulyssesyou18@gmail.com>
    5e53748b