-
Binjie Yang authored
### _Why are the changes needed?_ Add Kyuubi tools cache-file-cleaner on kubernetes. This tools help to start daemonSet on Kubernetes to clean shuffle file and spark cache file. By config some envs, cache-file-cleaner will help to check some old file(which depending on configuration) and delete them for free dist. And after first cleaning, if the free capacity lower than threshold, cache file cleaner will start deep clean to help free dist. With the cache-file-cleaner, add Dockerfile, entrypoint and daemonSet.yaml to help user to use this tool. Using threadPool to help delete shuffle files. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request Closes #768 from zwangsheng/shuffle-cleaner. Closes #768 3e3d5c65 [Binjie Yang] tini 2f7eab6e [Binjie Yang] dist d19b9d0c [Binjie Yang] dist 3c8b7828 [Binjie Yang] dist 21bf5b4d [Binjie Yang] big change 4d803504 [Binjie Yang] rename to spark-block-cleaner & fix bug ba6515d0 [Binjie Yang] do some change 1084feaa [Binjie Yang] delete empty dir ada682da [Binjie Yang] big change aabbf9ba [Binjie Yang] bugfix a504448a [Binjie Yang] big change 0e0d5811 [Binjie Yang] modify 262fe314 [Binjie Yang] pom ad9c533c [Binjie Yang] dist 0364f6e9 [Binjie Yang] pom 6fbb5cae [Binjie Yang] dist 797bcc39 [Binjie Yang] shuffle cleaner Authored-by:
Binjie Yang <2213335496@qq.com> Signed-off-by:
ulysses-you <ulyssesyou18@gmail.com>
5e53748b