• Cheng Pan's avatar
    [KYUUBI #2439] Using Pure Java TPC-DS generator · 3ecdd422
    Cheng Pan authored
    ### _Why are the changes needed?_
    
    This PR proposes change the Kyuubi TPC-DS generator to pure Java implementation instead of the original C binary.
    
    The new pure Java TPC-DS generator is under Apache License, and in fact, I don't know the original C binary License, so we exclude them from release in the past.
    
    Since the change removes the License issue of Kyuubi TPC-DS module, we can bundle the TPC-DS tool in the future release.
    
    And after migration, I haven't see "error=26, Text file busy" described in #2439 any more.
    
    ### _How was this patch tested?_
    - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
    
    - [x] Add screenshots for manual tests if appropriate
    
    1. Use old C binary based TPC-DS generator generate 1GB data under database `tpcds_s1`
    2. Use new pure Java base TPC-DS generator generate 1GB data under database `new_tpcds_sf1`
    3. Compare results of `select count(*)`, and `select sum(hash(*))`
    
    ```
    spark-sql> select count(*) from tpcds_s1.inventory;
    11745000
    Time taken: 0.161 seconds, Fetched 1 row(s)
    spark-sql> select count(*) from new_tpcds_sf1.inventory;
    11745000
    Time taken: 0.141 seconds, Fetched 1 row(s)
    spark-sql> select sum(hash(*)) from tpcds_s1.inventory;
    -556768665838
    Time taken: 0.252 seconds, Fetched 1 row(s)
    spark-sql> select sum(hash(*)) from new_tpcds_sf1.inventory;
    -556768665838
    Time taken: 0.232 seconds, Fetched 1 row(s)
    ```
    
    - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests
    
    ) locally before make a pull request
    
    Closes #2449 from pan3793/tpcds.
    
    Closes #2439
    
    a270bcba [Cheng Pan] Remove the exclusion in source release
    7c8d3271 [Cheng Pan] [KYUUBI #2439] Using Pure Java TPC-DS generator
    Authored-by: default avatarCheng Pan <chengpan@apache.org>
    Signed-off-by: default avatarKent Yao <yao@apache.org>
    3ecdd422