Skip to content
GitLab
Switch branch/tag
This project is mirrored from https://gitee.com/yxydde/incubator-kyuubi.git. Pull mirroring failed .
Repository mirroring has been paused due to too many failed attempts. It can be resumed by a project maintainer.
  1. 24 Apr, 2022 1 commit
    • Cheng Pan's avatar
      [KYUUBI #2439] Using Pure Java TPC-DS generator · 3ecdd422
      Cheng Pan authored 
      ### _Why are the changes needed?_

This PR proposes change the Kyuubi TPC-DS generator to pure Java implementation instead of the original C binary.

The new pure Java TPC-DS generator is under Apache License, and in fact, I don't know the original C binary License, so we exclude them from release in the past.

Since the change removes the License issue of Kyuubi TPC-DS module, we can bundle the TPC-DS tool in the future release.

And after migration, I haven't see "error=26, Text file busy" described in #2439 any more.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

1. Use old C binary based TPC-DS generator generate 1GB data under database `tpcds_s1`
2. Use new pure Java base TPC-DS generator generate 1GB data under database `new_tpcds_sf1`
3. Compare results of `select count(*)`, and `select sum(hash(*))`

```
spark-sql> select count(*) from tpcds_s1.inventory;
11745000
Time taken: 0.161 seconds, Fetched 1 row(s)
spark-sql> select count(*) from new_tpcds_sf1.inventory;
11745000
Time taken: 0.141 seconds, Fetched 1 row(s)
spark-sql> select sum(hash(*)) from tpcds_s1.inventory;
-556768665838
Time taken: 0.252 seconds, Fetched 1 row(s)
spark-sql> select sum(hash(*)) from new_tpcds_sf1.inventory;
-556768665838
Time taken: 0.232 seconds, Fetched 1 row(s)
```

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2449 from pan3793/tpcds.

Closes #2439

a270bcba [Cheng Pan] Remove the exclusion in source release
7c8d3271 [Cheng Pan] [KYUUBI #2439] Using Pure Java TPC-DS generator
Authored-by: default avatarCheng Pan <chengpan@apache.org>
Signed-off-by: default avatarKent Yao <yao@apache.org>
      3ecdd422
  3. 23 Apr, 2022 1 commit
  5. 22 Apr, 2022 3 commits
  7. 21 Apr, 2022 4 commits
  9. 20 Apr, 2022 4 commits
  11. 19 Apr, 2022 9 commits
  13. 18 Apr, 2022 7 commits
  15. 17 Apr, 2022 2 commits
  17. 16 Apr, 2022 5 commits
  19. 15 Apr, 2022 4 commits
    • Martin Tzvetanov Grigorov's avatar
      [KYUUBI #2385] Export JAVA_HOME. It seems TravisCI stopped doing it recently · b8d04af4
      Martin Tzvetanov Grigorov authored 
      ### _Why are the changes needed?_

Many builds at TravisCI recently failed due to error like
```
- hive process builder *** FAILED ***

  org.apache.kyuubi.KyuubiSQLException: JAVA_HOME is not set! For more information on installing and configuring JAVA_HOME, please visit https://kyuubi.apache.org/docs/latest/deployment


```

It seems TravisCI stopped exporting JAVA_HOME ...

### _How was this patch tested?_

TravisCI build should pass

Closes #2385 from martin-g/manually-export-java-home-travis.

Closes #2385

5c2a3f62 [Martin Tzvetanov Grigorov] The correct path for the JDK 8 installation is /usr/lib/jvm/adoptopenjdk-8-hotspot-arm64
78bac049 [Martin Tzvetanov Grigorov] Export JAVA_HOME. It seems TravisCI stopped doing it recently
Authored-by: default avatarMartin Tzvetanov Grigorov <mgrigorov@apache.org>
Signed-off-by: default avatarFei Wang <fwang12@ebay.com>
      b8d04af4
    • Fei Wang's avatar
      [KYUUBI #2353] [SUB-TASK][KPIP-4] Implement BatchJobSubmission operation and... · b6d5c64c
      Fei Wang authored 
      [KYUUBI #2353] [SUB-TASK][KPIP-4] Implement BatchJobSubmission operation and basic KyuubiBatchSessionImpl

### _Why are the changes needed?_

To close #2306 and close #2307

In this PR, I implement BatchJobSubmission operation and  introduce basic `KyuubiBatchSessionImpl`.
TODO:
- Normalize/validate the batch request
  -  batch request fields
  - merge with server pre-defined batch conf

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2353 from turboFei/KPIP_4_batch_submission_op.

Closes #2353

9bc6050c [Fei Wang] remove unused conf
ef8e962c [Fei Wang] check application in current thread
8738660b [Fei Wang] dedup code
27f22008 [Fei Wang] use static secret id instead of conf
6794ff7a [Fei Wang] Use Seq instead of java.util.List
2f4f9b15 [Fei Wang] Remove BatchType enumaration
7d380800 [Fei Wang] remove dead code
a94a9e6a [Fei Wang] remove jars,files fileds on BatchRequest
6021a1e7 [Fei Wang] add ut for result set
07a939c9 [Fei Wang] refactor long line
a918a496 [Fei Wang] address comments
73229e70 [Fei Wang] set engine max life time
bbe3f1f4 [Fei Wang] unique the application cehcker thread pool
9643e42c [Fei Wang] refactor
11dd71f7 [Fei Wang] add KyuubiBatchYarnClusterSuite
12169910 [Fei Wang] add ut for batch session
47da8c1a [Fei Wang] add open batch session api
6dcf60d9 [Fei Wang] add ut for static batch secret id
a212e62b [Fei Wang] [SUB-TASK][KPIP-4] Implement BatchJobSubmission operation and basic KyuubiBatchSessionImpl
Authored-by: default avatarFei Wang <fwang12@ebay.com>
Signed-off-by: default avatarFei Wang <fwang12@ebay.com>
      b6d5c64c
    • zwangsheng's avatar
      [KYUUBI #2359] [Test] Build WithKyuubiServerOnKuberntes · 65a272f6
      zwangsheng authored 
      ### _Why are the changes needed?_

Refactor KyuubiOnKubernetesTestsSuite
Build WithKyuubiServerOnKuberntes like WithKyuubiServerOnYarn

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2359 from zwangsheng/test/kyuubi_on_kubernetes.

Closes #2359

c1048c6a [zwangsheng] fix
81c98cd2 [zwangsheng] unused import
976f9091 [zwangsheng] fix
bad6720b [zwangsheng] fix
98d8995c [zwangsheng] refactor
628de557 [zwangsheng] style check
9dc6675a [zwangsheng] Build WithKyuubiServerOnKubernetes
Authored-by: default avatarzwangsheng <2213335496@qq.com>
Signed-off-by: default avatarulysses-you <ulyssesyou@apache.org>
      65a272f6
    • Min Zhao's avatar
      [KYUUBI #2330] [Subtask] Hive Backend Engine - GetTypeInfo Operation · 6d147894
      Min Zhao authored 
      Support GetTypeInfo Operation for hive engine.
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2366 from zhaomin1423/2330.

Closes #2330

fc67e0e9 [Min Zhao] [KYUUBI #2330] [Subtask] Hive Backend Engine - GetTypeInfo Operation
Authored-by: default avatarMin Zhao <zhaomin1423@163.com>
Signed-off-by: default avatarKent Yao <yao@apache.org>
      6d147894

免费DevSecOps平台，让您的项目体验完整的DevSecOps流程，让项目更安全