Skip to content
GitLab
Switch branch/tag
This project is mirrored from https://gitee.com/yxydde/incubator-kyuubi.git. Pull mirroring failed .
Repository mirroring has been paused due to too many failed attempts. It can be resumed by a project maintainer.
  1. 06 May, 2022 1 commit
  3. 25 Apr, 2022 4 commits
  5. 24 Apr, 2022 3 commits
    • Tianlin Liao's avatar
      [KYUUBI #1936][FOLLOWUP] Stop updating credentials when credentials are expired · 5ae7c9c0
      Tianlin Liao authored 
      ### _Why are the changes needed?_

This is the follow-up for the comments in https://github.com/apache/incubator-kyuubi/pull/2210

------
But considering the following scenario, I think we should also check whether userRef in `userCredentialsRefMap` has been replaced by a new one .

`userRef` was waiting for next renewal.
`userRef` expired and was removed from `userCredentialsRefMap`
A new `userRef` was created because of user visiting.
Renewal of old userRef started again.
If we do not check, both old and new userRef will renew periodically.

modified the code and added test case for the scenario.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2459 from lightning-L/kyuubi-1936.

Closes #1936

0cef07f0 [Tianlin Liao] [KYUUBI #1936][FOLLOWUP] stop updating credentials when credentials are expired
Authored-by: default avatarTianlin Liao <tiliao@ebay.com>
Signed-off-by: default avatarFei Wang <fwang12@ebay.com>
      5ae7c9c0
    • wforget's avatar
      [KYUUBI #2450] Update lastAccessTime in getStatus and add opHandle to opHandleSet before run · 86f016d9
      wforget authored 
      ### _Why are the changes needed?_

close #2450

1. The client obtains the status of the Operation by calling getOperationStatus cyclically. We need to update the lastAccessTime in the getStatus method to prevent timeouts caused by long-running.
2. We need to add opHandle to opHandleSet before running to avoid losing Operation by calling close during Operation running.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2452 from wForget/KYUUBI-2450.

Closes #2450

35002396 [wforget] add opHandle to opHandleSet before run
dc82f65b [wforget] unnecessary change
cca3377c [wforget] [KYUUBI-2450] Update lastAccessTime in getStatus and catch close launchEngineOp exception
Authored-by: default avatarwforget <643348094@qq.com>
Signed-off-by: default avatarFei Wang <fwang12@ebay.com>
      86f016d9
    • Cheng Pan's avatar
      [KYUUBI #2439] Using Pure Java TPC-DS generator · 3ecdd422
      Cheng Pan authored 
      ### _Why are the changes needed?_

This PR proposes change the Kyuubi TPC-DS generator to pure Java implementation instead of the original C binary.

The new pure Java TPC-DS generator is under Apache License, and in fact, I don't know the original C binary License, so we exclude them from release in the past.

Since the change removes the License issue of Kyuubi TPC-DS module, we can bundle the TPC-DS tool in the future release.

And after migration, I haven't see "error=26, Text file busy" described in #2439 any more.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

1. Use old C binary based TPC-DS generator generate 1GB data under database `tpcds_s1`
2. Use new pure Java base TPC-DS generator generate 1GB data under database `new_tpcds_sf1`
3. Compare results of `select count(*)`, and `select sum(hash(*))`

```
spark-sql> select count(*) from tpcds_s1.inventory;
11745000
Time taken: 0.161 seconds, Fetched 1 row(s)
spark-sql> select count(*) from new_tpcds_sf1.inventory;
11745000
Time taken: 0.141 seconds, Fetched 1 row(s)
spark-sql> select sum(hash(*)) from tpcds_s1.inventory;
-556768665838
Time taken: 0.252 seconds, Fetched 1 row(s)
spark-sql> select sum(hash(*)) from new_tpcds_sf1.inventory;
-556768665838
Time taken: 0.232 seconds, Fetched 1 row(s)
```

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2449 from pan3793/tpcds.

Closes #2439

a270bcba [Cheng Pan] Remove the exclusion in source release
7c8d3271 [Cheng Pan] [KYUUBI #2439] Using Pure Java TPC-DS generator
Authored-by: default avatarCheng Pan <chengpan@apache.org>
Signed-off-by: default avatarKent Yao <yao@apache.org>
      3ecdd422
  7. 23 Apr, 2022 1 commit
  9. 22 Apr, 2022 3 commits
  11. 21 Apr, 2022 4 commits
  13. 20 Apr, 2022 4 commits
  15. 19 Apr, 2022 9 commits
  17. 18 Apr, 2022 7 commits
  19. 17 Apr, 2022 2 commits
  21. 16 Apr, 2022 2 commits
    • Fei Wang's avatar
      [KYUUBI #2308][SUB-TASK][KPIP-4] Batch job configuration ignore list and... · 4b42c735
      Fei Wang authored 
      [KYUUBI #2308][SUB-TASK][KPIP-4] Batch job configuration ignore list and pre-defined configuration in server-side

### _Why are the changes needed?_

To close #2308

Support to ignore some batch configuration items and pre-define some configuration in server side.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2386 from turboFei/KYUUBI_2308_kpip4_batch_conf.

Closes #2308

2e517ec4 [Fei Wang] [KYUUBI #2308][SUB-TASK][KPIP-4] Batch job configuration ignore list and server predefined configurations
Authored-by: default avatarFei Wang <fwang12@ebay.com>
Signed-off-by: default avatarFei Wang <fwang12@ebay.com>
      4b42c735
    • Min Zhao's avatar
      [KYUUBI #2369] [Improvement] update developer.md to describe what append... · 1a58aaf7
      Min Zhao authored 
      [KYUUBI #2369] [Improvement] update developer.md to describe what append descriptions of new configurations to settings.md

### _Why are the changes needed?_

update developer.md to describe what append descriptions of new configurations to settings.md

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests

) locally before make a pull request

Closes #2372 from zhaomin1423/2369.

Closes #2369

a044c156 [Min Zhao] [KYUUBI #2369] [Improvement] update developer.md to describe what append descriptions of new configurations to settings.md
56f878a9 [Min Zhao] [KYUUBI #2369] [Improvement] update developer.md to describe what append descriptions of new configurations to settings.md
581894b7 [Min Zhao] [KYUUBI #2369] [Improvement] update developer.md to describe what append descriptions of new configurations to settings.md
Authored-by: default avatarMin Zhao <zhaomin1423@163.com>
Signed-off-by: default avatarKent Yao <yao@apache.org>
      1a58aaf7

免费DevSecOps平台，让您的项目体验完整的DevSecOps流程，让项目更安全