Unverified Commit fa49483c authored by Yujie Xia's avatar Yujie Xia Committed by GitHub
Browse files

parser: update doc links to tidb repo (#31944)

ref pingcap/tidb#28257
parent 373f0414
Showing with 20 additions and 60 deletions
+20 -60
......@@ -7,13 +7,13 @@
The goal of this project is to build a Golang parser that is fully compatible with MySQL syntax, easy to extend, and high performance. Currently, features supported by parser are as follows:
- Highly compatible with MySQL: it supports almost all features of MySQL. For the complete details, see [parser.y](https://github.com/pingcap/parser/blob/master/parser.y) and [hintparser.y](https://github.com/pingcap/parser/blob/master/hintparser.y).
- Highly compatible with MySQL: it supports almost all features of MySQL. For the complete details, see [parser.y](https://github.com/pingcap/tidb/blob/master/parser/parser.y) and [hintparser.y](https://github.com/pingcap/tidb/blob/master/parser/hintparser.y).
- Extensible: adding a new syntax requires only a few lines of Yacc and Golang code changes. As an example, see [PR-680](https://github.com/pingcap/parser/pull/680/files).
- Good performance: the parser is generated by goyacc in a bottom-up approach. It is efficient to build an AST tree with a state machine.
## How to use it
Please read the [quickstart](https://github.com/pingcap/parser/blob/master/docs/quickstart.md).
Please read the [quickstart](https://github.com/pingcap/tidb/blob/master/parser/docs/quickstart.md).
## Future
......@@ -24,7 +24,7 @@ Please read the [quickstart](https://github.com/pingcap/parser/blob/master/docs/
## Getting Help
- [GitHub Issue](https://github.com/pingcap/parser/issues)
- [GitHub Issue](https://github.com/pingcap/tidb/issues)
- [Stack Overflow](https://stackoverflow.com/questions/tagged/tidb)
- [User Group (Chinese)](https://asktug.com/)
......@@ -50,8 +50,6 @@ found you are one of the users but not listed here:
Contributions are welcomed and greatly appreciated. See [Contribution Guide](https://github.com/pingcap/community/blob/master/contributors/README.md) for details on submitting patches and the contribution workflow.
Here is how to [update parser for TiDB](https://github.com/pingcap/parser/blob/master/docs/update-parser-for-tidb.md).
## Acknowledgments
Thanks [cznic](https://github.com/cznic) for providing some great open-source tools.
......
......@@ -17,18 +17,20 @@ go mod init colx && touch main.go
## Import Dependencies
First of all, you need to use `go get` to fetch the dependencies through git hash. The git hashes are available in [release page](https://github.com/pingcap/parser/releases). Take `v4.0.2` as an example:
First, you need to use `go get` to fetch the dependencies through git hash. The git hashes are available in [release page](https://github.com/pingcap/tidb/releases). Take `v5.3.0` as an example:
```bash
go get -v github.com/pingcap/parser@3a18f1e
go get -v github.com/pingcap/tidb/parser@4a1b2e9
```
> **NOTE**
>
> The parser was merged into TiDB repo since v5.3.0. So you can only choose version v5.3.0 or higher in this TiDB repo.
>
> You may want to use advanced API on expressions (a kind of AST node), such as numbers, string literals, booleans, nulls, etc. It is strongly recommended to use the `types` package in TiDB repo with the following command:
> You may want to use advanced API on expressions (a kind of AST node), such as numbers, string literals, booleans, nulls, etc. It is strongly recommended using the `types` package in TiDB repo with the following command:
>
> ```bash
> go get -v github.com/pingcap/tidb/types/parser_driver@328b6d0
> go get -v github.com/pingcap/tidb/types/parser_driver@4a1b2e9
> ```
> and import it in your golang source code:
> ```go
......@@ -48,17 +50,18 @@ Now, open `main.go` with your favorite editor, and start coding!
## Parse SQL text
To convert a SQL text to an AST tree, you need to:
1. Use the [`parser.New()`](https://pkg.go.dev/github.com/pingcap/parser?tab=doc#New) function to instantiate a parser, and
2. Invoke the method [`Parse(sql, charset, collation)`](https://pkg.go.dev/github.com/pingcap/parser?tab=doc#Parser.Parse) on the parser.
1. Use the [`parser.New()`](https://pkg.go.dev/github.com/pingcap/tidb/parser?tab=doc#New) function to instantiate a parser, and
2. Invoke the method [`Parse(sql, charset, collation)`](https://pkg.go.dev/github.com/pingcap/tidb/parser?tab=doc#Parser.Parse) on the parser.
```go
package main
import (
"fmt"
"github.com/pingcap/parser"
"github.com/pingcap/parser/ast"
_ "github.com/pingcap/parser/test_driver"
"github.com/pingcap/tidb/parser"
"github.com/pingcap/tidb/parser/ast"
_ "github.com/pingcap/tidb/parser/test_driver"
)
func parse(sql string) (*ast.StmtNode, error) {
......@@ -80,6 +83,7 @@ func main() {
}
fmt.Printf("%v\n", *astNode)
}
```
Test the parser by running the following command:
......@@ -99,19 +103,19 @@ If the parser runs properly, you should get a result like this:
> Here are a few things you might want to know:
> - To use a parser, a `parser_driver` is required. It decides how to parse the basic data types in SQL.
>
> You can use [`github.com/pingcap/parser/test_driver`](https://pkg.go.dev/github.com/pingcap/parser/test_driver) as the `parser_driver` for test. Again, if you need advanced features, please use the `parser_driver` in TiDB (run `go get -v github.com/pingcap/tidb/types/parser_driver@328b6d0` and import it).
> You can use [`github.com/pingcap/tidb/parser/test_driver`](https://pkg.go.dev/github.com/pingcap/tidb/parser/test_driver) as the `parser_driver` for test. Again, if you need advanced features, please use the `parser_driver` in TiDB (run `go get -v github.com/pingcap/tidb/types/parser_driver@4a1b2e9` and import it).
> - The instantiated parser object is not goroutine safe. It is better to keep it in a single goroutine.
> - The instantiated parser object is not lightweight. It is better to reuse it if possible.
> - The 2nd and 3rd arguments of [`parser.Parse()`](https://pkg.go.dev/github.com/pingcap/parser?tab=doc#Parser.Parse) are charset and collation respectively. If you pass an empty string into it, a default value is chosen.
> - The 2nd and 3rd arguments of [`parser.Parse()`](https://pkg.go.dev/github.com/pingcap/tidb/parser?tab=doc#Parser.Parse) are charset and collation respectively. If you pass an empty string into it, a default value is chosen.
## Traverse AST Nodes
Now you get the AST tree root of a SQL statement. It is time to extract the column names by traverse.
Parser implements the interface [`ast.Node`](https://pkg.go.dev/github.com/pingcap/parser/ast?tab=doc#Node) for each kind of AST node, such as SelectStmt, TableName, ColumnName. [`ast.Node`](https://pkg.go.dev/github.com/pingcap/parser/ast?tab=doc#Node) provides a method `Accept(v Visitor) (node Node, ok bool)` to allow any struct that has implemented [`ast.Visitor`](https://pkg.go.dev/github.com/pingcap/parser/ast?tab=doc#Visitor) to traverse itself.
Parser implements the interface [`ast.Node`](https://pkg.go.dev/github.com/pingcap/tidb/parser/ast?tab=doc#Node) for each kind of AST node, such as SelectStmt, TableName, ColumnName. [`ast.Node`](https://pkg.go.dev/github.com/pingcap/tidb/parser/ast?tab=doc#Node) provides a method `Accept(v Visitor) (node Node, ok bool)` to allow any struct that has implemented [`ast.Visitor`](https://pkg.go.dev/github.com/pingcap/tidb/parser/ast?tab=doc#Visitor) to traverse itself.
[`ast.Visitor`](https://pkg.go.dev/github.com/pingcap/parser/ast?tab=doc#Visitor) is defined as follows:
[`ast.Visitor`](https://pkg.go.dev/github.com/pingcap/tidb/parser/ast?tab=doc#Visitor) is defined as follows:
```go
type Visitor interface {
Enter(n Node) (node Node, skipChildren bool)
......
# How to update parser for TiDB
Assuming that you want to file a PR (pull request) to TiDB, and your PR includes a change in the parser, follow these steps to update the parser in TiDB.
## Step 1: Make changes in your parser repository
Fork this repository to your own account and commit the changes to your repository.
> **Note:**
>
> - Don't forget to run `make test` before you commit!
> - Make sure `parser.go` is updated.
Suppose the forked repository is `https://github.com/your-repo/parser`.
## Step 2: Make your parser changes take effect in TiDB and run CI
1. In your TiDB repository, execute the `replace` instruction to make your parser changes take effect:
```
GO111MODULE=on go mod edit -replace github.com/pingcap/parser=github.com/your-repo/parser@your-branch
```
2. `make dev` to run CI in TiDB.
3. File a PR to TiDB.
## Step 3: Merge the PR about the parser to this repository
File a PR to this repository. **Link the related PR in TiDB in your PR description or comment.**
This PR will be reviewed, and if everything goes well, it will be merged.
## Step 4: Update TiDB to use the latest parser
In your TiDB pull request, modify the `go.mod` file manually or use this command:
```
GO111MODULE=on go get -u github.com/pingcap/parser@master
```
Make sure the `replace` instruction is changed back to the `require` instruction and the version is the latest.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment