Skip to content
GitLab
Unverified Commit f00a6d53 authored by Oilbeater's avatar Oilbeater Committed by GitHub
Browse files

Merge pull request #116 from alauda/release/v0.8.0 

release v0.8.0
parents 381270d3 6b57f61b
Hide whitespace changes
Inline Side-by-side
Showing
with 200 additions and 24 deletions
+200 -24
CHANGELOG.md
+ 21
- 0
View file @ f00a6d53
# CHANGELOG
## v0.8.0 -- 2019/10/08
### Gateway
* Support active-backup mode centralized gateway high available
### Diagnose Tools
* Kubectl plugin to trace/tcpdump/diagnose pod network traffic
* Pinger to test cluster network quality and expose metrics to Prometheus
### IPAM
* Join subnet ip now can be displayed by `kubectl get ip`
### Security
* Enable port security to prevent Mac and IP spoofing
* Allow nodes to pods traffic for private subnet
### Mics
* Support hostport
* Update OVN/OVS to 2.11.3
* Update Go to 1.13
## v0.7.0 -- 2019/08/21
### IPAM
......
README.md
+ 5
- 3
View file @ f00a6d53
......@@ -19,8 +19,9 @@ Kube-OVN integrates the OVN-based Network Virtualization with Kubernetes. It off
- **Namespaced Gateways**: Every Namespace can have a dedicated Gateway for Egress traffic.
- **Direct External Connectivity**：Pod IP can be exposed to external network directly.
- **Traffic Mirror**: Duplicated container network traffic for monitoring and diagnosing.
- **IPv6 support**: Kube-OVN support ipv6-only mode pod network.
- **Kubectl Plugin**: Handy tools to diagnose container network.
- **IPv6 Support**: Kube-OVN supports ipv6-only mode pod network.
- **TroubleShooting Tools**: Handy tools to diagnose, trace, monitor and dump container network traffic to help troubleshooting complicate network issues.
- **Prometheus Integration**: Exposing network quality metrics like pod/node/service/dns connectivity/latency in Prometheus format.
## Planned Future Work
- Hardware Offloading and DPDK Support
......@@ -48,7 +49,8 @@ If you want to install Kubernetes from scratch, you can try [kubespray](https://
- [Traffic Mirror](docs/mirror.md)
- [Webhook](docs/webhook.md)
- [IPv6](docs/ipv6.md)
- [Kubectl Plugin](docs/kubectl-plugin.md)
- [Tracing/Diagnose/Dump Traffic with Kubectl Plugin](docs/kubectl-plugin.md)
- [Prometheus Integration](docs/pinger.md)
## Kube-OVN vs. Other CNI Implementation
......
VERSION
+ 1
- 1
View file @ f00a6d53
v0.8.0-pre
\ No newline at end of file
v0.8.0
dist/images/kubectl-ko
+ 3
- 2
View file @ f00a6d53
......@@ -16,7 +16,8 @@ showHelp(){
tcpdump(){
namespacedPod="$1"; shift
namespace=$(echo "$namespacedPod" | cut -d "/" -f2)
namespace=$(echo "$namespacedPod" | cut -d "/" -f1)
podName=$(echo "$namespacedPod" | cut -d "/" -f2)
if [ "$podName" = "$namespacedPod" ]; then
nodeName=$(kubectl get pod "$podName" -o jsonpath={.spec.nodeName})
mac=$(kubectl get pod "$podName" -o jsonpath={.metadata.annotations.ovn\\.kubernetes\\.io/mac_address})
......@@ -81,7 +82,7 @@ trace(){
exit 1
fi
gwMac=$(kubectl exec -it ovn-central-8ddc7dd8-fm2c4 -n $KUBE_OVN_NS -- ovn-nbctl --data=bare --no-heading --columns=mac find logical_router_port name=ovn-cluster-"$ls" | tr -d '\r')
gwMac=$(kubectl exec -it $CENTRAL_POD -n $KUBE_OVN_NS -- ovn-nbctl --data=bare --no-heading --columns=mac find logical_router_port name=ovn-cluster-"$ls" | tr -d '\r')
if [ -z "$gwMac" ]; then
echo "get gw mac failed"
......
docs/install.md
+ 6
- 6
View file @ f00a6d53
......@@ -21,19 +21,19 @@ Kube-OVN includes two parts:
`kubectl label node <Node on which to deploy OVN DB> kube-ovn/role=master`
2. Install Kube-OVN related CRDs
`kubectl apply -f https://raw.githubusercontent.com/alauda/kube-ovn/v0.7.0/yamls/crd.yaml`
`kubectl apply -f https://raw.githubusercontent.com/alauda/kube-ovn/v0.8.0/yamls/crd.yaml`
3. Install native OVS and OVN components:
`kubectl apply -f https://raw.githubusercontent.com/alauda/kube-ovn/v0.7.0/yamls/ovn.yaml`
`kubectl apply -f https://raw.githubusercontent.com/alauda/kube-ovn/v0.8.0/yamls/ovn.yaml`
4. Install the Kube-OVN Controller and CNI plugins:
`kubectl apply -f https://raw.githubusercontent.com/alauda/kube-ovn/v0.7.0/yamls/kube-ovn.yaml`
`kubectl apply -f https://raw.githubusercontent.com/alauda/kube-ovn/v0.8.0/yamls/kube-ovn.yaml`
That's all! You can now create some pods and test connectivity.
For high-available ovn db, see [high available](high-available.md)
If you want to enable IPv6 on default subnet and node subnet, please apply https://raw.githubusercontent.com/alauda/kube-ovn/v0.7.0/yamls/kube-ovn-ipv6.yaml on Step 3.
If you want to enable IPv6 on default subnet and node subnet, please apply https://raw.githubusercontent.com/alauda/kube-ovn/v0.8.0/yamls/kube-ovn-ipv6.yaml on Step 3.
## More Configuration
......@@ -79,7 +79,7 @@ If you want to enable IPv6 on default subnet and node subnet, please apply https
1. Remove Kubernetes resources:
```bash
wget https://raw.githubusercontent.com/alauda/kube-ovn/v0.7.0/dist/images/cleanup.sh
wget https://raw.githubusercontent.com/alauda/kube-ovn/v0.8.0/dist/images/cleanup.sh
bash cleanup.sh
```
......@@ -91,4 +91,4 @@ If you want to enable IPv6 on default subnet and node subnet, please apply https
rm -rf /etc/openvswitch
rm -rf /etc/cni/net.d/00-kube-ovn.conflist
```
3. Reboot the Node to remove ipset/iptables rules and nics.
\ No newline at end of file
3. Reboot the Node to remove ipset/iptables rules and nics.
docs/ipv6.md
+ 1
- 1
View file @ f00a6d53
......@@ -2,4 +2,4 @@
Through Kube-OVN does support both protocol subnets coexist in a cluster, Kubernetes control plan now only support one protocol. So you will lost some ability like probe and service discovery if you use a protocol other than the kubernetes control plan. We recommend you use only one same ip protocol that same with kubernetes control plan.
To enable IPv6 support you need to modify the installation yaml to specify the default subnet and node subnet cidrBlock and gateway with a ipv6 format. You can apply this [v6 version yaml](https://raw.githubusercontent.com/alauda/kube-ovn/v0.7.0/yamls/kube-ovn-ipv6.yaml) at [installation step 3](install.md#to-install) for a quick start.
\ No newline at end of file
To enable IPv6 support you need to modify the installation yaml to specify the default subnet and node subnet cidrBlock and gateway with a ipv6 format. You can apply this [v6 version yaml](https://raw.githubusercontent.com/alauda/kube-ovn/v0.8.0/yamls/kube-ovn-ipv6.yaml) at [installation step 3](install.md#to-install) for a quick start.
docs/kubectl-plugin.md
+ 133
- 0
View file @ f00a6d53
......@@ -40,3 +40,136 @@ Available Subcommands:
trace {namespace/podname} {target ip address} {icmp|tcp|udp} [target tcp or udp port]
diagnose {all|node} [nodename] diagnose connectivity of all nodes or a specific node
```
1. Show ovn-sb overview
```bash
[root@node2 ~]# kubectl ko sbctl show
Chassis "36f129a9-276f-4d96-964b-7d3703001b81"
hostname: "node1.cluster.local"
Encap geneve
ip: "10.0.129.96"
options: {csum="true"}
Port_Binding "tiller-deploy-849b7c6496-5l9r6.kube-system"
Port_Binding "kube-ovn-pinger-5mq4g.kube-ovn"
Port_Binding "nginx-6b4b85b77b-rk9tq.acl"
Port_Binding "node-node1"
Port_Binding "piquant-magpie-nginx-ingress-default-backend-84776f949b-jthhh.kube-system"
Port_Binding "ds1-l6n7p.default"
Chassis "9ced77f4-dae4-4e0b-b3fe-15dd82104e67"
hostname: "node2.cluster.local"
Encap geneve
ip: "10.0.128.15"
options: {csum="true"}
Port_Binding "ds1-wqpdz.default"
Port_Binding "node-node2"
Port_Binding "kube-ovn-pinger-8xhhv.kube-ovn"
Chassis "dc922a96-97d4-418d-a45f-8989d2b6dc91"
hostname: "node3.cluster.local"
Encap geneve
ip: "10.0.128.35"
options: {csum="true"}
Port_Binding "ds1-dflpx.default"
Port_Binding "coredns-585c7897d4-59xkc.kube-system"
Port_Binding "node-node3"
Port_Binding "kube-ovn-pinger-gc8l6.kube-ovn"
Port_Binding "coredns-585c7897d4-7dglw.kube-system"
```
2. Dump pod ICMP traffic
```bash
[root@node2 ~]# kubectl ko tcpdump default/ds1-l6n7p icmp
+ kubectl exec -it kube-ovn-cni-wlg4s -n kube-ovn -- tcpdump -nn -i d7176fe7b4e0_h icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on d7176fe7b4e0_h, link-type EN10MB (Ethernet), capture size 262144 bytes
06:52:36.619688 IP 100.64.0.3 > 10.16.0.4: ICMP echo request, id 2, seq 1, length 64
06:52:36.619746 IP 10.16.0.4 > 100.64.0.3: ICMP echo reply, id 2, seq 1, length 64
06:52:37.619588 IP 100.64.0.3 > 10.16.0.4: ICMP echo request, id 2, seq 2, length 64
06:52:37.619630 IP 10.16.0.4 > 100.64.0.3: ICMP echo reply, id 2, seq 2, length 64
06:52:38.619933 IP 100.64.0.3 > 10.16.0.4: ICMP echo request, id 2, seq 3, length 64
06:52:38.619973 IP 10.16.0.4 > 100.64.0.3: ICMP echo reply, id 2, seq 3, length 64
```
3. Show ovn flow from a pod to a destination
```bash
[root@node2 ~]# kubectl ko trace default/ds1-l6n7p 8.8.8.8 icmp
+ kubectl exec ovn-central-5bc494cb5-np9hm -n kube-ovn -- ovn-trace --ct=new ovn-default 'inport == "ds1-l6n7p.default" && ip.ttl == 64 && icmp && eth.src == 0a:00:00:10:00:05 && ip4.src == 10.16.0.4 && eth.dst == 00:00:00:B8:CA:43 && ip4.dst == 8.8.8.8'
# icmp,reg14=0xf,vlan_tci=0x0000,dl_src=0a:00:00:10:00:05,dl_dst=00:00:00:b8:ca:43,nw_src=10.16.0.4,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0
ingress(dp="ovn-default", inport="ds1-l6n7p.default")
-----------------------------------------------------
0. ls_in_port_sec_l2 (ovn-northd.c:4143): inport == "ds1-l6n7p.default" && eth.src == {0a:00:00:10:00:05}, priority 50, uuid 39453393
next;
1. ls_in_port_sec_ip (ovn-northd.c:2898): inport == "ds1-l6n7p.default" && eth.src == 0a:00:00:10:00:05 && ip4.src == {10.16.0.4}, priority 90, uuid 81bcd485
next;
3. ls_in_pre_acl (ovn-northd.c:3269): ip, priority 100, uuid 7b4f4971
reg0[0] = 1;
next;
5. ls_in_pre_stateful (ovn-northd.c:3396): reg0[0] == 1, priority 100, uuid 36cdd577
ct_next;
ct_next(ct_state=new|trk)
-------------------------
6. ls_in_acl (ovn-northd.c:3759): ip && (!ct.est || (ct.est && ct_label.blocked == 1)), priority 1, uuid 7608af5b
reg0[1] = 1;
next;
10. ls_in_stateful (ovn-northd.c:3995): reg0[1] == 1, priority 100, uuid 2aba1b90
ct_commit(ct_label=0/0x1);
next;
16. ls_in_l2_lkup (ovn-northd.c:4470): eth.dst == 00:00:00:b8:ca:43, priority 50, uuid 5c9c3c9f
outport = "ovn-default-ovn-cluster";
output;
....Skip More....
```
4. Diagnose network connectivity
```bash
[root@node2 ~]# kubectl ko diagnose all
### start to diagnose node node1
I1008 07:04:40.475604 26434 ping.go:139] ovs-vswitchd and ovsdb are up
I1008 07:04:40.570824 26434 ping.go:151] ovn_controller is up
I1008 07:04:40.570859 26434 ping.go:35] start to check node connectivity
I1008 07:04:44.586096 26434 ping.go:57] ping node: node1 10.0.129.96, count: 5, loss rate 0.00%, average rtt 0.23ms
I1008 07:04:44.592764 26434 ping.go:57] ping node: node3 10.0.128.35, count: 5, loss rate 0.00%, average rtt 0.63ms
I1008 07:04:44.592791 26434 ping.go:57] ping node: node2 10.0.128.15, count: 5, loss rate 0.00%, average rtt 0.54ms
I1008 07:04:44.592889 26434 ping.go:74] start to check pod connectivity
I1008 07:04:48.669057 26434 ping.go:101] ping pod: kube-ovn-pinger-5mq4g 10.16.0.12, count: 5, loss rate 0.00, average rtt 0.18ms
I1008 07:04:48.769217 26434 ping.go:101] ping pod: kube-ovn-pinger-8xhhv 10.16.0.10, count: 5, loss rate 0.00, average rtt 0.64ms
I1008 07:04:48.769219 26434 ping.go:101] ping pod: kube-ovn-pinger-gc8l6 10.16.0.13, count: 5, loss rate 0.00, average rtt 0.73ms
I1008 07:04:48.769325 26434 ping.go:119] start to check dns connectivity
I1008 07:04:48.777062 26434 ping.go:129] resolve dns kubernetes.default.svc.cluster.local to [10.96.0.1] in 7.71ms
### finish diagnose node node1
### start to diagnose node node2
I1008 07:04:49.231462 16925 ping.go:139] ovs-vswitchd and ovsdb are up
I1008 07:04:49.241636 16925 ping.go:151] ovn_controller is up
I1008 07:04:49.241694 16925 ping.go:35] start to check node connectivity
I1008 07:04:53.254327 16925 ping.go:57] ping node: node2 10.0.128.15, count: 5, loss rate 0.00%, average rtt 0.16ms
I1008 07:04:53.354411 16925 ping.go:57] ping node: node1 10.0.129.96, count: 5, loss rate 0.00%, average rtt 15.65ms
I1008 07:04:53.354464 16925 ping.go:57] ping node: node3 10.0.128.35, count: 5, loss rate 0.00%, average rtt 15.71ms
I1008 07:04:53.354492 16925 ping.go:74] start to check pod connectivity
I1008 07:04:57.382791 16925 ping.go:101] ping pod: kube-ovn-pinger-8xhhv 10.16.0.10, count: 5, loss rate 0.00, average rtt 0.16ms
I1008 07:04:57.483725 16925 ping.go:101] ping pod: kube-ovn-pinger-5mq4g 10.16.0.12, count: 5, loss rate 0.00, average rtt 1.74ms
I1008 07:04:57.483750 16925 ping.go:101] ping pod: kube-ovn-pinger-gc8l6 10.16.0.13, count: 5, loss rate 0.00, average rtt 1.81ms
I1008 07:04:57.483813 16925 ping.go:119] start to check dns connectivity
I1008 07:04:57.490402 16925 ping.go:129] resolve dns kubernetes.default.svc.cluster.local to [10.96.0.1] in 6.56ms
### finish diagnose node node2
### start to diagnose node node3
I1008 07:04:58.094738 21692 ping.go:139] ovs-vswitchd and ovsdb are up
I1008 07:04:58.176064 21692 ping.go:151] ovn_controller is up
I1008 07:04:58.176096 21692 ping.go:35] start to check node connectivity
I1008 07:05:02.193091 21692 ping.go:57] ping node: node3 10.0.128.35, count: 5, loss rate 0.00%, average rtt 0.21ms
I1008 07:05:02.293256 21692 ping.go:57] ping node: node2 10.0.128.15, count: 5, loss rate 0.00%, average rtt 0.58ms
I1008 07:05:02.293256 21692 ping.go:57] ping node: node1 10.0.129.96, count: 5, loss rate 0.00%, average rtt 0.68ms
I1008 07:05:02.293368 21692 ping.go:74] start to check pod connectivity
I1008 07:05:06.314977 21692 ping.go:101] ping pod: kube-ovn-pinger-gc8l6 10.16.0.13, count: 5, loss rate 0.00, average rtt 0.37ms
I1008 07:05:06.415222 21692 ping.go:101] ping pod: kube-ovn-pinger-5mq4g 10.16.0.12, count: 5, loss rate 0.00, average rtt 0.82ms
I1008 07:05:06.415317 21692 ping.go:101] ping pod: kube-ovn-pinger-8xhhv 10.16.0.10, count: 5, loss rate 0.00, average rtt 0.64ms
I1008 07:05:06.415354 21692 ping.go:119] start to check dns connectivity
I1008 07:05:06.420595 21692 ping.go:129] resolve dns kubernetes.default.svc.cluster.local to [10.96.0.1] in 5.21ms
### finish diagnose node node3
```
docs/pinger.md 0 → 100644
+ 19
- 0
View file @ f00a6d53
Pinger makes network requests between pods/nodes/services/dns to test the connectivity in the cluster and expose metrics in Prometheus format.
## Prometheus Integration
Pinger exposes metrics at `:8080/metrics`, it will show following metrics
```bash
pinger_ovs_up
pinger_ovs_down
pinger_ovn_controller_up
pinger_ovn_controller_down
pinger_dns_healthy
pinger_dns_unhealthy
pinger_dns_latency_ms
pinger_pod_ping_latency_ms
pinger_pod_ping_lost_total
pinger_node_ping_latency_ms
pinger_node_ping_lost_total
```
yamls/kube-ovn-ipv6.yaml
+ 4
- 4
View file @ f00a6d53
......@@ -38,7 +38,7 @@ spec:
hostNetwork: true
containers:
- name: kube-ovn-controller
image: "index.alauda.cn/alaudak8s/kube-ovn-controller:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-controller:v0.8.0"
imagePullPolicy: Always
command:
- /kube-ovn/start-controller.sh
......@@ -109,7 +109,7 @@ spec:
hostPID: true
initContainers:
- name: install-cni
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0"
imagePullPolicy: Always
command: ["/kube-ovn/install-cni.sh"]
volumeMounts:
......@@ -119,7 +119,7 @@ spec:
name: cni-bin
containers:
- name: cni-server
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0"
command: ["sh", "/kube-ovn/start-cniserver.sh"]
args:
- --enable-mirror=false
......@@ -202,7 +202,7 @@ spec:
hostPID: true
containers:
- name: pinger
image: "index.alauda.cn/alaudak8s/kube-ovn-pinger:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-pinger:v0.8.0"
imagePullPolicy: Always
securityContext:
runAsUser: 0
......
yamls/kube-ovn.yaml
+ 4
- 4
View file @ f00a6d53
......@@ -38,7 +38,7 @@ spec:
hostNetwork: true
containers:
- name: kube-ovn-controller
image: "index.alauda.cn/alaudak8s/kube-ovn-controller:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-controller:v0.8.0"
imagePullPolicy: Always
command:
- /kube-ovn/start-controller.sh
......@@ -112,7 +112,7 @@ spec:
hostPID: true
initContainers:
- name: install-cni
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0"
imagePullPolicy: Always
command: ["/kube-ovn/install-cni.sh"]
volumeMounts:
......@@ -122,7 +122,7 @@ spec:
name: cni-bin
containers:
- name: cni-server
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-cni:v0.8.0"
imagePullPolicy: Always
command:
- sh
......@@ -206,7 +206,7 @@ spec:
hostPID: true
containers:
- name: pinger
image: "index.alauda.cn/alaudak8s/kube-ovn-pinger:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-pinger:v0.8.0"
imagePullPolicy: Always
securityContext:
runAsUser: 0
......
yamls/ovn.yaml
+ 2
- 2
View file @ f00a6d53
......@@ -158,7 +158,7 @@ spec:
hostNetwork: true
containers:
- name: ovn-central
image: "index.alauda.cn/alaudak8s/kube-ovn-db:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-db:v0.8.0"
imagePullPolicy: Always
env:
- name: POD_IP
......@@ -245,7 +245,7 @@ spec:
hostPID: true
containers:
- name: openvswitch
image: "index.alauda.cn/alaudak8s/kube-ovn-node:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-node:v0.8.0"
imagePullPolicy: Always
securityContext:
runAsUser: 0
......
yamls/webhook.yaml
+ 1
- 1
View file @ f00a6d53
......@@ -38,7 +38,7 @@ spec:
hostNetwork: true
containers:
- name: kube-ovn-webhook
image: "index.alauda.cn/alaudak8s/kube-ovn-webhook:v0.8.0-pre"
image: "index.alauda.cn/alaudak8s/kube-ovn-webhook:v0.8.0"
imagePullPolicy: Always
command:
- /kube-ovn/start-webhook.sh
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

免费DevSecOps平台，让您的项目体验完整的DevSecOps流程，让项目更安全