Summary: TSIA. This was printed by bazel debug logs.

Test Plan: Exsiting

Reviewers: zasgar, oazizi, #third_party_approvers

Reviewed By: zasgar, #third_party_approvers

Differential Revision: https://phab.corp.pixielabs.ai/D9848

GitOrigin-RevId: d4880419730285227bc44576179a6977021c7bc5

Summary:
As part of the renaming, ExtractGolangInterfaces() is moved to go_syms.{h,cc}
This cleanup is to make it easier to discover where ElfReader locates.

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9840

GitOrigin-RevId: 83d2f95d2e1a0ba4f000ec7d81b216a925346fa9

Summary:
In preparation for web workers, which rely on some bug fixes in Webpack and some features in ES2020.
- Webpack: version 5.51.2 fixes [a bug](https://github.com/webpack/webpack/releases/tag/v5.51.2) that prevents web worker code from building properly.
- Axios: We updated to 0.21.4 [two weeks ago](https://github.com/pixie-io/pixie/commit/c46d717baca758d72470f15f6ce2f6b3a819ccc9) for a security advisory, but forgot to update `package.json` to lock it in.
- `tsconfig.json`: `es2020` includes several features that we'll need for web workers.
- `jest.config.js`, `webpack.config.js`: Match `es2020` configuration.
- NodeJS: We already expect Node 16 everywhere else and install it in our Chef configuration. It's the currently supported release, and becomes LTS next month. So, just lining up here.

Test Plan: `yarn test`. `yarn dev`; smoke test scripts, admin pages, and so on for anything that behaves differently from normal. It should behave the same as before. If you aren't already running Node 16, install it through your preferred method (tested with a bare installation and with `nvm use 16` on my system).

Reviewers: michelle, vihang, #third_party_approvers

Reviewed By: vihang, #third_party_approvers

JIRA Issues: PP-3039

Differential Revision: https://phab.corp.pixielabs.ai/D9844

GitOrigin-RevId: 9ffb7fe6f181974524fe5213da3284aca848de65

Test Plan: Jenkins

Reviewers: #stirling, jps

Reviewed By: #stirling, jps

Subscribers: jps

Differential Revision: https://phab.corp.pixielabs.ai/D9831

GitOrigin-RevId: 2f278a50ddc24ec416aa16cafdd844a8bee8268c

Summary:
Request Graph evolved over time, and in the meantime some cruft has built up. This diff addresses the following issues:

1. In clustered mode, previously we would actually select one of the pod->pod percentiles randomly that applied to that service->service latency. This is fine when there is one pod per service, but incorrect when there are multiple pods communicating per service.
2. We would show <unknown service> instead of the information that we do have about a particular edge/node. Instead, let's show IP address (PxL updates to come next -- but this works without the PxL updates so it's not a breaking change) for inbound/outbound traffic that we can't attribute to a particular pod/service.
3. Move some of the pure logic outside of the React component in order to make the React component a bit easier to understand.

Next up will be updating the PxL script to produce IP addresses.

Test Plan: ran the UI with various options being toggled.

Reviewers: michelle, nlanam, zasgar

Reviewed By: nlanam

Subscribers: oazizi

JIRA Issues: PC-1190

Differential Revision: https://phab.corp.pixielabs.ai/D9834

GitOrigin-RevId: b6a3cc176a75b91b54fb84715e15ab0243b98dad

Summary:
https://github.com/bazelbuild/rules_docker/pull/1831 made it into
a release so we can stop patching.

Test Plan: build and test all

Reviewers: zasgar, #third_party_approvers

Reviewed By: zasgar, #third_party_approvers

Differential Revision: https://phab.corp.pixielabs.ai/D9826

GitOrigin-RevId: 50c3b0705aa8039ed57b1b9f39bfea909bb48372

Summary: Address a TODO, make BPFtrace a member, add FieldAnalyser to be consistent with upstream usage model.

Test Plan: Existing tests

Reviewers: #stirling, yzhao

Reviewed By: #stirling, yzhao

Subscribers: yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D9823

GitOrigin-RevId: e2131a04bc948613d4380066123cd692bda80417

Summary: this is now in the operator, so we dont need the duplicated logic

Test Plan: n/a

Reviewers: nserrino, vihang, philkuz

Reviewed By: vihang

Differential Revision: https://phab.corp.pixielabs.ai/D9842

GitOrigin-RevId: 88f4b90f7e7bb7c700927ba1cde8bb9930cdf12b

Summary: our images only run on linux nodes. rather than requiring users with mixed windows/linux envs to manually patch their YAMLs with nodeselectors, we should just select linux nodes by default

Test Plan: created vizier rc with affinity, deployed to gke cluster, test eks cluster. also created operator rc and verified it starts up on GKE

Reviewers: philkuz, vihang, zasgar

Reviewed By: vihang

Differential Revision: https://phab.corp.pixielabs.ai/D9839

GitOrigin-RevId: ae98b1e17c0f8ffccfd3ee67fde96547ecf1a20d

Summary:
we have a run into a case where a cluster gets mistakenly classified as having no default storage class, when it actually does.
it turns out that some older clusters are still using `storage.k8s.io/v1beta1` which has a slightly different annotation.

Test Plan: n/a

Reviewers: philkuz, vihang, nserrino

Reviewed By: vihang

Differential Revision: https://phab.corp.pixielabs.ai/D9837

GitOrigin-RevId: 35815917ee799bb5fb0d461e0f2e29eb1af6d7ad

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9828

GitOrigin-RevId: f8eed075f7001311f772efe7d080d9fc81dece6d

Summary: We were still expecting old units when coloring request graph edges (milliseconds rather than nanoseconds for latency, and 1 rather than 0.01 for 1% in error rates). This diff fixes that problem so that the colors aren't all red/green anymore. These functions aren't used anywhere else right now, but I also added a comment to them to make sure consumers in the future will be aware of the contract.

Test Plan: ran it on clusters, it colors correctly now

Reviewers: michelle, nlanam, zasgar

Reviewed By: nlanam

JIRA Issues: PC-1193

Differential Revision: https://phab.corp.pixielabs.ai/D9833

GitOrigin-RevId: 7ee7b02f0957755c89c11f4b197ec36dbcac699b

UI: Request graph hierarchical mode should be up-down, not left-right, to fit inside the area better

Summary: See screenshots. We were oddly utilizing the space when request graph mode was hierarchical. By changing the setting to Up down instead of left right, it utilizes the available space more nicely. Want to get weigh in from @zasgar and @michelle since they made the initial decision to use RL in the first place.

Test Plan: Ran it on a few clusters

Reviewers: zasgar, michelle, nlanam

Reviewed By: michelle

Subscribers: zasgar, michelle

JIRA Issues: PP-1190

Differential Revision: https://phab.corp.pixielabs.ai/D9836

GitOrigin-RevId: ce65aa9c547cbd371b7cbc0caa7e3b3e891061d7

Summary: For some records, we have an array of services like ["pl/nats", "pl/stan"]. For those records, we don't know which service to attribute the measurement to. This presents a problem with some of the deep linking in the UI. When you double click an array service in the service graph, the url generator expects the non array form of service. This diff fixes that, by checking to see if the service is an array (like we do in deep links in tables), and if it is, take the user to the first of the services in the array. In the future we will want to allow the user to select which service in the array, but this is an improvement for now.

Test Plan: updated unit test

Reviewers: michelle, nlanam, zasgar

Reviewed By: nlanam

JIRA Issues: PC-1179

Differential Revision: https://phab.corp.pixielabs.ai/D9832

GitOrigin-RevId: 80b27e1a6faa4712a7833025b2cefd0e322223c7

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Subscribers: oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9824

GitOrigin-RevId: c0a4e91072ea60c223894c762a5bae690a46bb2e

Summary: We set default values for vz during `deploy()` but don't write them to the spec. We assume that these values are set for other parts of the code and have run into some issues. This is a critical change that will probably resolve some unhealthy clusters to healthy immediately as soon as we update Vizier for users.

Test Plan: Tested on skaffold with a cluster that would get classified as not PVC compatible (and therefore automatically set useEtcdOperator set to true).

Reviewers: vihang, michelle, nserrino

Reviewed By: vihang

JIRA Issues: PP-3045

Differential Revision: https://phab.corp.pixielabs.ai/D9820

GitOrigin-RevId: 912e3566311a87855c950f8d3a1067013a52d957

Summary:
We should use the entire query string to cache. In addition after
kubernetes/ingress-gce#1387 lands we can disable negative caching.

Test Plan: Tested GKE config.

Reviewers: michelle

Reviewed By: michelle

Differential Revision: https://phab.corp.pixielabs.ai/D9835

GitOrigin-RevId: 44418bcc4b3f06701edb1b144929b79014bf82e2

Summary:
The standard health endpoint for kubernetes apps is `/healthz` (rather than `/health`).

This helps remove some clutter in our service maps.

Test Plan: Manual

Reviewers: nserrino, zasgar

Reviewed By: nserrino

Differential Revision: https://phab.corp.pixielabs.ai/D9830

GitOrigin-RevId: 67f6e03dc300cda89858e6f540344d600c295c9d

Summary: Remove the drop first packet hack and prepend the length header in user space.

Test Plan: existing tests and fixed bug in kafka-demo application.

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Subscribers: oazizi, yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D9825

GitOrigin-RevId: 2ec04a09d7ce94f86a0a580a09932bd2e17491b0

Summary:
Instead of tracing open/creat and keeping all open files in a map,
we can trace sock_sendmsg and sock_recvmsg to figure out whether the traffic is going to a socket.

This is part 2 of the diff, where we remove the old approach.

Test Plan: Test case added

Reviewers: #stirling, yzhao

Reviewed By: #stirling, yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D9667

GitOrigin-RevId: 0f5f1dd6412e0324ac65185ac5305faed20bc546

Summary: This format makes it easier to switch to a local repo during development.

Test Plan: Jenkins

Reviewers: #stirling, zasgar, vihang, #third_party_approvers

Reviewed By: vihang, #third_party_approvers

Differential Revision: https://phab.corp.pixielabs.ai/D9819

GitOrigin-RevId: 5164aecad5a1eb4636ce14ebde411726d3b52dd9

PP-3023: Extend px.ip_to_service_id to include ClusterIPs and add 2 more related IP UDFs for services.

Summary: Depends on D9816. We are adding service cluster IPs and external IPs to the metadata we track about each service. This will help us resolve IP addresses better. We currently can only resolve remote addresses that are pod IPs, but with this change, we will also resolve remote addresses that are service Cluster IPs, which will help for resolving client-side traced traffic. Since I already piped this information through, I also went ahead and added px.service_id_to_cluster_ip and px.service_id_to_external_ips in case we wanted to augment our service-related live views to contain this information.

Test Plan: added / edited unit tests

Reviewers: oazizi, michelle

Reviewed By: michelle

JIRA Issues: PP-3023

Differential Revision: https://phab.corp.pixielabs.ai/D9818

GitOrigin-RevId: a70a76d8fcd8faccb116e2161431acd721c300f9

Summary: We want to track service IPs in addition to pod IPs when when we resolve a remote address to a particular Kubernetes service. This diff takes service updates containing service IP information and propagates it to our metadata state. The next step is to update the px.ip_to_service_id UDF in order to fully close the loop on this information. We also may want to consider adding UDFs to fetch external IP and cluster IP for a given service.

Test Plan: updated existing unit tests

Reviewers: oazizi, michelle

Reviewed By: michelle

JIRA Issues: PP-3023

Differential Revision: https://phab.corp.pixielabs.ai/D9816

GitOrigin-RevId: 6415340dc8b2284ad89cbc0dc33272783a63ca1a

Summary: BPF code runs in the Linux kernel and needs to be GPL. The rest of the code remains licensed as is, since BPF code runs in a separate environment.

Test Plan: N/A

Reviewers: oazizi, michelle

Reviewed By: oazizi, michelle

Differential Revision: https://phab.corp.pixielabs.ai/D9814

GitOrigin-RevId: 8d5bb75f3af70f9a0567b196a6aa8c56c1ac4ddd

Summary: Additionally add comment for arguments.

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Subscribers: oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9813

GitOrigin-RevId: 77b243ea3db3f784f5c40213cc09ed7e6a7a7fbf

Summary: Include the last activity timestamp for each {pod, remote endpoint} pair.

Test Plan: Manual

Reviewers: #stirling, zasgar

Reviewed By: zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D9812

GitOrigin-RevId: 7ba1c9df1f9ae7bf67c69e470f3df4951542a4f0

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Subscribers: oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9810

GitOrigin-RevId: 46f6846b33c11c1fa9bf4ba9db9ab9131bd5ad8c

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9811

GitOrigin-RevId: e9cba4a8ba56b797b93f0b1577000bc89251298f

Summary: TSIA.

Test Plan: Run `px/http_data`. Change `df = df.head(...)` to `df = df.stream()` and run it again. While it's streaming, sort by any column other than time. It should remember this sort (and continuously apply it) as new data comes in.

Reviewers: michelle, vihang

Reviewed By: michelle

JIRA Issues: PC-1184

Differential Revision: https://phab.corp.pixielabs.ai/D9809

GitOrigin-RevId: a8d3396eb0b0816d34f3a124c4bf186a09912787

Summary:
Noticed that ProcExe() in proc_path_tools.cc and ProcParser::GetExePath() overlapping.
Merge them into ProcParser::GetExePath().

Update callers of ProcExe()

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9768

GitOrigin-RevId: 9603b23bf705736f6feddb63b4fc6c1bbe780b05

Summary:
node 15.0 is the oldest version that is supported by the existing hard-coded offsets.
We are working on more generic dwarf-based runtime offset resolution, and alternatives like
enumerate supported versions' offsets.

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Subscribers: oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9805

GitOrigin-RevId: 3e2201b59a6f317e90ec4c3f75bbfe8b865c799d

Summary: we maxed out our sentry errors in a single day, with a quota of 1mil. this diff updates the most common errors and sets them to `Info` rather than `Error`

Test Plan: n/a

Reviewers: vihang, nserrino, philkuz, zasgar

Reviewed By: vihang

Differential Revision: https://phab.corp.pixielabs.ai/D9807

GitOrigin-RevId: 58c3cd03b16a200d04e8d7f9e4f50506f9dddefb

Summary: Extend the outbound_conns script with an optional IP filter.

Test Plan: Manual

Reviewers: #stirling, zasgar, yzhao

Reviewed By: #stirling, yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D9806

GitOrigin-RevId: b29344cfa7246a7e4a8ac8c14102ac8911cc5543

Summary: We had a left over condition check before the ReconcilliationPhase addition that would check whether VizierPhase was None to determine whether to create the Vizier components or not. Unfortunately, this wasn't sufficient as there are conditions where Reconcile is called and the Vizier is mid-update. Adjusted to also check whether the ReconcilliationPhase is not None.

Test Plan: Tested on skaffold

Reviewers: vihang, michelle

Reviewed By: vihang, michelle

Differential Revision: https://phab.corp.pixielabs.ai/D9802

GitOrigin-RevId: 771773f721e458489979fc0a8b74fcf0274524ba

Summary:
The ANTLRInputStream constructor which converts from utf-8 to utf-32 was previously throwing a range_error if there were invalid utf-8 byte sequences. This diff adds a utf8cpp library to check if the input is valid UTF-8 before passing it into the ANTLRInputStream. If it's invalid I return an errmsg.
It's possible there's a bug in our postgres and/or mysql parsing that is causing these invalid utf-8 byte sequences.

Test Plan: Added test

Reviewers: nserrino, #third_party_approvers, vihang

Reviewed By: nserrino, #third_party_approvers, vihang

JIRA Issues: PP-3029

Differential Revision: https://phab.corp.pixielabs.ai/D9796

GitOrigin-RevId: 541ff44c27b2eeef9add22371c2f225d7e03b0d3

Summary:
Look up all pods in a cluster that match the Vizier labels and then use those as the data behind the debug
service. Now this should match the `px-operator` pod. To add extra differentiation, prepend the namespace before the pod name.

This mirrors what we do in the collect-logs (in fact we share the same selector between the two of them in this diff).

```
$ px debug pods -c "philkuz"
  NAME                                          PHASE    RESTARTS  MESSAGE  REASON  START TIME
  pl/pl-nats-0                                  RUNNING  0                          2021-09-16T14:00:26-07:00
  pl/vizier-certmgr-79d4cf6cc9-v4w6h            RUNNING  0                          2021-09-17T15:21:55-07:00
  pl/vizier-cloud-connector-57dc86dd8c-lrkfk    RUNNING  0                          2021-09-17T15:25:37-07:00
  pl/vizier-metadata-0                          RUNNING  0                          2021-09-17T15:21:55-07:00
  pl/vizier-proxy-5984bf6dd5-nfjpc              RUNNING  0                          2021-09-17T15:21:55-07:00
  pl/vizier-query-broker-7766ffb7d7-vm67d       RUNNING  0                          2021-09-17T15:21:55-07:00
  px-operator/vizier-operator-7b4d4c9575-cljqg  RUNNING  0                          2021-09-17T15:28:35-07:00
  pl/kelvin-657585786b-hcrqg                    RUNNING  0                          2021-09-17T15:21:54-07:00
  pl/vizier-pem-4cfw4                           RUNNING  0                          2021-09-17T15:21:55-07:00
  pl/vizier-pem-4wdj6                           RUNNING  0                          2021-09-17T15:21:55-07:00
```

Test Plan: Tested on skaffold vizier, got this data back

Reviewers: vihang, michelle

Reviewed By: michelle

JIRA Issues: PX-131

Differential Revision: https://phab.corp.pixielabs.ai/D9797

GitOrigin-RevId: d16d637272dcfed816e3a2545af5478c104c6ebf

Summary:
A lot of these blank imports were seemingly copy-pastaed.
vizier + operator work on many different kubernetes engines  and likely don't
need to rely on GCP auth.
We are also trying to ensure that pixie cloud works on a variety of environments
and should probably just import the entire auth package where needed.

Test Plan:
Ran a dev cloud + dev vizier + dev operator on my cluster.
Also tested a bazel built `px` cli

Reviewers: michelle, philkuz, zasgar

Reviewed By: zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D9798

GitOrigin-RevId: fa747111015fb4c15e94fb7c3aa2e1e1e9d7fd84

Summary: Addresses https://github.com/advisories/GHSA-cph5-m8f7-6c5x

Test Plan: Existing tests

Reviewers: michelle, nlanam, #third_party_approvers, zasgar

Reviewed By: #third_party_approvers, zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D9799

GitOrigin-RevId: bc770881424d4bf1a6e49b5878cd77238b2adde1

Summary:
This is a test that shows an unexpected behavior with uprobes that can occur when two seemingly
different files, which share the same inode, are probed.

Note this is not a case of symlinks, this is actually a case where two separate files which share the same inode are probed.

This can occur if a container is run twice on the same host.

We will need a fix for this in the future.

Test Plan: This is a test that shows the faulty behavior.

Reviewers: #stirling, yzhao

Reviewed By: #stirling, yzhao

Subscribers: yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D9791

GitOrigin-RevId: 22684c0a6997c4688c4606ec4f0fbc05ee183954

Summary: Primarily to record the requirements.

Test Plan: Jenkins

Reviewers: #stirling, oazizi

Reviewed By: #stirling, oazizi

Subscribers: oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D9793

GitOrigin-RevId: e164e508574390afab800be95c50f4f86875649a