Summary:
Type formatter function was only on tyhe column, not an index. This adds index support
so this is better.

Test Plan: Tested locally

Reviewers: zasgar, vihang, nserrino, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7180

GitOrigin-RevId: 3ce74ac21c4dff7438d2ea63dd6f30cc7a35cce7

Summary: Py API shold support direct connections. Actually had to make an envoy proxy fix to make this owrk.

Test Plan: Tested on a direct conn and added a unit test and both should pass

Reviewers: htroisi, vihang, nserrino, zasgar, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7179

GitOrigin-RevId: 79c6b2eda23732199c129687e4f0da2c71da4770

Summary: As suggested by Zain because our multi-cluster needs more stability improvements before we can default encourage our users to go for it.

Test Plan: Updated tests and seems to connect fine.

Reviewers: zasgar, vihang, nserrino, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7178

GitOrigin-RevId: 787c6465dafb35e778bee2a896a1f29378864463

Test Plan: Manual test

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7168

GitOrigin-RevId: b521a7ab66949f7f746aa8d34fc4666dd1dfcbe3

Summary: Added new docs for the python api that are necessary.

Test Plan: n/a

Reviewers: vihang, nserrino, htroisi, #engineering, zasgar

Reviewed By: #engineering, zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D7177

GitOrigin-RevId: 572a1a6b75da24a18cc0ec458f2da0c1dace1e75

Summary: Discovered that python API couldn't talk directly with Vizier, had an issue where ALDP wasn't found. Go API doesn't need this but c-based grpc apis do.

Test Plan: Tested on my personal cluster and fixed the issue

Reviewers: zasgar, michelle, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7176

GitOrigin-RevId: f8438e47446773c220c715c8142279f910c4af35

Summary:
Choose the offset based on the OpenSSL version number (currently a simple placeholder--Pete to implement more robust version),
and test different OpenSSL versions.

Test Plan: Two different nginx servers, one with OpenSSL 1.1.0 and one with OpenSSL 1.1.1

Reviewers: yzhao, jps, #engineering

Reviewed By: yzhao, jps, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7165

GitOrigin-RevId: fa4d01146d357c5f5a5ac59589076320a00bd13c

Summary: Added a UDF to convert a pod ip to service id.

Test Plan: Unit Test.

Reviewers: nserrino, michelle, #engineering, philkuz

Reviewed By: #engineering, philkuz

Subscribers: vihang

JIRA Issues: PP-2311

Differential Revision: https://phab.corp.pixielabs.ai/D7172

GitOrigin-RevId: 4cb9d15947e145db2007192a31792e398e9247ce

Summary:
TSIA. We use GetAll for tracepoints.
Depends on D7125

Test Plan: Added tests.

Reviewers: michelle, nserrino, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7166

GitOrigin-RevId: c951bcbf6547ef71c71917e03d4a95c83e6f7f0a

Summary:
TSIA. This will let us filter on the tag for CI as need be.
Depends on D7125

Test Plan: N/A

Reviewers: nserrino, michelle, #engineering, zasgar

Reviewed By: #engineering, zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D7171

GitOrigin-RevId: 459a2ac7e56d81a99d0eb4fe53831460eca2b360

Summary:
This seems cleaner since our datastore are basically wrappers and
convenience methods to unify different backends.
Depends on D7125

Test Plan: Existing tests.

Reviewers: nserrino, michelle, #engineering

Reviewed By: nserrino, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7170

GitOrigin-RevId: bbd3f5a40a5a3e7432a06bcbf6f1cf6e9200f065

Summary:
While this doesn't quite follow the pattern of having the interface implemented at the consumer, it attempts to split up the interface into smaller logical pieces.

To avoid test dupe, and to ensure that both implementations behave similarly, I decided to combine the `etcd` and `pebbledb` tests into one test.
To run a single test on both implementations, we need a interface for the two datastores. Hence this does the same.

Test Plan: Refactored existing tests.

Reviewers: nserrino, michelle, #engineering

Reviewed By: michelle, #engineering

JIRA Issues: PP-2376

Differential Revision: https://phab.corp.pixielabs.ai/D7125

GitOrigin-RevId: 311944a8fb67c7753278c8976012758a2ac5e482

Summary: Part number next.

Test Plan: No manual testing needed. Coverage improvements.

Reviewers: michelle, vihang, #engineering, zasgar

Reviewed By: #engineering, zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D7173

GitOrigin-RevId: 7a44a710ea81c0462db4a71b0c50d26b0219fa5f

Summary: We need different symbol addresses depending on the OpenSSL version. This diff adds the mechanism by which we can change the symbol addresses used in the uprobe dynamically.

Test Plan: Existing tests for now. Will need more tests based on OpenSSL version once things are pieced together.

Reviewers: yzhao, jps, #engineering

Reviewed By: yzhao, jps, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7160

GitOrigin-RevId: 68890fb972d67dca732c1614f9cd5f51fc21c689

Summary: This helps readers understand the format of the syscall probes.

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7150

GitOrigin-RevId: b934a958b208a1c7740d79014ad934937d1f7b88

Summary:
For https://pixie-labs.atlassian.net/browse/PP-2400
This should provides the context on why the exception throws.

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7163

GitOrigin-RevId: e05a95c56f2047f0f1dd5bff92b7ea75f875865a

Summary: Depends on D7154. Add a GetTables UDTF to Carnot which calls GetSchemas() in the metadata service (similar to GetTableSchemas()).

Test Plan: We didn't have a test for GetTableSchemas, likely due to the fact that that it would need to rely on a mock of the MDS and would likely not add much value beyond the template checks that are done during compilation, so I followed the same pattern here.

Reviewers: zasgar, philkuz, #engineering

Reviewed By: zasgar, #engineering

Subscribers: vjain

JIRA Issues: PP-2396

Differential Revision: https://phab.corp.pixielabs.ai/D7155

GitOrigin-RevId: 1d24e0b6bc3611c0d16ef826569d6a41efc15f32

Summary:
we updated the run command to do so, but never updated `get pem` to use the current context cluster if its available.
pulled the logic for doing this out of the `run` command, so that `run` and `get` can both use it.

Test Plan: ran CLI `get pem`

Reviewers: philkuz, nserrino, #engineering

Reviewed By: nserrino, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7157

GitOrigin-RevId: eb555be587cd96576b05cb2bd8503e7e779fc442

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7142

GitOrigin-RevId: 924544b02a2ba4e7a304d27164aa9097e2c48926

Summary:
Adds a etcd implementation for use as a KVStore.
(Note: This is largely the same as the existing etcd implementation, but done
in a clean room to make switching to this easier. Also lines up with the pebbledb
implementation to have a common interface).

Test Plan: Added unit tests.

Reviewers: michelle, nserrino, #engineering

Reviewed By: michelle, #engineering

Subscribers: vjain

JIRA Issues: PP-2379

Differential Revision: https://phab.corp.pixielabs.ai/D7158

GitOrigin-RevId: e887543a6e412e2476c652c1a952c7da472f7df7

Summary:
This helps the importname not stutter ... it's `pebbledb.DataStore`
instead of `pebbledb.PebbleWrapper`.
Also set a test size so that the timeout is shorter.

Test Plan: Existing tests.

Reviewers: michelle, nserrino, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7159

GitOrigin-RevId: a53e745147c0d5e9812b02b24295d5a89640ee7b

Summary: Will make further changes, but clean-up for now.

Test Plan: Existing tests.

Reviewers: yzhao, jps, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7136

GitOrigin-RevId: 15e7f53de4830f79a25680a69dd8fabee6ce6ec4

Summary: TSIA

Test Plan: `go mod tidy`, `make gazelle`, `bazel build //...`

Reviewers: michelle, vihang, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7161

GitOrigin-RevId: 58420dff9c4a85212412c5214a83014e0344600f

Summary:
Noticed stirling was spending quite a bit of time during initialization trying to uprobe itself.

By skipping that, we can speed up init time.

This means we won't see any SSL traffic traced at Stirling/PEM, but that should be okay, since today we use grpc_cpp, which we don't trace anyways.

Test Plan: Manual

Reviewers: yzhao, jps, zasgar, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7135

GitOrigin-RevId: 41934f44bd9d81880fc0b5392b89d44f28a7a9ba

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7151

GitOrigin-RevId: 228992a36958090bf0618a397dce6578d8ad8689

openssl_trace.c: add comment to explain usage of type 'int' that is inconsistent with other similar cases where bytes_count is captured.

Summary: openssl_trace.c: add comment to explain usage of type 'int' that is inconsistent with other similar cases where bytes_count is captured.

Test Plan: None, this is a comment.

Reviewers: oazizi, yzhao, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7156

GitOrigin-RevId: 4de08b42689e7a5c4044898d4c05ca7fe52672a1

Summary: We want to have a UDTF that returns table names and their descriptions. This diff adds that functionality to GetSchemas in the metadata service, which will be used by the UDTF to produce that result table. Next up, add that UDTF.

Test Plan: edited existing

Reviewers: michelle, philkuz, #engineering, zasgar

Reviewed By: #engineering, zasgar

Subscribers: vjain

JIRA Issues: PP-2396

Differential Revision: https://phab.corp.pixielabs.ai/D7154

GitOrigin-RevId: 09d2fccb50dedaab60609c7d0a51548b03056ccb

Summary:
This has been sitting unimplemented for months so we should just remove it for now. I think a better solution for this API will be to give GetSchemas an optional agent ID list. Then, GetSchemasResponse will return a map of table to agents that contain that table. (this way we don't duplicate a full schema for each agent)

This API is unused and unimplemented so the API breaking change errors should be ok to ignore.

Test Plan: n/a

Reviewers: michelle, vihang, #engineering, zasgar

Reviewed By: #engineering, zasgar

Subscribers: vjain

Differential Revision: https://phab.corp.pixielabs.ai/D7153

GitOrigin-RevId: faf6ee060e1c0184b5ef7ab776ce84005f0c3088

openssl_trace.c: declare bytes_count as int, because open ssl returns as int, not as long. Prevent aliasing of -1 to 2^32-1 e.g., prevent bug where we continue to process data that we should skip.

Summary: openssl_trace.c: declare bytes_count as int, because open ssl returns as int, not as long. Prevent aliasing of -1 to 2^32-1 e.g., prevent bug where we continue to process data that we should skip.

Test Plan: Jenkins

Reviewers: yzhao, oazizi, #engineering

Reviewed By: oazizi, #engineering

JIRA Issues: PP-2393

Differential Revision: https://phab.corp.pixielabs.ai/D7152

GitOrigin-RevId: ad8029a9c87587ab961ec198cd090eb324f527ab

Summary: Need to have kRoleClient and kRoleServer maintain their old meanings to avoid breaking PxL scripts.

Test Plan: Existing tests + manual. We'll add a test for this afterwards.

Reviewers: yzhao, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7149

GitOrigin-RevId: 6aa59b0d752c7adc577c9dff969dc44475cc2bf3

Summary: Moves to directory based jenkins structure.

Test Plan: N/A

Reviewers: oazizi, vihang, michelle, #engineering

Reviewed By: oazizi, vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7148

GitOrigin-RevId: 785f57adedc216649782446f2672aeef54391f28

Test Plan: Manual Testing

Reviewers: nserrino, philkuz, zasgar, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7145

GitOrigin-RevId: fbb4cb5758e7ae485aeecb670f318f017db322da

Summary: Part of effort to add proper RBAC to jenkins.

Test Plan: N/A

Reviewers: michelle, oazizi, vihang, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7146

GitOrigin-RevId: 1d8b863a0c9e5fac55acf5196e46c736bd528044

Summary: More stress on OpenSSL tracing.

Test Plan: This is a test.

Reviewers: yzhao, jps, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7134

GitOrigin-RevId: ce76401ab2106e0174673d408111fec74faae6c5

Summary: CloudAPI is necessary to get info about available clusters for an org. This adds some API support in the pixie client to enable users to select cluster programmatically rather than manually.

Test Plan: Going to add tests to api_test.py for the new Client methods.

Reviewers: zasgar, vihang, htroisi, #engineering

Reviewed By: vihang, #engineering

JIRA Issues: PP-2399

Differential Revision: https://phab.corp.pixielabs.ai/D7144

GitOrigin-RevId: 9e6a8ced92966f2853bc0cc39bdc90f9c72574e8

Summary:
cloud api has a bunch of protos dependencies. :This converts them into the python format to use in the pixie python API. Most are necessary because they are needed by the VizierClusterInfo service. Not sure how to get around this.

Side note: also added a new proto compile function for just proto libs so we don't have to make grpc file when we only need proto. GRPC proto file is basically just an import grpc so is super useless.

Test Plan: n/a

Reviewers: vihang, zasgar, nserrino, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7141

GitOrigin-RevId: 920983918bffb78a91921913ba1e8b8b6728472c

Summary: Noticed a good amount of the Pixie API should be hidden. This diff hides them. Also removes unused variables.

Test Plan: tests all pass

Reviewers: vihang, nserrino, zasgar, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7143

GitOrigin-RevId: 40717170a805baec1256e594f1c6185ce139a312

Summary: Depends on D7138. Agent should tell the metadata service the description for each table when it sends its agent update.

Test Plan: existing

Reviewers: philkuz, michelle, zasgar, #engineering

Reviewed By: zasgar, #engineering

JIRA Issues: PP-2396

Differential Revision: https://phab.corp.pixielabs.ai/D7139

GitOrigin-RevId: a4ddf493c3d956421e36f369f78b4d11c1064024

Summary: We want to be able to query tables and their descriptions in a UDTF. This diff adds tracking of table descriptions in the RelationInfo class used by the agents to manage their relations. Next up, update the protobuf so that these descriptions are sent to the metadata service.

Test Plan: updated existing

Reviewers: philkuz, oazizi, #engineering, zasgar

Reviewed By: #engineering, zasgar

Subscribers: htroisi

JIRA Issues: PP-2396

Differential Revision: https://phab.corp.pixielabs.ai/D7138

GitOrigin-RevId: 9c1adb2cf933e21ae2c240b4f7ccb6be541ec31f

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7118

GitOrigin-RevId: bbb06279e78f7025db54a11d95cc264c720e02b2