Summary: This diff enables the TaskStructResolver for BCC instances that can't find Linux headers on the host.

Test Plan: Jenkins is not a good test environment, because it does have host headers. Still thinking of ways of testing this. Will not release until more rigorously tested.

Reviewers: yzhao, jps, rcheng, #engineering

Reviewed By: jps

Differential Revision: https://phab.corp.pixielabs.ai/D7525

GitOrigin-RevId: 5a8feba523770e63d089226d13225dfbab831f72

Summary: Some scripts for deploying on k8s.

Test Plan: Manual

Reviewers: yzhao, jps, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7545

GitOrigin-RevId: 30002a264c655d41878a195ad584a1f0e474ead3

Summary:
Made this to investigate the performance issues on gke:prod,
but good to have around.

Test Plan: None

Reviewers: jps, yzhao, rcheng, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7537

GitOrigin-RevId: 0146c01421f82dd290409d2f2d98e9cf4761aeee

Summary: Tidying up.

Test Plan: Existing tests.

Reviewers: yzhao, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7555

GitOrigin-RevId: beb0e17fcc1c690f2d3c893de1eddce944b96471

Summary: passthrough proxy currently returns any query GRPC error as a stream read error, which masks the actual error and makes it harder to debug problems with the query broker. non-passthrough mode will just return the error, so let's do the same thing here.

Test Plan: edited

Reviewers: michelle, vihang, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7556

GitOrigin-RevId: dbae85153735aeef7421bf0c1b87c1ac99debfad

Summary:
This prevents multiple missing request responses from relying on
the same topic/channels.

Test Plan:
skaffld deploy cloud, restart the vzconn and vzmgr pods. Make sure
everything recovers as expected

Reviewers: michelle, #engineering

Reviewed By: michelle, #engineering

Subscribers: nserrino

Differential Revision: https://phab.corp.pixielabs.ai/D7548

GitOrigin-RevId: b23f375a244fc938cbaac1b7ffc1804866c9369d

Summary: Just tidying up a bit. Forked off from another diff.

Test Plan: Existing tests.

Reviewers: yzhao, jps, rcheng, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7550

GitOrigin-RevId: ec6902327d868ecaddca38646d175131d029f75b

Summary: One node on `gke:prod` was using a lot of CPU. Turns out its related to how we deal with failed protobuf parsing. Fix it.

Test Plan: Existing tests.

Reviewers: yzhao, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7551

GitOrigin-RevId: cbc013d481e63f0d83b1bafaca3a0c16cf80d105

Summary: Depends on D7528, which added UDF source executor as a concept to scalar UDFs, re-using the same approach as UDTF. This diff adds the ability to the registry info class to look up which UDF Source executor type a given scalar UDF has.

Test Plan: added unit test

Reviewers: philkuz, zasgar, jamesbartlett, michelle, #engineering

Reviewed By: philkuz, #engineering

JIRA Issues: PP-2445

Differential Revision: https://phab.corp.pixielabs.ai/D7530

GitOrigin-RevId: 8518fd3bbddeeaf6e32ccb5159c41ca5b2e405fc

Summary:
An argument to a script is required if its variable definition omits `defaultValue`. The UI shows this with an asterisk. This diff also refuses to run the script, and warns the user, if they try to run a script with such an argument missing.

In this image, `defaultValue` is omitted in the vis spec, the argument is marked with an asterisk, the value is missing, and the run button was pressed.
{F115650}

Test Plan: Open a script, remove the `defaultValue` field from one of its variables in the vis spec, and then try to run it with the arg missing. It should refuse, and tell you to specify the arg.

Reviewers: michelle, vihang, #engineering

Reviewed By: michelle, #engineering

Subscribers: nserrino

JIRA Issues: PC-548

Differential Revision: https://phab.corp.pixielabs.ai/D7536

GitOrigin-RevId: 0d9679bafe0c3f250a8e78af86a45d0f1f474977

Test Plan: Jenkins

Reviewers: oazizi, nserrino, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7553

GitOrigin-RevId: 976791bdc0380f799170fe46b7b8eb4211ba05d7

Summary: Similar to UDTFs, some UDFs should only be executed on Kelvin or PEM Carnot instances. This diff follows the same protocol used for UDTF in order to add that distinction for UDF. This will be used to help plan queries so that Kelvin-only UDFs are scheduled on Kelvin, and also alert the user with a compiler error if they are trying to run a PEM-only UDF after a blocking node (aka on a Kelvin node). While some aspects of how we handle metadata and compilation of distributed plans will change in the future with Carnot ETL, the concept of certain UDFs needing to execute in on a specific type of agent is expected to stick around in some form.

Test Plan: updated unit tests

Reviewers: philkuz, zasgar, jamesbartlett, michelle, #engineering

Reviewed By: philkuz, #engineering

JIRA Issues: PP-2445

Differential Revision: https://phab.corp.pixielabs.ai/D7528

GitOrigin-RevId: 2e8674fd914978926bb3f0e52831f175acb76bcb

Summary: TSIA. Also we no longer need `beta` for container commands.

Test Plan: Run chef

Reviewers: zasgar, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7472

GitOrigin-RevId: 552f28a2357038aca81e8b777f3cb0fff9607113

Summary:
This makes it easier to grab a profile on demand if there are any
problems with the service.
Note, if running `go tool pprof` on a port forwarded service, use `https+insecure`
as the scheme to ignore the TLS cert.

Test Plan:
Deploy pixie cloud with skaffold. Portforward one of the services.
Run `go tool pprof`, ensure that we can inspect profiles.

Reviewers: zasgar, michelle, #engineering

Reviewed By: zasgar, #engineering

Subscribers: yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D7538

GitOrigin-RevId: c8db56d7dcb0afe0164bebf2180c52432ed4972b

Summary:
Previously, semantic type rules involved manually specifying the semantic types that should be inferred from a UDF's arguments. This can be a bit tedious for UDFs like px.select and px.any, which will always propagate their input arguments (as long as they match, in px.select's case). This diff extends the InheritTypeFromArgs rule in order to generate a list of all the current SemanticTypes, so that UDFs like px.select don't have to enumerate every existing UDF every time.
This is to support htroisi's work improving the redis_flow_graph live view.

Test Plan: tested it on cluster

Reviewers: jamesbartlett, zasgar, philkuz, #engineering

Reviewed By: jamesbartlett, #engineering

Subscribers: htroisi

Differential Revision: https://phab.corp.pixielabs.ai/D7513

GitOrigin-RevId: c5ce2905ee823f5117bfb30602643cb976deb5d8

Summary: While addressing PP-2445, I noticed that there was unused distributed splitter code, so this diff removes that code.

Test Plan: existing

Reviewers: philkuz, #engineering, zasgar

Reviewed By: #engineering, zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D7544

GitOrigin-RevId: 84596f3295280df3642fcc61d6bf22254eeaa6cb

Summary: TSIA

Test Plan: N/A

Reviewers: zasgar, michelle, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7546

GitOrigin-RevId: 3170c64178bc0951fb7083a2cd2f5b41b9082e71

Summary:
this is for the profiler project. the input columns from the profiler are stacktrace and count.
the frontend will be responsible for processing these columns and creating a JSON which can be used by Vega.
this diff contains only the vispec so far.

Test Plan: n/a

Reviewers: nserrino, nlanam, jamesbartlett, philkuz, #engineering

Reviewed By: nserrino, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7540

GitOrigin-RevId: 3dcbdeabdc9a5d43f6ccf1fce01bebaf559df460

Summary: TSIA

Test Plan: Run `arc lint` from within `src/ui`

Reviewers: nlanam, #engineering

Reviewed By: nlanam, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7547

GitOrigin-RevId: 69e146238edaddd9c6475227dfa87a341b3edd63

Summary:
Previously, we were running the `eslint_ui.sh` helper script as part of the version check even if we didn't have any files to lint with `eslint`.
To speed it up, I removed the version check.

However arcanist runs linters in parallel, which now meant that we had multiple `yarn install`s running concurrently.
These would stomp on the contents of `node_modules` and the `eslint` binary causing some lint runs to emit no output which in turn cause `arc` to fatal.

Instead we hook into `willLintPaths` and run a setup script once if there are any paths before calling the parent class's `willLintPaths` which will actually create the futures to run the linter.

Test Plan:
Before this change
- edit multiple `ts` files
- `rm -rf src/ui/node_modules; arc lint --trace`
- notice failure

After this change
- edit multiple `ts` files
- `rm -rf src/ui/node_modules; arc lint --trace`
- notice `eslint_setup.sh` being run first
- notice `eslint_ui.sh` being run in...

Summary: some cleanup that I made in D6949 led to this bug, where non-minikube clusters were getting categorized as the minikubeOther cluster type.

Test Plan: ran px deploy

Reviewers: oazizi, zasgar, #engineering, vihang

Reviewed By: #engineering, vihang

Differential Revision: https://phab.corp.pixielabs.ai/D7542

GitOrigin-RevId: bf48124f4a010b3df13ac2377c7cd06e96a84b94

Summary: these arent very helpful, and extremely spammy

Test Plan: n/a

Reviewers: vihang, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7539

GitOrigin-RevId: a63f316556a6589965573e84933e52108090e2cd

Test Plan: Jenkins

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7535

GitOrigin-RevId: 991a71d231697bdeba8aeceb11af4d60a5f21e9c

Summary: The purpose is primarily to get consistent CONN_TRACE() coverage.

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7509

GitOrigin-RevId: 6942415ef3627523122b8fb6247481b464287f8b

Test Plan: Jenkins

Reviewers: jps, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7505

GitOrigin-RevId: a6e471c5a986aaa66c31f8e0aa44f8d500a3f975

Summary: TSIA

Test Plan: Existing tests

Reviewers: #engineering, michelle, nserrino

Reviewed By: #engineering, michelle

Differential Revision: https://phab.corp.pixielabs.ai/D7526

GitOrigin-RevId: d05ebed96fb3229c811bdb0d0410dfddedd4c2b1

Summary:
we are seeing cases where users are deploying to clusters that we don't support.
following the discussion on Slack, we want to move to more of a whitelist/allowlist model. for clusters that we know we do support, we do not display any errors/warnings. for clusters that we aret sure if we support (such as custom k8s clusters), then we still allow them to deploy, but notify them that things may not work as expected.
i decided to still keep some of the blacklist model, since we have cluster types that we know we do not support. we might as well prevent the deployment in these cases.

feedback on this flow, and the copy of the messaging, are appreciated.

Test Plan: ran deploy on minikube/gke.

Reviewers: oazizi, zasgar, philkuz, #engineering

Reviewed By: oazizi, zasgar, #engineering

Subscribers: htroisi

Differential Revision: https://phab.corp.pixielabs.ai/D6949

GitOrigin-RevId: 16873f84d6cc95a210db14513a07cd0a832f8dd9

stirling/testing/common.h: add FindRecordIdxMatchesPIDs(), same as FindRecordIdxMatchesPID(), but for multiple pids. Returns row indices, in a set of columns, that match the pids passed in by a vector.

Summary: stirling/testing/common.h: add FindRecordIdxMatchesPIDs(), same as FindRecordIdxMatchesPID(), but for multiple pids. Returns row indices, in a set of columns, that match the pids passed in by a vector.

Test Plan: Existing tests.

Reviewers: oazizi, yzhao, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7534

GitOrigin-RevId: 0279e666ebfe2e478d89794a52ba7f79b4a44146

subprocess.h|cc: inherit from NotCopyable & add a pedantic check that sub-process has not already been started.

Summary: subprocess.h|cc: inherit from NotCopyable & add a pedantic check that sub-process has not already been started.

Test Plan: Verified by trying to reproduce bugs experienced in PerfProfilerTest. Bugs were impossible to reproduce because NotCopyable caused compilation errors.

Reviewers: oazizi, yzhao, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7531

GitOrigin-RevId: 1d8c1a5ae9c1065c77c440850e45dc88a7f0f904

Summary:
All the library maps were stale. Run `arc liberate` and also gitignore
the liberate module cache.

Test Plan: N/A

Reviewers: zasgar, michelle, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7532

GitOrigin-RevId: 08fcdf97f487c5ec2a41c669bbd321dec5bd5f0f

Summary:
We should probably add this as a linter but that's more work than just
running it once.

Test Plan: N/A

Reviewers: michelle, zasgar, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7533

GitOrigin-RevId: 2ed95c20427c22e62ecc960e7e177efc82718028

Summary: Combine CLI & updater program. Ensure that we deploy the correct metadata service.

Test Plan: Manual deploy with and without pv, test we update properly

Reviewers: nserrino, michelle, vihang, #engineering

Reviewed By: michelle, #engineering

JIRA Issues: PP-2383

Differential Revision: https://phab.corp.pixielabs.ai/D7512

GitOrigin-RevId: 9e3f0e8405228885e6c652b6aa99010b7cc72ab9

Summary: Metadata server uses pebble db if the appropriate flag is set.

Test Plan: Manual testing

Reviewers: michelle, nserrino, vihang, #engineering

Reviewed By: vihang, #engineering

JIRA Issues: PP-2478

Differential Revision: https://phab.corp.pixielabs.ai/D7517

GitOrigin-RevId: 3cb06225f5aef73edf8284640a62059160f247c2

Summary: Required to break a circular dependency in a future diff.

Test Plan: Existing tests.

Reviewers: yzhao, jps, rcheng, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7518

GitOrigin-RevId: bca981b316276f0382e0e3cb8de0150c773c2269

Summary:
Makes sure that if a source file affects a library but not a binary, the dependent library is still added to the buildables.

Triggered by a case where a file was under development, with no enabled test, which ended up breaking a coverage build on main.

Test Plan: Jenkins

Reviewers: vihang, zasgar, #engineering

Reviewed By: vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7507

GitOrigin-RevId: 032de79b3e3d550bd9ab7fff2db8e7ba1db7bb71

Summary:
The original code would find the start_time twice when the OS had never been suspended.
This would cause resolution failures.

This fixes that by looking for two identical start times back-to-back.

Test Plan: Existing tests (which are now enabled).

Reviewers: yzhao, jps, #engineering

Reviewed By: jps

Differential Revision: https://phab.corp.pixielabs.ai/D7481

GitOrigin-RevId: 37bed94d115bab6503efbe6c9de5eed9122c1125

Summary: Changes to support: https://github.com/pixie-labs/pixie-docs/pull/246

Test Plan: n/a

Reviewers: philkuz, #engineering

Reviewed By: philkuz, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7529

GitOrigin-RevId: 70d70469916b4fd1fc4f656afd194b0551ac27dc

Summary: TSIA

Test Plan: NA

Reviewers: nserrino, #engineering, michelle

Reviewed By: #engineering, michelle

Differential Revision: https://phab.corp.pixielabs.ai/D7523

GitOrigin-RevId: 187594a50b191cb7b18742b23c7d3e1fe0b06d8a

Summary:
Simple fix.

D7507 will prevent things like this from leaking out into main.

Test Plan: Existing tests.

Reviewers: yzhao, jps, rcheng, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7524

GitOrigin-RevId: 5a78646c5c3d917a591144a765b04ac1955cf8c5

perf_profiler/stack_traces_table.h: pull out the sample period and push period into their own constexprs for use elsewhere.

Summary: perf_profiler/stack_traces_table.h: pull out the sample period and push period into their own constexprs for use elsewhere.

Test Plan: this is in support of the perf profiler test app.

Reviewers: oazizi, yzhao, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7520

GitOrigin-RevId: 9eeeda3954dceb2cb94cc871290b1fc3acb09e0e