Test Plan: Jenkins

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D7039

GitOrigin-RevId: 6abf0e4ac469ac29ea6b2033145f114abb571b2e

Summary:
hsperfdata of java binaries can conflict because they run in bazel sandboxes which operate with pid namespaces. This means PIDs are often the same.
The /tmp files that those binaries use, however, are shared. This results in clobbering of files.

Address this with sandbox_tmpfs_path, which mounts a /tmp inside each sandbox, making each sandbox operate independently.

Test Plan: Jenkins and --runs_per_test=100

Reviewers: yzhao, zasgar, #engineering, vihang

Reviewed By: #engineering, vihang

Differential Revision: https://phab.corp.pixielabs.ai/D6834

GitOrigin-RevId: ea7cce87cb2df92c9728abb53c7a2950cb5628fe

Summary: Most of the build and phabricator build is now moved to K8s. Only BPF tests (VM)  and CLI mac signing (mac) run outside.

Test Plan: Tested extensively, though I still expect some instabilities.

Reviewers: vihang, michelle, oazizi, #engineering

Reviewed By: vihang, #engineering

JIRA Issues: PP-1769

Differential Revision: https://phab.corp.pixielabs.ai/D6816

GitOrigin-RevId: bd72c16669ef8965c42887cc4163b41768953d34

Summary: This adds a machine specific bazelrc that we can use.

Test Plan: N/A

Reviewers: vihang, michelle, philkuz, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D6802

GitOrigin-RevId: 6c9a04ec6cbee461ad7df51c896a464d31dcf543

Summary: This also fixes and issue that gobpf is having building bcc dependencies.

Test Plan: Existing tests/Jenkins.

Reviewers: michelle, oazizi, vihang, #engineering

Reviewed By: oazizi, vihang, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D6609

GitOrigin-RevId: 14983216056b69a92af0414c506b3715afd26c97

Summary: TSIA

Test Plan: Used tensorflow in subsequent diffs.

Reviewers: zasgar, #engineering

Reviewed By: zasgar, #engineering

Subscribers: yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D6290

GitOrigin-RevId: fbde26db522ca0d3fa623ce3f897f147292fdb14

Summary:
These build failures only happen with GCC opt which I didn't test before submitting the GCC 10 upgrade.

UPB inside of GRPC has an issues with GCC-10 and this disables the relevant warnings.

Test Plan: N/A

Reviewers: oazizi, philkuz, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D5709

GitOrigin-RevId: eb6524f5985ae5d6cea1601df73107a6b02908de

Summary: Change default compiler to be gcc-10. It was set to gcc-9.3 before.

Test Plan: Jenkins

Reviewers: michelle, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D5695

GitOrigin-RevId: 9b6df9c1313b0197dbee1aa4e1c968670eed0d5d

Summary:
UBSAN was running, but failures were not being treated as such by bazel. This fixes that.

After enforcing UBSAN, had to make a few fixes, including an upgrade to rapidjson.

Test Plan: Existing tests.

Reviewers: zasgar, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D5656

GitOrigin-RevId: 67cf1b58933ff6d65bb75774c155ce317250da59

Summary:
We had to disable coverage builds due to an internal error in GCC. It's now been fixed and we can run coverage tests and upload to lcov.

Had to disable JAVA tests, but perhaps @yzhao can take a look after this lands.

Test Plan: Tested by uploading to lcov. Will test after it lands on main.

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Subscribers: yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D5641

GitOrigin-RevId: b0b360a9a6bd764504e31b9f1ce66cab1176d89a

Summary:
This fixes the following issue related to libtinfo on PEM/Kelvin start:
`libtinfo.so.5: cannot open shared object file: No such file or directory`

When we updated to 20.04, I think I changed the base image but not the active dev image.
This means that we were building all the images using the old ubuntu 18.04.
James noticed a break with skaffold that caused the PEMs to break because of relying on deprecated terminfo5 so he changed our sid deploy dependency to terminfo6. Unfortunately, this broke our deploy images.

I updated the dev images to be ubuntu 18.04, but sadly Clang on that relies on terminfo5 so it's broken with ubuntu 20.04. It's fixable by installing terminfo5, but this might break our code if it links to v5 instead of v6, since we only ship v6.

Updating the old clang install didn't work. clang-9 does not compile with clang-10. Ubuntu 20.04 ships with clang-10 so the compiler can't build our old clang-9 deb with terminfo6...

This updates the compiler to clang-10 with all the related changes necessary to make sure the code still compiles.

Test Plan:
`bazel test //...` in the docker VM.
`bazel run src/vizier/services/agent:pem_image` in the docker VM.

Reviewers: oazizi, michelle, jamesbartlett, philkuz, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D5623

GitOrigin-RevId: a26680e32211f9aec5e42c1349c4cd35015e2213

Summary:
go binaries with -c opt have their debug symbols stripped by the go_binary rules.

One way to prevent this is to pass the --strip=never flag.

Right now this applies globally, so C++ fastbuild targets are affected too.

When we upgrade io_bazel_rules_go, there's a better way to do this.

If someone knows a better way in the meantime, I'm all ears.

Test Plan: Jenkins

Reviewers: zasgar, yzhao, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D4853

GitOrigin-RevId: 2eaa8b97a82421087b39a1baca13346d7d627014

Summary: This allows to run bpf test without needing to set the --strategy flag.

Test Plan: Jenkins

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D3630

GitOrigin-RevId: b9a9bd8cbbf901cf3562c0f4cfd0a0a2d303cf1b

Summary: Abort on errors is causing issues.

Test Plan: tested using skaffold..

Reviewers: michelle, #engineering, oazizi

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D3408

GitOrigin-RevId: bd39867346263ba22b67df40c729364a84d4539c

Summary: This gets rid of a bazel warning.

Test Plan: N/A

Reviewers: michelle, philkuz, #engineering, oazizi

Reviewed By: #engineering, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D3407

GitOrigin-RevId: 9e7d03cbf6bb0cf5bdc552013ae19205a952e1ce

Summary:
Tag should build on Jenkins, but not run the test.

In contrast, marking a target as manual will prevent it from both building and running.

Test Plan: Jenkins

Reviewers: yzhao, zasgar, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D3228

GitOrigin-RevId: af2036406e7528fa2c73574aa0bc186c36afbd13

Summary: Container test needs to be run in exclusive mode, else it will conflict with other tests also deploying probes.

Test Plan: Enabling a test.

Reviewers: yzhao, #engineering

Reviewed By: yzhao, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D3214

GitOrigin-RevId: fbb309a00786250a20cfc31bac0fae215d35bc33

Summary:
In D3178, I accidently landed from the wrong machine and submitted the stale change.
This is the patch to get it to the correct version.

Test Plan:
  `bazel test --config=libc++ --keep_going //...`
  `bazel test --config=clang --keep_going //...`
  `bazel test --config=gcc --keep_going //...`

Reviewers: #engineering, michelle, oazizi, philkuz

Reviewed By: #engineering, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D3192

GitOrigin-RevId: e9092fdf0754c157778f8edafd98acca2393c2cb

Summary: Still has two failing tests, but will visit those after this diff is checked in.

Test Plan:
  `bazel test --config=libc++ --keep_going //...`
  `bazel test --config=clang --keep_going //...`
  `bazel test --config=gcc --keep_going //...`

Reviewers: michelle, oazizi, philkuz, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D3178

GitOrigin-RevId: 1d08f95bf9e7b10eea946b2b324fc1f3af85a976

Summary:
requires_root is defined but not actually supported in any way, so it is misleading. Remove it.

Note that D3148 is dependent on this.

Test Plan: Jenkins

Reviewers: zasgar, #engineering, yzhao

Reviewed By: zasgar, #engineering, yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D3150

GitOrigin-RevId: 413b3608c23b17322cbbed384d76ef6acfa7ed0a

Summary: msan vs asan typo

Test Plan: Jenkins

Reviewers: zasgar, #engineering

Reviewed By: zasgar, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D3149

GitOrigin-RevId: cdf92d691483a42fd1c21c52afba25f7e17569dd

Summary: This updates to clang-9 and also updates our llvm build to use 9.0

Test Plan: `bazel test //...` and jenkins.

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D2235

GitOrigin-RevId: 1bfc34c0d46fadca0a1c3f53f2ac9a675e9795c4

Summary: TSIA

Test Plan: Jenkins

Reviewers: michelle, philkuz, #engineering, oazizi

Reviewed By: #engineering, oazizi

Differential Revision: https://phab.corp.pixielabs.ai/D2772

GitOrigin-RevId: 12ad7b3fc7be20bb87d4e824bc760ff2ee4d8a1b

Summary: This should unbreak master.

Test Plan: Jenkins

Reviewers: philkuz, oazizi, #engineering

Reviewed By: philkuz, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D2764

GitOrigin-RevId: 3be0cd6e0c120c779bf23a97667a61aba0326588

Summary:
Recent updates to bazel render cc_configure unnecessary. This also fixes
static libstdc++ builds.

Change gcc/clang to be config based. `bazel build --config=gcc` to get gcc. Clang is default.

Test Plan: Jenkins

Reviewers: michelle, oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D2745

GitOrigin-RevId: 5c32408363f5d76c87d74fee1a0e700d6bb202f1

Summary: This adds a job that runs on Jenkins to build the release CLI when tags are pushed. It also adds the infromation to the artifact DB.

Test Plan: Tested on Jenkins.

Reviewers: michelle, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D2335

GitOrigin-RevId: 45eb917d3cecb58b1084942f226117d71555efa9

Summary: This is needed for the docker plugins to actually work.

Test Plan: Tested push

Reviewers: michelle, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D2317

GitOrigin-RevId: 8995e0bad9be3edf9eff336b7abfcebdd9edd7b1

Summary: No longer need the hacks :) Removed the TODOs.

Test Plan: jenkins

Reviewers: michelle, oazizi, #engineering, nserrino

Reviewed By: #engineering, nserrino

Differential Revision: https://phab.corp.pixielabs.ai/D2307

GitOrigin-RevId: a4eed59fe5817cac5f9d74a454134de1507902cf

Summary: TSIA

Test Plan: `bazel test //...`

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D2147

GitOrigin-RevId: 21716a6133b564b53b4a15854e5c5b334a4e3150

Summary:
This is currently breaking master, likely because of the Go 1.13 upgrade. With this upgrade it
seems that we need to explicitly pass in gcov flags.

Test Plan: bazel test //... and bazel coverage //src/...

Reviewers: oazizi, philkuz, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D1916

GitOrigin-RevId: 83631e5bf6c51fdc78e6068f42635ee05991c4ba

Summary: This should make linking faster and enable easier support for LTO in the future.

Test Plan: Jenkins

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D1414

GitOrigin-RevId: 0a3e9b09d36b3107fc8ee7ae5ad1f62a1ff3dfbd

Summary: Learned from last time using a .0 release, this time it's a .1 release :)

Test Plan: Jenkins

Reviewers: oazizi, #engineering, yzhao

Reviewed By: #engineering, yzhao

Differential Revision: https://phab.corp.pixielabs.ai/D1329

GitOrigin-RevId: 85ffc743fa18f5c35e190bc65f5fa3b2a66a674b

Summary: Looks like we are sometimes getting failures: https://jenkins.pixielabs.ai/blue/organizations/jenkins/pixielabs-master-nightly-test-regression/detail/pixielabs-master-nightly-test-regression/1/pipeline

Test Plan: Jenkins

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D1330

GitOrigin-RevId: cd343b7d71964b27d2be6ca062c4d723636e0fb4

Summary: Filter out manual targets in ASAN/MSAN/TSAN.

Test Plan: Jenkins

Reviewers: oazizi, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D1307

GitOrigin-RevId: 034426c7e8b5a59e16cc33f508f2f32d1519d0f4

Summary:
Improve test coverage. Also these are essential to C++ code.

Fixed one asan bug where misaligned pointer access.

Test Plan: Jenkins test.

Reviewers: #engineering, oazizi, zasgar, chengruizhe

Reviewed By: #engineering, zasgar

Differential Revision: https://phab.corp.pixielabs.ai/D1024

GitOrigin-RevId: ed640ca0724ddc5e3ac1845fd7be7b61c2928240

Summary: Some flags added to make the build work, will try to remove them after updating our deps.

Test Plan: Build, jenkins.

Reviewers: oazizi, michelle, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D1028

GitOrigin-RevId: ba9c1f93b5d14ce4c98ed378a3c4ee5112ea9a34

Summary:
Moving to a newer bazel.
   TODO: Need to remove compatibility flag

Test Plan: jenkins

Reviewers: michelle, philkuz, oazizi, #engineering

Reviewed By: michelle, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D891

GitOrigin-RevId: 75b1afab6d821790d5351f5a4874a04dd54ccd24

Summary: This will allow the build on // to work without defining a tag.

Test Plan: bazel build //...

Reviewers: philkuz, chengruizhe, #engineering, michelle

Reviewed By: #engineering, michelle

Differential Revision: https://phab.corp.pixielabs.ai/D885

GitOrigin-RevId: 324beca770c73c675c6f62aafea5f2e7d49f1c57

Summary:
These tests run inside a container with root privilege, and bazel tests them directly not
inside any sandbox.

Test Plan: Jenkins

Reviewers: kgandhi, zasgar, oazizi, #engineering

Reviewed By: zasgar, #engineering

Subscribers: philkuz

Differential Revision: https://phab.corp.pixielabs.ai/D743

GitOrigin-RevId: d5ac6f7048d2398cfda1feaddc5690041a625cd4

Summary: TSIA

Test Plan: tested manually by editing some code locally.

Reviewers: oazizi, michelle, #engineering

Reviewed By: oazizi, #engineering

Differential Revision: https://phab.corp.pixielabs.ai/D566

GitOrigin-RevId: 10297a93ee21b32a9e4f73fa2cd2a9b0cbbe37cc