This change disables using msgpack generated serializers in dev by
default.

In released binaries, we use code-generated msgpack serializers to
improve performance.  However, in development, code generated
serializers are a pain. If a developer forgets to re-generate code, the
code generated gets out of sync with the go structs, and result into
subtle bugs where some values appear not to persist as expected.

The CI and release scripts will continue to use the msgpack
code-generation. Devs who want to test locally can set
`GO_TAGS=codegen_generated` as well.

Golang 1.16 added better support for installing tools, without updating
go.mod files: https://blog.golang.org/go116-module-changes#TOC_4. .

This is required because Go does not pull CC from the make variable. This uses
whatever Go's default CC unless CC is overridden, as it is for the ARM targets.

This also makes it easier to build Nomad on a native ARM device, via:

```
make CC= pkg/linux_arm/nomad
```

* Set 'only' ALL_TARGETS rather than append

This is functionally no different than before, but it's more correct.

* Re-scope VERBOSE=true

Previously this was only set when the OS was Linux; this was added in
805ade7d.

* Warn about unsupported OS rather than error

Also:

* Only print the warning when trying to build Nomad
* Print correct list of supported OSes

This removes small differences between the targets, like the statement
about what's being built.

The CGO/Windows related comments were deleted as being not relevant. 
See https://github.com/hashicorp/nomad/pull/9643 for context.

Add a build target for Apple Silicon (m1) macs.

Note that Go must have been built with c4f497da6f for
Nomad to work on darwin/arm64 (i.e. wait for go1.16).

Closes #9408

Parameterize it so we can arbitrary target other versions, if we
are doing some manual checking, specially in the beginning when we may
want to validate compatibilities for skip release upgrades.

Also, introduce `checkbuf` target so we can run buf linter without the
rest.

use beta

This replaces all usage of `protoc` with `buf`. See `tools/buf/README.md` for more.

Previously, it was required that you `go get github.com/hashicorp/nomad` to be
able to build protos, as the protoc invocation added an include directive that
pointed to `$GOPATH/src`, which is how dependent protos were discovered. As
Nomad now uses Go modules, it won't necessarily be cloned to `$GOPATH`.
(Additionally, if you _had_ go-gotten Nomad at some point, protoc compilation
would have possibly used the _wrong_ protos, as those wouldn't necessarily be
the most up-to-date ones.)

This change modifies the proto files and the `protoc` invocation to handle
discovering dependent protos via protoc plugin modifier statements that are
specific to the protoc plugin being used.

In this change, `make proto` was run to recompile the protos, which results in
changes only to the gzipped `FileDescriptorProto`.

-I ../../.. is meant to navigate from `GOPATH/src/github.com/hashicorp/nomad` to `GOPATH/src`

This is fine but it assumes a few things about how the dev has setup nomad, which is also fine if that is the expected dev environment, however the `../../..` is not as explicit as "GOPATH/src" and it would also enable a few more scenarios so it seems strictly better to me.

Random example: nomad is a subrepo of ours, but with this change we can symlink from GOPATH/src/github.com/hashicorp/nomad and `make proto` will work.

Currently we compile (but don't run) the e2e tests as part of `test-other`,
which is skipped for branches named `e2e-*`. Move this check into the
`test-e2e` job. Split out the vault compatibility integration check as its own
makefile target for clarity.

* rename vault API compatibility test for clarity
* exercise vault secrets lease renewal

* use specific sha

* use submodule version

* add hclog vet to check step

* ...

With Go modules, `go mod tidy` supplants `vendorfmt`. Unfortunately,
`tidy` will try to reach out to the network and download modules, and
there is no way to disable that behavior (e.g. the -mod=vendor) option
does not apply. This means we cannot use the `tidy` target in nomad
enterprise, which will be unable to reach private repositories like
consul-enterprise.

This isn't a big deal, since `vendorfmt` served the purpose of rewriting
the output of `govendor`, wheras `tidy` is a part of the `sync` target
that is required to be run when modifying dependencies anyway.

go-bindata released tags are very old, few years old, so ensure we use
the latest master as of now.

Point to gopsutil fork with shirou/gopsutil#889 changes. I used hashicorp/gopsutil rather than pointing to https://github.com/Lomanic/gopsutil/tree/issue832 directly just in case..

This PR switches the Nomad repository from using govendor to Go modules
for managing dependencies. Aspects of the Nomad workflow remain pretty
much the same. The usual Makefile targets should continue to work as
they always did. The API submodule simply defers to the parent Nomad
version on the repository, keeping the semantics of API versioning that
currently exists.

We have been using fatih/hclfmt which is long abandoned. Instead, switch
to HashiCorp's own hclfmt implementation. There are some trivial changes in
behavior around whitespace.

Use v1.1.5 of go-msgpack/codec/codecgen, so go-msgpack codecgen matches
the library version.

We branched off earlier to pick up
https://github.com/hashicorp/go-msgpack/commit/f51b5189210768cf0d476580cf287620374d4f02
, but apparently that's not needed as we could customize the package via
`-c` argument.

Examples for HTTP based task-group service healthchecks are
covered by the `countdash` demo, but gRPC checks currently
have no runnable examples.

This PR adds a trivial gRPC enabled application that provides
a Service implementing the standard gRPC healthcheck interface.

Running `make dev` runs `hclfmt`, but this isn't checked as part of
CI. That makes it possible to merge un-formatted HCL and Nomad
jobspecs that later will make for dirty git staging areas when
developers pull master.

This changeset adds HCL linting to the `make check` target.

Use go mod for github.com/hashicorp/go-bindata/go-bindata and
github.com/elazarl/go-bindata-assetfs/go-bindata-assetfs but use
`@master` to pull the latest master.  These packages don't have release
tags so `@master` worksaround it.

Adding `-trimpath` to builds removes the local working directory from
the goroutine stack traces, which makes our builds more reproducible
and doesn't leak information about our local development workstations
or CI environment.

These dep tools aren't tagged properly so shouldn't be installed with go
mod

This allows using https download and go mod cache proxies, over using
git and downloading entire dependencies git history, hopefully,
resulting into a faster installation process.