If no storage provider is found, let's assign env.testProvider with an
empty string so that we wil run all the tests.
Signed-off-by: Sébastien Han <seb@redhat.com>

ceph: run ceph processes with the 'ceph' user

At the beginning of every container startup we now change the ownership
of every known ceph directories to ceph:ceph.

This will allow daemons to start properly as well as allowing daemon to
writes their logs on files.

This is needed because ceph daemon now run as 'ceph' instead of 'root'
Signed-off-by: Sébastien Han <seb@redhat.com>

Ceph daemons will run as the 'ceph' user instead of the 'root' user.
Existing data store (osd and mon) owned by root will continue to run as
'root'.
ALl CLI calls also inherate from this and will use them when fetching a key
for instance. If we run on a version of ceph that does not know these
  flags they will simply get ignored.

Consider the following:

[leseb@tarox~/go/src/github.com/rook/rook][use-ceph-user !] kubectl get pods -n rook-ceph
NAME                                     READY     STATUS      RESTARTS   AGE
rook-ceph-mds-myfs-a-67cff7df69-fq4l5    1/1       Running     0          3m
rook-ceph-mds-myfs-b-85bd5f59f5-d8hdn    1/1       Running     0          3m
rook-ceph-mgr-a-7d88b9457b-jzln6         1/1       Running     2          4m
rook-ceph-mon-a-64bf697496-r94tc         1/1       Running     0          4m
rook-ceph-osd-0-c5fdcd74c-9kvb4          1/1       Running     0          3m
rook-ceph-osd-prepare-minikube-shz9l     0/2       Completed   0          3m
rook-ceph-rbd-mirror-a-5cbff8469-xv64c   1/1       Running     0          3m
rook-ceph-rgw-my-store-f7645b44-vlbp4    1/1       Running     0          3m

MDS daemon:
ceph         1  0.2  0.1 449508 29200 ?        Ssl  15:10   0:00 ceph-mds --fsid=bf359e2c-a1df-4a80-9137-8134b41a5459 --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug  --mon-host=10.110.204.228:6789 --mon-initial-members=a --name=mds.myfs-a --setuser=ceph --setgroup=ceph --foreground --mds-standby-for-fscid=1 --mds-standby-replay=true

MGR daemon:
ceph         1  0.8  0.6 867176 104732 ?       Ssl  15:10   0:02 ceph-mgr --fsid=bf359e2c-a1df-4a80-9137-8134b41a5459 --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug  --mon-host=10.110.204.228:6789 --mon-initial-members=a --name=mgr.a --setuser=ceph --setgroup=ceph --foreground

MON daemon:
ceph         1     0  0 15:10 ?        00:00:02 ceph-mon --fsid=bf359e2c-a1df-4a80-9137-8134b41a5459 --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug  --mon-host=10.110.204.228:6789 --mon-initial-members=a --name=mon.a --setuser=ceph --setgroup=ceph --foreground --public-addr=10.110.204.228 --public-bind-addr=172.17.0.6 --setuser-match-path=/var/lib/ceph/mon/ceph-a

OSD daemon:
ceph     30985  0.9  0.5 885736 81236 ?        Ssl  15:10   0:04      |   \_ ceph-osd --foreground --id 0 --conf /var/lib/rook/osd0/rook-ceph.config --osd-data /var/lib/rook/osd0 --setuser-match-path /var/lib/rook/osd0 --setuser ceph --setgro
up ceph --keyring /var/lib/rook/osd0/keyring --cluster rook-ceph --osd-uuid 98b4527d-f7a8-44f0-ac47-f8b55ff5ee05 --osd-journal=/var/lib/rook/osd0/journal

RBD mirror daemon:
ceph         1  0.0  0.1 914500 21848 ?        Ssl  15:10   0:00 rbd-mirror --setuser ceph --setgroup ceph --foreground -n client.rbd-mirror.a --conf /etc/ceph/ceph.conf --keyring /etc/ceph/keyring

RGW daemon:
ceph         1  0.2  0.3 5179452 52244 ?       Ssl  15:11   0:01 radosgw --setuser ceph --setgroup ceph --foreground --name=client.radosgw.gateway --rgw-mime-types-file=/var/lib/rook/rgw/mime.types

Even though Ceph processes are running as Ceph, the container itself runs as root.
We haven't reached the situation where we can runAsUser != "root" and enable SecurityContext to RunAsNonRoot, but that's our ultimate goal.

Resolves: https://github.com/rook/rook/issues/2664

Signed-off-by: Sébastien Han <seb@redhat.com>

ceph manifests: add psp to common.yaml & integration tests

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>

Add Rook-Ceph PSP requirements to the `common.yaml` file. This is
commented for users to inspect and make informed decisions about their
own clusters just as with the rest of the file. For user clusters
without pod security policies enabled, this part of the config will be
a don't-care and not harm anything.

Because the PSP setup is now codified in `common.yaml`, it is no longer
necessary to have documentation for pod security policies, and the
`Documentation/ceph-psp.md` file is deleted.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>

Allow executing integration tests for specific storage providers

ceph: allow to set multiple tolerations for agent and discover pods

With the storage providers growing, we should make it
clear in the source which tests belong to which storage
provider. This change renames the ceph integration tests.
Signed-off-by: travisn <tnielsen@redhat.com>

The relative links in the PR template are broken so we
need to specify full paths.
Signed-off-by: travisn <tnielsen@redhat.com>

Since most PRs are focused on a single storage provider,
the tests for other storage providers do not need to be
run. The CI will allow skipping tests for other storage
providers by specifying keywords in the PR description.
Similar to [skip ci], you can now specify [test storage-provider].
For example, [test ceph] will only trigger the test
suites for ceph and none of the other storage providers.
Signed-off-by: travisn <tnielsen@redhat.com>

Closes #2896
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>

Add/remove nodes via EdgeFS cluster' CRD

Remove legacy Ceph CRD conversion from v1beta1 to v1

Signed-off-by: Anton Skriptsov <sabbotagge@gmail.com>

ceph: improve upgrade procedure

We do not determine the daemon type and name by string parsing anymore
but passing that info directly in the update call, it's more reliable.
Also removing a bunch of unused functions.
Signed-off-by: Sébastien Han <seb@redhat.com>

This commit makes OnAdd() and OnUpdate() consistent in behavior. They
will do their best detecting if we can upgrade, continue an upgrade or
refuse to do one. It also refuses to downgrade to any version.
It also load ClusterInfo earlier in the controller so that we can
compare the spec version versus the cluster running one.
Signed-off-by: Sébastien Han <seb@redhat.com>

Fix mgr prometheus annotations

If we detect all OSDs are running on the same host, we don't perform any
checks for upgrade.
Signed-off-by: Sébastien Han <seb@redhat.com>

Logs will be lost if we collect them after pods are uninstalled.
So we need to collect them before purging.
Signed-off-by: Sébastien Han <seb@redhat.com>

If we successfully changed all the Ceph version of all the daemon we
consider the upgrade complete and successfully, thus we display a
message.
Signed-off-by: Sébastien Han <seb@redhat.com>

Do not consider clusterInfo Ceph version since it's the one picked by
the spec image.
The CephVersion provided here needs to be the current running version
of Ceph for when it is later passed to OkToStop and checked with
IsAtLeast. If you are upgrading from a pre 14.2.1 version to a
version 14.2.1 or newer, IsAtLeast will incorrectly return true and
then fail trying to run mon ok-to-stop on a mon pod running a pre 14.2.1 version.
Signed-off-by: Sébastien Han <seb@redhat.com>

Add more details on how upgrades are performed, as well as updating the
release note.
Signed-off-by: Sébastien Han <seb@redhat.com>

If the cluster shows a state of HEALTH_ERR, Rook will refuse to upgrade
it if the image version changed.
Signed-off-by: Sébastien Han <seb@redhat.com>

These were used on a previous patch as a debug, let's remove them. They
are not intended to be displayed.
Signed-off-by: Sébastien Han <seb@redhat.com>

This method does not start anything but creates the deployment. So let's
rename it to createDeployment() instead.
Signed-off-by: Sébastien Han <seb@redhat.com>

rgw wasn't using UpdateDeploymentAndWait() to update its deployment
configuration. We now use it so that it's consistent with the other
daemons configuration.
Signed-off-by: Sébastien Han <seb@redhat.com>

When a cluster is updated with a different image version, this triggers
a serialized restart of all the pods. Prior to this commit, no safety
check were performed and rook was hoping for the best outcome.

Now before doing restarting a daemon we check it can be restarted. Once
it's restarted we also check we can pursue with the rest of the
platform. For instance, with monitors we check that they are in quorum,
for OSD we check that PGs are clean and for MDS we make sure they are
 all active.

Fixes: https://github.com/rook/rook/issues/2889

Signed-off-by: Sébastien Han <seb@redhat.com>

ceph: schedule mons across failure domains

This fixes https://github.com/rook/rook/issues/3204



Move prometheus Annoation logic before Deployment is initialized,
otherwise, podSpec is not uptodate
Signed-off-by: Dinar Valeev <k0da@opensuse.org>

cmdreporter: fix minor error bug

contrib: add an helper script to perform backports

In the 0.9 release rook converted the v1beta1 CRD resources
to v1 resources. This conversion code is no longer necessary
as we will be using v1 resources going forward.
The code paths will not be triggered anymore, thus
removing the dead code.
Signed-off-by: travisn <tnielsen@redhat.com>

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>

github: re-add default template for new issues

ceph manifests: update upgrade-from yaml files

k8sutil: 1. Use label max length constant from k8s, 2. Add missing docstrings in pod.go

Add missing docstrings for container image
functions in pod.go, noting the behavior
when getting a container image from a pod
which only has a single container defined.
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>