Commit Graph

107090 Commits

Author SHA1 Message Date
Sage Weil
68d3c86106 Merge PR #32972 into master
* refs/pull/32972/head:
	python-common/ceph/deployment/translate: use 'prepare' instead of 'batch' for trivial case
	qa/tasks/cephadm: pass short dev name to osd prepare
	mgr/cephadm: fix detection of just-created OSDs
	mgr/cephadm: properly indent raise conditions
	mgr/cephadm: add warning to other orchestrators
	mgr/cephadm: separate acceptance criterias for Devices
	mgr/cephadm: fix typos
	mgr/cephadm: move utils in test/utils.py
	mgr/ssh: increase disk size to 20G
	drivegroups: add support for drivegroups + tests
	mgr/orch_cli: allow multiple drivegroups
	drivegroups: translate disk spec to ceph-volume call

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-01-30 07:01:47 -06:00
Gregory Farnum
ff530327a0
Merge pull request #32980 from ceph/gregsfortytwo-doc-rm-hadoop-2
doc: remove the CephFS-Hadoop instructions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-30 13:40:58 +01:00
Sebastian Wagner
8be1556bcf
Merge pull request #32967 from rhcs-dashboard/ignore-python-common-build
.gitignore: ignore /src/python-common/build

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-01-30 12:57:27 +01:00
Abhishek L
a09dff2244
Merge pull request #30777 from GaryHyg/origin/master
rgw: dmclock: wait until the request is handled.

Reviewed-By: Kefu Chai <kchai@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-01-30 12:55:11 +01:00
Abhishek L
fd2904774d
Merge pull request #32669 from IlsooByun/rel_cb_arg
rgw: release unused callback argument

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-01-30 12:53:56 +01:00
Abhishek L
58b310925b
Merge pull request #32476 from theanalyst/rgw/multisite/user-policy
rgw: user policy: forward write requests to master zone

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-01-30 12:50:02 +01:00
Gregory Farnum
548fc13549 doc: remove the CephFS-Hadoop instructions
These have not aged gracefully, and in particular include instructions
for setting pool size 1 to let Hadoop control the replication — but I've
heard reports of users setting up multiple size-1 pools and then wondering
where their data went when an OSD dies.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-01-30 03:14:45 -08:00
Jan Fajerski
3913835a8f
Merge pull request #32370 from AlexZzz/prometheus_pg_status_bypool
mgr/prometheus: report per-pool pg states
2020-01-30 11:10:34 +01:00
Sage Weil
83edd88d03 Merge PR #32975 into master
* refs/pull/32975/head:
	mon/ConfigMonitor: only propose if leader

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-29 22:11:14 -06:00
Sage Weil
1a32a2a1af Merge PR #32970 into master
* refs/pull/32970/head:
	cephadm: include daemon/unit id in unit name

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-01-29 21:31:14 -06:00
David Zafman
6bb36f862f
Merge pull request #32945 from dzafman/wip-43864
test: Update pg log test for new trimming behavior

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-29 16:03:44 -08:00
Sage Weil
ab64a220cc mon/ConfigMonitor: only propose if leader
Introduced by 08fcf01c04, which activated
this (broken) code path.

Fixes: https://tracker.ceph.com/issues/43892
Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-29 17:18:04 -06:00
Sage Weil
48526b90c7 Merge PR #32623 into master
* refs/pull/32623/head:
	ceph_release: octopus rc 15.1.0

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-29 13:59:53 -06:00
Sébastien Han
2447962111
Merge pull request #32828 from leseb/raw-db-wal
ceph-volume: add db and wal support to raw mode
2020-01-29 20:29:15 +01:00
Sage Weil
614c0eb77e python-common/ceph/deployment/translate: use 'prepare' instead of 'batch' for trivial case
This works around ceph-volumes lack of support for preexisting LVs in the
batch mode.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-29 11:41:41 -06:00
Sage Weil
1634f91f1e cephadm: include daemon/unit id in unit name
This makes the log messages more descriptive.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-29 11:05:51 -06:00
Yuval Lifshitz
2fda1e76d9
Merge pull request #32941 from cbodley/wip-qa-rgw-multisite-pubsub-checkpoint
qa/rgw: multisite checkpoints consider pubsub zone
2020-01-29 18:54:44 +02:00
Sage Weil
1ce0b70cc0 Merge PR #32943 into master
* refs/pull/32943/head:
	qa/tasks/ceph_manager: fix chmod on log dir during pg export copy

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-29 10:34:09 -06:00
Casey Bodley
13de7c1859
Merge pull request #32932 from smanjara/add-bucket-chown-nonmaster
rgw/multisite: warn if bucket chown command is run on non-master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-01-29 10:49:32 -05:00
Alfonso Martínez
1b215488f2 .gitignore: ignore /src/python-common/build
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-01-29 16:41:15 +01:00
Sage Weil
dd1fe6acce
Merge pull request #32957 from tchaikov/wip-mgr-cephadm
mgr/cephadm: init attrs created by settattr()
2020-01-29 08:51:39 -06:00
Sebastian Wagner
276ab76446
Merge pull request #32893 from sebastian-philipp/orchestrator-document-client-usage
mgr/orchestrator: Add doc about how to use OrchestratorClientMixin

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-01-29 15:45:02 +01:00
Aleksei Zakharov
a37cf380ad mgr/grafana: sum pg states for cluster
Also, revert table formatting.

Signed-off-by: Aleksei Zakharov <zaharov@selectel.ru>
2020-01-29 17:28:36 +03:00
Aleksei Zakharov
4eb58f7ccc monitoring/grafana,prometheus: add per-pool pg states support
Signed-off-by: Aleksei Zakharov <zaharov@selectel.ru>
2020-01-29 17:28:36 +03:00
Aleksei Zakharov
96bd77d509 mgr/prometheus: pg counters per pool descriptions
Update pg metrics descriptions to show that we have per
pool stats now.

Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
2020-01-29 17:26:20 +03:00
Aleksei Zakharov
41e7d20d2c mgr/prometheus: pg count by pool
If we have all other stats by pool, it's better to have total
count by pool too. We always can sum() all of total, but it's
hard to count by-pool total.

Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
2020-01-29 17:26:20 +03:00
Aleksei Zakharov
8fb16e43e6 mgr/prometheus: report per-pool pg states
This commit adds per-pool pg states metrics
with unique 'pool_id' label.

Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
2020-01-29 17:26:20 +03:00
Sage Weil
4fa9397841 Merge PR #32939 into master
* refs/pull/32939/head:
	mon/MgrMonitor.cc: add always_on_modules to the output of "ceph mgr module ls"

Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-29 07:46:38 -06:00
Kefu Chai
e39f34f9a9 mgr/cephadm: drop "type: ignore" annotations
since these options are already initialized in the constructor, there is
no need to ignore their types when checking those lines using mypy

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-29 21:40:49 +08:00
Kefu Chai
7c8e3e88d1 mgr/cephadm: init attrs created by settattr()
to address the test falure caused by mypy:
```
mypy run-test: commands[0] | mypy --config-file=../../mypy.ini ansible/module.py cephadm/module.py mgr_module.py mgr_util.py orchestrator.py orchestrator_cli/module.py rook/module.py
test_orchestrator/module.py
cephadm/module.py: note: In member "_check_for_strays" of class "CephadmOrchestrator":
cephadm/module.py:596: error: "CephadmOrchestrator" has no attribute "warn_on_stray_hosts"
cephadm/module.py:596: error: "CephadmOrchestrator" has no attribute "warn_on_stray_services"
cephadm/module.py:599: error: "CephadmOrchestrator" has no attribute "warn_on_stray_services"
Found 3 errors in 1 file (checked 8 source files)
```
see also https://github.com/python/mypy/issues/5719

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-29 21:40:49 +08:00
Shilpa Jagannath
c85b29e533 rgw: add bucket_chown to list of commands that are recommended to be run on master zone.
Fixes: https://tracker.ceph.com/issues/43884

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-01-29 18:54:15 +05:30
Sage Weil
115148e126 Merge PR #32823 into master
* refs/pull/32823/head:
	common/bl: fix memory corruption in bufferlist::claim_append()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-29 07:24:02 -06:00
Sage Weil
c227b5d831 Merge PR #32940 into master
* refs/pull/32940/head:
	qa: remove rados/basic/tasks/rgw_snaps.yml

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-01-29 07:23:34 -06:00
Sage Weil
2ea06e0203 Merge PR #32942 into master
* refs/pull/32942/head:
	osd: dispatch_context and queue split finish on early bail-out

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-01-29 07:22:58 -06:00
Daniel Gryniewicz
c1b88a9c15
Merge pull request #31879 from joke-lee/rgw-sts-post-upload-auth
rgw: use STSEngine::authenticate when post upload with x_amz_security_token
2020-01-29 07:48:10 -05:00
Daniel Gryniewicz
e61d1322fe
Merge pull request #31214 from IlsooByun/lc_stale_head
rgw: prevent LC from reading stale head when transitioning object
2020-01-29 07:47:30 -05:00
Sébastien Han
c8f23c7182 ceph-volume: add db and wal support to raw mode
Using the raw mode, the OSD preparation can now be called with extra
arguments to specify a block device for either rocksdb db or wal.

The code expects the device to be available and no validation will be
performed. Also, the entire block device will be consumed.

The goal is run this on Kubernetes with Rook and OSD are running on PVC.
Users will request PVC from a certain size, the request will be
acknowledged and a block will be created, then attached to the machine.
Later on consumed by Rook's preparation job.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-01-29 12:13:06 +01:00
Jan Fajerski
ec2e53f59c
Merge pull request #31700 from rishabh-d-dave/cv-list-using-dash-S
ceph-volume: refactor listing.py
2020-01-29 11:58:10 +01:00
Abhishek L
b92cbad060
Merge pull request #32808 from theanalyst/rgw/test-iam-bits
test: modify iam tests to use a function to set bits

Reviewed-By: Adam Emersen <aemerson@redhat.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-01-29 11:19:51 +01:00
Lenz Grimmer
0e158c7fc2
mgr/dashboard: smart: add support for NVMe (#32708)
mgr/dashboard: smart: add support for NVMe

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-01-29 08:51:37 +00:00
Jason Dillaman
3451f8dabb
Merge pull request #32938 from trociny/wip-fix-freebsd-build
librbd: fix build on freebsd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-01-28 18:57:20 -05:00
David Zafman
e18519ad09 test: Update pg log test for new trimming behavior
Fixes: https://tracker.ceph.com/issues/43864

Signed-off-by: David Zafman <dzafman@redhat.com>
2020-01-28 15:23:45 -08:00
Casey Bodley
9003006196 qa/rgw: multisite checkpoints consider pubsub zone
pubsub zones are configured to only sync_from the master zone, so bucket
checkpoints should ignore its sync status with respect to other zones

Fixes: https://tracker.ceph.com/issues/43768

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-28 15:32:58 -05:00
Sage Weil
f026a1c9f6 qa/tasks/cephadm: pass short dev name to osd prepare
Zap needs a full path, but create/prepare needs the VG/LV
only if it is an existing LV.

We'll make c-v more friendly later.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-28 14:21:53 -06:00
Sage Weil
36f5e3b115 Merge PR #32502 into master
* refs/pull/32502/head:
	pybind/mgr: Add ceph_module.pyi to improve type checking

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-01-28 14:09:55 -06:00
Sage Weil
b7456a3bb6 Merge PR #32903 into master
* refs/pull/32903/head:
	mgr/devicehealth: fix telemetry stops sending device reports after 48 hours

Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-28 14:09:29 -06:00
Sage Weil
ba604080a0 Merge PR #32365 into master
* refs/pull/32365/head:
	mon: Don't put session during feature change

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-28 14:08:51 -06:00
Sage Weil
3664ca0295 Merge PR #32897 into master
* refs/pull/32897/head:
	osd/PG: do not leak cluster message when there's no con

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-28 14:07:41 -06:00
Casey Bodley
6eb1d5f0f1
Merge pull request #31686 from yehudasa/wip-rgw-sync-bucket-6
rgw: bucket granularity sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-01-28 15:07:00 -05:00
Sage Weil
9a4dd1fb3d qa/tasks/ceph_manager: fix chmod on log dir during pg export copy
With cephadm, we should chmod both /var/log/ceph and /var/log/ceph/$fsid.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-28 13:54:18 -06:00