Fixes: http://tracker.ceph.com/issues/24436
To fully support the role based authentication/authorization system it is necessary to replace the RGW proxy controller by separate controllers for RGW user and bucket.
Signed-off-by: Volker Theile <vtheile@suse.com>
Avoid sporadic failures in combination with msgr-failures/many.yaml,
where assert_locked() might take over 10 seconds.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* refs/pull/22596/head:
os/bluestore: use vector instead of set for zombies
os/bluestore: reuse zombie OpSequencers by collection id
qa/suites/rados/objecstore/backends/objectstore: capture coredumps
os/bluestore: more debug output
os/bluestore: print cnode from _open_collections
os/bluestore: print cnode on fsck
qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
What we actually want is a purge, not a destroy. Destroy leaves the OSD
ID in used and allows it to be recreated. What ceph-volume wants is to
purge all trace of the failed OSD setup.
Signed-off-by: Sage Weil <sage@redhat.com>
The RGW API user id (set via 'ceph dashboard set-rgw-api-user-id <xxx>') is optional but the user ID is required internally for some situations. Because of that the user ID is requested via a RGW Admin Ops API call if it is not configured via CLI.
Signed-off-by: Volker Theile <vtheile@suse.com>
Add ability to list, set and unset cluster-wide OSD flags.
Flags can be listed and changed through the `/api/osd/flags` API
resource. By using a GET request, the list is retrieved. By using a PUT
request, the flags are updated (all at once). Flags not contained in the
data of the PUT are removed, additional once are added. Note that the
PUT requests require a JSON body with the data contained as value of the
'flags' key like so:
{"flags": ["flag1", "flag2", ...]}
Fixes: http://tracker.ceph.com/issues/24056
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
* refs/pull/22554/head:
qa/standalone/ceph-helpers.sh: Fixing comment for wait_for_health()
tests: Protecting rados bench against endless loop
qa/standalone/ceph-helpers.sh: Defining custom timeout for wait_for_clean()
Reviewed-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
mgr/dashboard: Add help menu entry
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
If the cluster dies during the rados bench, the maximum running time is
no more considered and all emitted aios are pending.
rados bench never quits and the global testing timeout (3600 sec : 1
hour) have to be reach to get a failure.
This situation is dramatic for a background test or a CI run as it locks
the whole job for too long for an event that will never occurs.
This ideal solution would be having 'rados bench' considering a failure
once the timeout is reached when aios are pending.
A possible workaround here is to put use the system command 'timeout'
before calling rados bench and fail if rados didn't completed on time.
To avoid side effects, this patch is doubling rados timeout. If rados
didn't completed after twice the expected time, it have to fail to avoid
locking the whole testing job.
Please find below the way it worked on a real test case.
We can see no IO after t>2 but despite timeout=4 the bench continue.
Thanks to this patch, the bench is stopped at t=8 and return 1.
5: /home/erwan/ceph/src/test/smoke.sh:55: TEST_multimon: timeout 8 rados -p foo bench 4 write -b 4096 --no-cleanup
5: hints = 1
5: Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 4 seconds or 0 objects
5: Object prefix: benchmark_data_mr-meeseeks_184960
5: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s)
5: 0 0 0 0 0 0 - 0
5: 1 16 1144 1128 4.40538 4.40625 0.00412965 0.0141116
5: 2 16 2147 2131 4.16134 3.91797 0.00985654 0.0109079
5: 3 16 2147 2131 2.77424 0 - 0.0109079
5: 4 16 2147 2131 2.0807 0 - 0.0109079
5: 5 16 2147 2131 1.66456 0 - 0.0109079
5: 6 16 2147 2131 1.38714 0 - 0.0109079
5: 7 16 2147 2131 1.18897 0 - 0.0109079
5: /home/erwan/ceph/src/test/smoke.sh:55: TEST_multimon: return 1
5: /home/erwan/ceph/src/test/smoke.sh:18: run: return 1
Signed-off-by: Erwan Velu <erwan@redhat.com>
The wait_for_clean() is using the default timeout aka 300sec = 5mn.
wait_for_clean() is trying to find a clean status within that timeout
_or_ reset its counter if any progress got made in between loops.
In a case where the cluster is sane, the recovery should be made in
shorter than 5mn but it the cluster died, waiting for 5mn for nothing is
unefficient.
This patch is about defining a custom timeout for a wait_for_clean() not
to wait much more that 1m30 (90sec). If no progress is made in that
period, there is very few chance this will read the a valid state
anyhow.
Signed-off-by: Erwan Velu <erwan@redhat.com>
This commit adds the config options stored by the MON database to the
configuration documentation page.
One can filter for these config options by setting the 'Source' filter
to 'mon' on the configuration documentation page.
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
Since we only support Jewel and later releases, which both support
object-map and fast-diff, enabling/disabling object-map should always
enable/disable fast-diff.
Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>
mon: add 'osd destroy-new' command that only destroys NEW osd slots
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>
ceph-volume may run into a problem and want to clean up, but we do not
want to give it blanket access to the 'osd destroy' command. Instead,
make an 'osd destroy-new' that can only create new OSDs (ones that are
in the process of being created but have never booted yet).
Signed-off-by: Sage Weil <sage@redhat.com>
If there is a stray clone (one that does not appear in the SnapSet) and
we do any sort of recovery on it the OSD will crash. Log an error instead
but continue.
This addresses a problem where a cluster has both (1) an unexpected clone
and (2) the clone is not present on all replicas. Doing repair on that
PG will both not fix the unexpected clone and also cause the remaining
OSDs to crash trying to recover it.
Include a test.
Fixes: https://tracker.ceph.com/issues/24396
Signed-off-by: Sage Weil <sage@redhat.com>
In EC recovery read, if the object's attrs read failed or with errors, we erase the attrs we have read and
try to read it again from left shards. This will make the primary osd get the object's attrs correct and
avoid assert.
Signed-off-by: xiaofei cui <cuixiaofei@sangfor.com>
This reverts commit 886606bfd7.
Signed-off-by: David Zafman <dzafman@redhat.com>
Conflicts:
qa/standalone/scrub/osd-scrub-repair.sh (manually made equivalent changes)
mgr/dashboard: controller infrastructure refactor and new features
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
pg sends pg-stats to mgr every 5 seconds, so we cannot check for the
number of pgs right after creating the pool, at that moment, the number
of pgs could be 0, that's why manger.wait_for_clean() returns right
away, and leaves us with 0 pgs: the pgs serving the pool are still being
created. that's why `manager.get_num_active_clean()` returns `0`
sometimes. so, we should force osd to flush their stats to mgr, and wait
until the pg stats converages.
Fixes: http://tracker.ceph.com/issues/24321
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/22230/head:
test/cli-integration/rbd: fixed unit formatting
qa/workunits/rbd: fix unit formatting tests
librbd: clones should not inherit implicit feature bits from parent
Reviewed-by: Mykola Golub <mgolub@suse.com>
mon/MgrMonitor: change 'unresponsive' message to info level
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
We generate a MGR_DOWN health warning at the appropriate points; having
this at WRN level just triggers failed teuthology runs but doesn't much
value for the user.
Clear out teuthology whitelisting for this message.
Fixes: http://tracker.ceph.com/issues/24222
Signed-off-by: Sage Weil <sage@redhat.com>
rgw: Do not modify email if argument is not set
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
If the bucket is empty or does not have weight-set weights yet, avoid
crashing when populating the parent bucket.
Fixes: http://tracker.ceph.com/issues/23386
Signed-off-by: Sage Weil <sage@redhat.com>
* so we just rely on a single build system instead of two of them, the
other place we use cmake is cmake/modules/BuildRockDB.cmake.
* disable gflags when building rocksdb, it's optional and does not help
in the sense of testing librados support.
* disable prompts when installing on debian, to silence warnings like:
debconf: unable to initialize frontend: Dialog
* drop --force-yes option, as it is deprecated, and is replaced with
--allow-downgrades, --allow-remove-essential,
--allow-change-held-packages, but none of them apply in our case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Just make caller happy. there is no easy way to support timeout.
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/24053
the distro specified by random-distro$ will be overwrited by the one
specfied by valgrind.yaml. and teuthology-suite will give
KeyError: '16.04 not a centos version or codename'
when scheduling a suite involving the facets above. also, i think it's
of not much value to run valgrind/lockdep with different distros.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The RGW Admin OPS API overwrites the users email either it is not set via argument, e.g. when executing /{admin}/user?uid=test5&suspended=true.
Signed-off-by: Volker Theile <vtheile@suse.com>
