The changes to the way EC/ReplicatedBackend communicate read
t showerrors had a side effect of making first eio on the object in
TEST_rados_get_subread_eio_shard_[01] repair itself depending
on the timing of the killed osd recovering. The test should
be improved to actually test that behavior at some point.
Signed-off-by: Samuel Just <sjust@redhat.com>
We currently do not expose perf counters in the restful module but this
is a useful piece of information to be exposed by the restful module.
Since we already have a function to gather all the information about the
perf counters, we can make this easily accessible through the restful
API.
This also supports filtering by daemons through regexps.
Signed-off-by: Boris Ranto <branto@redhat.com>
Now it's possible to decrease the pg's of an existing pool. The decrease
will take place over time like the increase. As for both no notification
will be created when this is done (issue #39482).
Fixes: https://tracker.ceph.com/issues/35917
Signed-off-by: Stephan Müller <smueller@suse.com>
* refs/pull/27655/head:
common/options: flag misc ms_* options STARTUP
common/options: flag misc options STARTUP
common/options: mark cluster log options with FLAG_RUNTIME, use get_val
common/options: mark a bunch of options with FLAG_STARTUP
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Use OSD_POOL_PRIORITY_MAX and OSD_POOL_PRIORITY_MIN constants
Scale legacy priorities if exceeds maximum
Signed-off-by: David Zafman <dzafman@redhat.com>
Case 1: A more recent update exists
Case 2: The first entry in the divergent sequence is a create
Case 3 NOT TESTED - Ohject currently missing
Case 4: We can rollback all of the entries
Case 5: We cannot rollback at least 1 of the entries
Support starting OSDs even when "noup" is set (don't wait for up).
Move create_ec_pool() to ceph-helpers.sh
Fixes: https://tracker.ceph.com/issues/39162
Signed-off-by: David Zafman <dzafman@redhat.com>
This was used for API backwards compatibility testing, but now that
the C++ API will not remain stable, it serves no purpose.
Fixes: http://tracker.ceph.com/issues/39072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Also tweaked the existing writeback cache test permutations to
specify the use of the writeback cache policy.
Fixes: http://tracker.ceph.com/issues/39386
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
stop command can be used to force stopping a specified osd daemon, e.g.,
you don't have to pre-figure out where it located.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
* refs/pull/27481/head:
qa: add a method to test if a file can be created
qa: add wrappers for vstart's run() with check_status as False
qa: add a method to create an empty file with any user
qa: add stdin parameter to run_shell
qa: command to be ran as a different user should be a single string
qa: looks for quotes in Python shell and get rid of them
qa: move checks on command arguments to a different method
qa: accept args as strings
qa: add a method to run given command as root user
qa: add a method to run a command as a different user
qa: allow keeping sudo in command arguments
qa: make execution abortion optional for command ran through run_shell()
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Too often in tests it is required to run a command that we want to (or
will probably) fail so that we can check if it failed for expected
reason(s) (eg. a testcase where command must fail with "Permission
denied"). Therefore, add few wrapper methods for vstart_runner.py's
run() with check_status's default value as False so that we don't need
to pass check_status's value at every call to run() or it's other
wrappers.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/27564/head:
ceph-objectstore-tool: Rename dump-import to dump-export
osd: Fix fast path to use version bumped value
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
If user specifies dump-import it will still work, but isn't
in the usage that way.
Fixes: http://tracker.ceph.com/issues/39284
Signed-off-by: David Zafman <dzafman@redhat.com>
Since quotes are not a token for Python's shell simulation (unlike
Bash), raise a RuntimeError for them and add an exception for Python
command since it quotes to define a program on commandline.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Till now the run() method in vstart_runner.py accepts commands only as
a (Python) list. It is very inconvenient to write commands as
['setfacl', '-m', 'user:someuser:rwx', 'somedir/somefile'] when they
can be simply be written as
'setfacl -m user:someuser:rwx somedir/somefile'. Also, converting the
latter form to former one is exceedingly simple. So writing commands as
strings should be allowed.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
EPEL7 has switched over to python3.6 as the main python3. and we started
packaging python bindings for python3.6 since
https://github.com/ceph/ceph-build/pull/1283
Signed-off-by: Kefu Chai <kchai@redhat.com>
The ceph cli tool checks for the presence of the variable, not its value.
Fixes: http://tracker.ceph.com/issues/38359
Signed-off-by: Sage Weil <sage@redhat.com>
Change run_osd() to default objectstore bluestore
Use run_osd_filestore() to use the non-default objectstore
Fix inject_eio to handle any objectstore if config prefixed with type
Remaining tests using filestore:
osd-pool-create.sh TEST_pool_create_rep_expected_num_objects
Test filestore directory creation
qa/standalone/osd/osd-dup.sh TEST_filestore_to_bluestore
Obvious
qa/standalone/osd/osd-rep-recov-eio.sh TEST_rep_read_unfound
Requires data digest in object info
qa/standalone/scrub/osd-scrub-repair.sh multiple tests
Erasure code pools append mode for filestore is tested
qa/standalone/special/ceph_objectstore_tool.py
Test code verifies COT by directly examining filestore contents
Fixes: https://tracker.ceph.com/issues/39162
Signed-off-by: David Zafman <dzafman@redhat.com>
sudo cannot be omitted from the given command's arguments, when running
passwd, chown and, specially, when sudo is used for running the given
command as different user.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Instead of looking at the number of threads (used by the simple messenger) to
judge the coming and going of connections, use the (async) messenger perf
counters.
Plus some other minor improvements.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Direct upgrades from Luminous to Octopus are not supported. These snap format
upgrade tests are now only going to be run in the mimic/nautilus branches.
Fixes: http://tracker.ceph.com/issues/39020
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/27291/head:
qa/suites/upgrade/nautilus-x-singleton: upgrade mgrs early
qa/suites/upgrade/mimic-x-singleton: upgrade mgrs later
qa/suites: add upgrade/nautilus-x-singleton and symlink from rados/upgrade
qa/suites: move rados/upgrade/mimic-x-singleton to upgrade/ and symlink
Reviewed-by: Sage Weil <sage@redhat.com>
The mgrmap may change when the mons upgrade, causing the mgr daemons to
respawn. Restart mgr daemons before mons to avoid a mismatch between
the running version and the ondisk binary.
Signed-off-by: Sage Weil <sage@redhat.com>
The mgrmap may change when the mons upgrade, causing the mgr daemons to
respawn. Delay the upgrade until later so that the ondisk binary does
not change out from underneath them.
Signed-off-by: Sage Weil <sage@redhat.com>
* refs/pull/27265/head:
valgrind.supp: add clauses for uninitialised gcm.Xi in OpenSSL.
src/valgrind.supp: replace with the teuthology's whitelist.
Reviewed-by: Sage Weil <sage@redhat.com>
Helpers to decide when it is safe to stop a mon, add a mon that is
not started, or remove a mon. (Adding and start a mon would always
be safe, but it takes time to sync, so it's not really possible to do
quickly.)
Signed-off-by: Sage Weil <sage@redhat.com>
This change drops src/valgrind.supp file and transplates
the current teuthology's whitelist into qa/valgrind.supp.
This commit has sibiling in the teuthology repo dropping
the file there.
The rationale behind the entire change is duplication of these
files between two repos, their inconsistencies and confusion.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
- don't specify ceph.py options in the ceph.py
- instead, specify them in the per-version facet
Note that we don't currently have a way to do v2 only for the mon IPs, so
in the v2only cases, we are still binding the mons to v1.
Signed-off-by: Sage Weil <sage@redhat.com>
mgr/dashboard: Add separate option to config SSL port
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
GET /{admin}/user?format=json HTTP/1.1
Host: {fqdn}
This api gets the information is incomplete relative to radosgw-admin user info --uid xxxx
This modification will change the information returned by the three calls :
RGWUserAdminOp_User::info
RGWUserAdminOp_User::create
RGWUserAdminOp_User::modify
Signed-off-by: Li Shuhao <lishuhao@unitedstack.com>
* refs/pull/27169/head:
common/config: parse --default-$option as a default value
Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sometimes it is useful to specify an alternative default value for an
option via the command line such that it has a lower priority than the
mon config database, config file, the rest of the command line, or the
environment.
Signed-off-by: Sage Weil <sage@redhat.com>
Leave repair pg state on until recovery finishes or a new scrub starts
Fixes: http://tracker.ceph.com/issues/38616
Signed-off-by: David Zafman <dzafman@redhat.com>
With discard_granularity set to alloc_size, we no longer get object
size alignment from blk_bio_discard_split().
This assumption is pretty deeply ingrained in krbd_data_pool.sh, so
make it explicit. For krbd_fallocate.sh, just fix the expectation.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
The test case is issuing discards that span two objects: the tail of
the first object is truncated, the head of the second object is zeroed.
These discards aren't serial, so there is a race:
discard i ~ i + 1: truncate i, zero i + 1
discard i + 1 ~ i + 2: truncate i + 1, zero i + 2
can be executed as
truncate i + 1, zero i + 2, truncate i, zero i + 1
For object i + 1, the sequence ends up being truncate tail, then zero
head. This zero op is munged to truncate on the OSD, resulting in size
0 instead of OBJECT_SIZE / 2.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This way it can be used to fast cancel/undo the last command.
Also make the tip message a litter bit nicer..
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Allow auto_repair for replicated bluestore pools
Regular scrub within auto repair parameters will trigger deep scrub
New state failed_repair if PG repair attempt could not fix everything
Set failed_repair if not possible to repair anything
Fixes: http://tracker.ceph.com/issues/38616
Signed-off-by: David Zafman <dzafman@redhat.com>
Fix for argument handling of create_ec_pool()
Always pass a value for allow_overwrites for consistency
Caused by: 3ca750d41d
Signed-off-by: David Zafman <dzafman@redhat.com>
- start with msgr2 enabled (defaults)
- run nautilus branch for workunits
- drop msgr2 enable step at teh end
- add octopus placeholder (although it is empty for now)
Signed-off-by: Sage Weil <sage@redhat.com>
* refs/pull/27112/head:
qa/suites: do not test luminous-x upgrade path
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
qa/objectstore: test with reduced value of osd_memory_target
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
in this change,
* suites/rados/upgrade: luminous-x-singleton => mimic-x-singleton
* suites/upgrade: luminous-x => nautilus-x
we support upgrade from n to n+2 release. otherwise monitor refuses to
do so:
mon.a@-1(probing) e1 current monmap has min_mon_release 15 (luminous)
which is >2 releases older than me 15 (octopus), stopping.
Fixes: https://tracker.ceph.com/issues/38845
Signed-off-by: Kefu Chai <kchai@redhat.com>
The changes of this PR were done while trying to fix the failing test. The problem has been solved by another PR, but the changes are worth to be integrated because they help debugging and an additional test has been added (check if previously active manager is listed as standby).
Signed-off-by: Volker Theile <vtheile@suse.com>
