* refs/pull/30217/head:
crimson: common/admin_socket kludge so that it builds
mon/MonClient: fix sending mon command to a specific rank
src/.gitignore: ignore .tox
mon/MonClient: interpret numeric mon target name as rank
mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
common/ceph_context: make 'config unset' idempotent
qa/tasks/dump_stuck: mon.a, not mon.0
qa/suites/rados/singleton/all/admin-socket: fix test
common/config: EPERM setting config option after startup
qa/workunits/cephtool/test.sh: fix tell output error check
common/admin_socket: pass Formatter from generic infrastructure
common/admin_socket: pass ostream to call() for error output
os/bluestore: fix asok hook return value
rgw: fix asok return value
common/ceph_context: return error code from asok commands
test/pybind/test_rados: fix accidental mon tell test
mon: print entity_name along with caps to debug log
PendingReleaseNotes: notes about asok changes
mgr/MgrClient: empty target string for 'tell' means active mgr
common/admin_socket: report error code as part of output string
osd: change trigger_[deep_]scrub tommands to a pg tell command
osd: remove old command workqueue, threadpool
osd: drop MMonCommand handling
osdc/Objecter: resend OSD tell commands on EAGAIN
osd: route tell commands to asok; migrate commands
osd: use unique_ptr<Formatter> for asok_command
common/ceph_context: add generic asok 'injectargs'
common/admin_socket: allow dup prefixes
common/admin_socket: refactor with sync and async execute_command variants
common/admin_socket: pass input bufferlist
osd: transition to call_async() for asok
common/admin_socket: support alternative call_async()
mon/MonClient: send tell commands out of band via MCommand
mon: accept tell commands via MCommand and send them to asok handler
common/admin_socket: return int from hook call()
mgr/DaemonServer: route MCommand (for octopus+) to asok commands
do not use 'ceph tell mgr'
pybind/ceph_argparse: disambiguate mgr tell and CLI commands
ceph: make 'ceph tell mgr.*' send to the active mgr
ceph: send 'ceph tell mgr.X' to the right mgr
librados: add rados_mgr_command_target
mgr/MgrClient: add start_command variant that takes a target
common/admin_socket: drop unregister_command(); use per-hook variant
common/admin_socket: drop explicit prefix arg to register_command
common/admin_socket: simplify command routing
common/admin_socket: add ability to process MCommand via asok queue
common/admin_socket: pass cmdvec to execute_command
common/admin_socket: use pipe for general wakeup
include/compat: add flags arg to pipe_cloexec
common/admin_socket: drop unused args
Reviewed-by: Neha Ojha <nojha@redhat.com>
Therefore the v1 parent spec should not attempt to populate the
namespace.
Fixes: https://tracker.ceph.com/issues/41938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
We can't set the filestore setting because filestore isn't active and so
the option isn't observed, so it isn't changeable.
Signed-off-by: Sage Weil <sage@redhat.com>
This is cleaner. All users are currently standalone tests; updated.
It also means that *all* commands that have a name=pgid arg are pg tell
commands.
Signed-off-by: Sage Weil <sage@redhat.com>
Increasing osd_object_clean_region_max_num_intervals to track more
clean regions, resulting in more partial recovery.
Signed-off-by: Neha Ojha <nojha@redhat.com>
* refs/pull/30627/head:
qa/suites/rados/verify: whitelist MON_DOWN when using valgrind
qa/suits/rados/basic/tasks/rados_api_tests: pgs can go degraded
Reviewed-by: Neha Ojha <nojha@redhat.com>
Restructure SSE-KMS tests which now has 3 scenarios for each KMS
backend: Barbican, Vault, and testing (keys stored in ceph.conf).
Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
* refs/pull/30525/head:
qa/tasks/ceph.conf.template: disable power-of-2 warning
qa/standalone/mon/health-mute: use power of 2 for pg_num
osd/OSDMap: remove remaining g_conf() usage
PendingReleaseNotes: add note for 14.2.5 so we can backport this
osd/OSDMap: health alert for non-power-of-two pg_num
Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Add a new command-line switch to allow it to use the kernel client
instead, and add all of the machinery to handle local kcephfs mounts.
Document this in the developer guide, along with the appropriate scary
warnings about using this on a machine that you care about. While we're
in there, also correct a typo about FUSE configuration.
Fixes: https://tracker.ceph.com/issues/41910
Signed-off-by: Jeff Layton <jlayton@redhat.com>
* refs/pull/30297/head:
test_cephfs_shell: add note on how to run this testsuite
cephfs-shell: rewrite help text for put and get commands
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/29906/head:
vstart_runner: name booleans for options differently
qa/vstart_runner.py: add an option to remove old log
qa/vstart_runner.py: make log initialization code reusable
qa/vstart_runner.py: make printing of stdout of ps optional
qa/vstart_runner.py: add timeout for vstart.sh and stop.sh
qa/vstart_runner.py: add an option to teardown cluster
Reviewed-by: Jeff Layton <jlayton@redhat.com>
* refs/pull/28834/head:
libcephfs: Add test for lazyio via libcephfs
libcephfs: Add lazyio_propogate and lazyio_synchronize methods to libcephfs
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
* refs/pull/30462/head:
qa: have kclient tests use new mount.ceph functionality
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
mgr/dashboard: Adapt test_mgr_module QA test to latest controller changes
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
This achieves 2 things: makes it more understandable for the reader that
the boolean variable is an option and reads less like a method name.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Add an option "--clear-old-log" to get rid of log entries before making
entries for current batch of tests.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
stdout of ps command is generally pretty huge which makes it harder to
interpret logs. Don't print it by default and add "--log-ps-output" to
enable printing it.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Override default timeout for execution of vstart.sh and stop.sh since
neither scripts take more than 100 seconds normally.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Add "--teardown" option so that vstart_runner.py tears down Ceph
cluster automatically after the batch of tests has completed running.
Also delete build/dev and build/out directories, after running stop.sh.
Update the doc for vstart_runner.py as well.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/29824/head:
qa: whitelist new FS_INLINE_DATA_DEPRECATED health warning
mds: add a HEALTH_WARN message when inline_data is enabled
mds: log a warning message when mds is started on an fs with inline_data
mon: deprecate CephFS inline_data support
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Version 3.6.0 is not available so the version
of maven is being bumped up to most recent
version (3.6.2).
Signed-off-by: Ali Maredia <amaredia@redhat.com>
* refs/pull/30475/head:
qa/standalone/ceph-helpers: default pg autoscale mode off for standalone
os/bluestore: fix objectstore_blackhole read-after-write
test,misc: do not specify pg_num per pool
mgr/volumes: do not specify pg_num
pybind/ceph_volume_client: do not specify pg_num for new pools
doc: remove all pg_num arguments to 'osd pool create'
mon: do not require pg_num to 'osd pool create'
common: default pg_autoscale_mode=on for new pools
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
mgr/dashboard: fix error when linking a bucket to a tenanted user
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
On Debian-based distros, hexdump(1) comes from FreeBSD in bsdmainutils
package (other distros ship a Linux version from util-linux). It turns
out that '-s skip' for anything but a regular file has been broken for
ages in FreeBSD -- instead of calling fseek(), it consumed the input
byte by byte with getchar(). This was fixed in commit 7cd5ae492557
"hexdump(1): Speed up -s flag on devices" last year, but it hasn't made
it to the Debian world yet.
bsdmainutils_9.0.6ubuntu3 in xenial had a custom patch for that, but
it's no longer there in bsdmainutils_11.1.2ubuntu1 in bionic. hexdump
-s <large offset> is now a fancy infinite loop!
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Now that the mount helper has the ability to discover mon addrs and
can scrape secrets from the keyring, take advantage of it and simplify
the KernelMount class.
Fixes: https://tracker.ceph.com/issues/41892
Signed-off-by: Jeff Layton <jlayton@redhat.com>
The plan is to start deprecating this feature now so that we can remove
it in a future release. Change it to require the
--yes-i-really-really-mean-it flag, and to emit a custom
warning when that isn't specified.
For now, we leave the testing in place since we do want to be notified
if something breaks before we're ready to rip it out completely.
Fixes: https://tracker.ceph.com/issues/41311
Signed-off-by: Jeff Layton <jlayton@redhat.com>
- Prefix bucket name with '/' when linking bucket to a tenanted user
- Add qa tests
Fixes: https://tracker.ceph.com/issues/41611
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
rbd rounds off and ignores discards that are smaller than alloc_size,
but the page cache is invalidated according to the original offset and
length, before calling into the driver. For a 512~4193792 discard, all
pages except for the first will be invalidated, but the first page may
remain cached after being partially zeroed out. This results in an
inconsistency when alloc_size != 512. E.g. for 64k:
0 512 4k 64k 4M
| | | | |
page cache: xxxxxxxx00000000xxxxxxxx0000000000000000
RADOS: xxxxxxxxxxxxxxxxxxxxxxxx0000000000000000
Given that these tests asserts on-disk state, we need to read directly
from disk. krbd_discard_512b.t and krbd_zeroout.t are fine -- there is
no rounding there.
Fixes: https://tracker.ceph.com/issues/41753
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
The problem was that the CephFS details component was entirely responsible for
managing all other tabs besides it's own. Besides that many API requests
were triggered, which now has been reduced to just one request which is
fired inside the tabs component.
I added an endpoint which contains all adaptations that were done in the
frontend before. This limits the request size and increases the
performance (at least a bit).
Fixes: https://tracker.ceph.com/issues/41372
Signed-off-by: Stephan Müller <smueller@suse.com>
rgw: Allow admin APIs that write metadata to be executed first on the mast…
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
* refs/pull/28560/head:
cephfs-shell: handle du's arguments elsewhere outside do_du()
cephfs-shell: reuse code
cephfs-shell: rewrite call to perror in do_du
pybind/cephfs: define variable for hexcode used in stat()
test_cephfs_shell: test cephfs-shell command at invocation
cephfs-shell: refactor do_du()
cephfs-shell: option -r is not for reverse
cephfs-shell: extend to_bytes()
test_cephfs_shell: test du with no args
test_cephfs_shell: test du with multiple paths in args
test_cephfs_shell: test behaviour of "du -r"
test_cephfs_shell: test du's output for softlinks
qa/cephfs: add convenience method lstat()
qa/cephfs: add option to make stat() unfollow symlinks
test_cephfs_shell: test du's output for hardlinks
test_cephfs_shell: test du's output for directories
test_cephfs_shell: test du's output for regular files
test_cephfs_shell: add a method to get command output
test_cephfs_shell: allow cmd as list too
test_cephfs_shell: rename and rewrite _cephfs_shell()
test_cephfs_shell: copy humanize() from cephfs-shell
cephfs-shell: print disk usage for non-directory files too
pybind/cephfs: add method that stats symlinks without following
cephfs-shell: Fix 'du' command error
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
the tox command fails to invoke tools/tox_install.sh because it passes
a stale url for its extra requirements. patch that url in tox.ini
Signed-off-by: Casey Bodley <cbodley@redhat.com>
works around a VersionConflict error by including the requirements on
jsonschema when installing python-openstackclient
also pin the version of python-openstackclient, because newer clients
don't seem to recognize --os-url
Signed-off-by: Casey Bodley <cbodley@redhat.com>
downloads the keystone repository first, because we need
keystone/bindep.txt to discover the binary dependencies
Signed-off-by: Casey Bodley <cbodley@redhat.com>
* refs/pull/28702/head:
qa: update json format from session listing
mds: recall caps from quiescent sessions
mds: use Session::dump method uniformly
mds: use auto to deduce iterator type
mds: simplify method definition
mds: remove useless debug message
mds: use const get_request_count
mds: use session_info_t socket inst for addr
mds: refactor session lookup
mds: add explicit trim flag
mds: alphabetize tracked config keys
common: provide method to get half-life
common: correct variable name
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Let run_cephfs_shell_cmd() return the object instead of output in string
format so that stderr, return value, etc. of the command executed
remain available. And add a new method that returns only output of the
CephFS shell command executed.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Convert cmd to string if it is passed as list to
run_cephfs_shell_cmd(). Allowing cmd to be list would be nice since
elsewhere, in tests, command to be executed must be a list of arguments
rather than string of arguments separated by spaces.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Rename _cephfs_shell() to run_cephfs_shell_cmd() and add a default
client for run_cephfs_shell_cmd() since most of tests in this
testsuite will need only one client.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Copy the method named humanize from cephfs-shell so that output can be
compared. Unfortunately, importing this method isn't possible since the
dash in "cephfs-shell" leads to an syntax error.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
AFAICS this has been the case for basically forever. Not sure why/how
the mon_thrash task hasn't had a problem with that?
Signed-off-by: Sage Weil <sage@redhat.com>
Make dumping of reservation info congruent between scrub and recovery
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
This is similar to how recovery reservations are split between
local and remote.
It was the case that scrubs_pending was used for reservations at
the replicas as well as at the primary while requesting reservations
from the replicas. There was no need for scrubs_pending to turn
into scrubs_active at the primary as nothing treated that value
as special. scrubber.active = true when scrubbing is
actually going.
Now scurbber.local_reserved indicates scrubs_local incremented
Now scrubber.remote_reserved indicates scrubs_remote incremented
Fixes: https://tracker.ceph.com/issues/41669
Signed-off-by: David Zafman <dzafman@redhat.com>
Removing the symlink since we do not want to run performance tests
on filestore-xfs.yaml.
Fixes: https://tracker.ceph.com/issues/41661
Signed-off-by: Neha Ojha <nojha@redhat.com>
qa/mgr/progress: fix timeout error when waiting for osd in event
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
* refs/pull/30220/head:
test: Allow fractional milliseconds to make test possible
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
If we stop, start, and then mark down, we may (likely) end up marking
the *new* instance down, which is noisy (generates a cluster warning
message) and inefficient.
Signed-off-by: Sage Weil <sage@redhat.com>
feature: Health warnings on long network ping times, add "dump_osd_network" to get a report
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
