Commit Graph

114357 Commits

Author SHA1 Message Date
Neha Ojha
ab54d3821a tools/osdmaptool.cc: add ability to clean_temps
This is particularly useful for debugging purposes when clean_temps()
takes abnormally high amount of time due to flaws in crush rules etc.

Fixes: https://tracker.ceph.com/issues/47159
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-27 17:01:41 +00:00
Patrick Donnelly
d2a3fcc728
Merge PR #36773 into master
* refs/pull/36773/head:
	mgr/volumes: Prevent subvolume recreate if trash is not-empty
	mgr/volumes: Disallow subvolume group level snapshots
	mgr/volumes: Add test case to ensure subvolume is marked
	mgr/volumes: handle idempotent subvolume marks
	mgr/volumes: Tests amended and added to ensure subvolume trash functionality
	mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
	mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
	mgr/volumes: maintain per subvolume trash directory
	mgr/volumes: make subvolume_v2::_is_retained() object property

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-26 10:46:07 -07:00
Shyamsundar Ranganathan
00e4ab054f mgr/volumes: Prevent subvolume recreate if trash is not-empty
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:33 -04:00
Shyamsundar Ranganathan
f97e57c28c mgr/volumes: Disallow subvolume group level snapshots
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:26 -04:00
Shyamsundar Ranganathan
3527711a4a mgr/volumes: Add test case to ensure subvolume is marked
with the vxattr ceph.dir.subvolume set to true.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:21 -04:00
Venky Shankar
2f5eed200a mgr/volumes: handle idempotent subvolume marks
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 13:41:14 -04:00
Shyamsundar Ranganathan
88cffe6ea9 mgr/volumes: Tests amended and added to ensure subvolume trash functionality
Amended a few test cases to ensure created subvolumes and snaps
are removed, and trash stays empty at the end of the test.

Further added one test case for create errors in a retained
v2 subvolume, to ensure metadata is sane, and created incarnation
is not present.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:08 -04:00
Shyamsundar Ranganathan
fe0c0bd5ea mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
This enables snapshot scaling of subvolumes, and hence prevents
renaming, hardlinking etc. outside the subvolume.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:03 -04:00
Shyamsundar Ranganathan
20c8a62939 mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
For v2 subvolumes without any snapshots or pending purges
move the subvolume to trash, for the purge operation.

When removing only incarnations, leverage symlink based purge
job creation, to avoid renaming the incarnation out of the
subvolume.

For create errors, where the subvolume was retained, just remove
the created incarnation directory.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:40:57 -04:00
Venky Shankar
aae7a70ed2 mgr/volumes: maintain per subvolume trash directory
PR https://github.com/ceph/ceph/pull/36472 introduces changes
that disallow nested nested snapshots in a subtree (subvolume)
and renames across subvolumes. This effect asynchronous purge
in mgr/volumes as subvolume are moved to a trash directory for
asynchronous deletion by purge threads.

To workaround this, start maintaining a subvolume specific
trash directory. Use the trash directory as an index to the
subvolume specific trash directory entry.

This changes subvolume deletion logic which currently relies
on `--retain-snapshots` flag to decide if the subvolume user
directory should get purged or the subvolume base directory
itself. Deleting a subvolume moves the user facing directory
to its specific trash directory. Purge threads take care of
deleting user facing directories (in trash) and the subvolume
base directory if required (when certain conditions are met).

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 13:40:51 -04:00
Venky Shankar
9a76c01536 mgr/volumes: make subvolume_v2::_is_retained() object property
... and drop the leading underscore.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 13:40:11 -04:00
Patrick Donnelly
da9f387e6a
Merge PR #36805 into master
* refs/pull/36805/head:
	pybind/mgr/volumes: add global lock debug

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 08:27:43 -07:00
Mykola Golub
f34d1d3ac0
Merge pull request #36774 from dillaman/wip-46669-2
librbd: drop deferred AsioEngine destruction

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-08-26 18:22:34 +03:00
Lenz Grimmer
0c2d3940c4
Merge pull request #36582 from rhcs-dashboard/fix-redirected-URL
mgr/dashboard: redirect to original URL after successful login

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-08-26 16:25:56 +02:00
Patrick Donnelly
d399d4c05b
pybind/mgr/volumes: add global lock debug
Fixes: https://tracker.ceph.com/issues/47149
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-26 07:10:58 -07:00
Sebastian Wagner
1a3d63761f
Merge pull request #36817 from sebastian-philipp/cephadm-pep8-tests
mgr/cephadm: PEP8tify some tests

Reviewed-by: Adam King <adking@redhat.com>
2020-08-26 14:27:57 +02:00
Sebastian Wagner
8954dfba07
Merge pull request #36315 from p-se/fix-monitoring-images-ceph-salt
cephadm: deploying of monitoring images partially broken

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2020-08-26 13:28:00 +02:00
Sebastian Wagner
27d20304dc
Merge pull request #36571 from pcuzner/cephadm-tox-update
cephadm: remove py2 from tox tests

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2020-08-26 12:56:26 +02:00
Sebastian Wagner
a385c3cc28 mgr/cephadm: PEP8tify some tests
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-26 11:45:56 +02:00
Sebastian Wagner
3561060c23
Merge pull request #36794 from sebastian-philipp/cephadm-pep8-cephadmservice.py
mgr/cephadm: PEP8tify cephadmservice.py

Reviewed-by: Adam King <adking@redhat.com>
2020-08-26 11:37:51 +02:00
Lenz Grimmer
b849f8597f
Merge pull request #36761 from wjwithagen/wjw-fix-dashboard-package.json
dashboard: Fix building build shell script

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-08-26 10:24:18 +02:00
Neha Ojha
fb44c2f500
Merge pull request #36527 from neha-ojha/wip-whitelist-blacklist
Replace 'whitelist' and 'blacklist' language

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2020-08-25 12:24:36 -07:00
Sebastian Wagner
f144e5e76a
Merge pull request #36509 from pcuzner/cephadm-add-host-facts
cephadm: add gather_facts command

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-08-25 20:29:18 +02:00
Sebastian Wagner
712326c1ae
Merge pull request #36793 from sebastian-philipp/cephadm-pep8-schedule.py
mgr/cephadm: PEP8tify schedule.py

Reviewed-by: Adam King <adking@redhat.com>
2020-08-25 20:27:27 +02:00
Sebastian Wagner
7c49e4266a
Merge pull request #36757 from mgfritch/cephadm-raw-str-warn
mgr/cephadm: use raw string during `re.search`

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-25 18:31:33 +02:00
Kefu Chai
cdf5f29384
Merge pull request #36787 from tchaikov/wip-47031
crimson/osd: drop misdirected ops

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-25 22:14:42 +08:00
Sebastian Wagner
56f11c015a mgr/cephadm: PEP8tify cephadmservice.py
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-25 15:33:06 +02:00
Sebastian Wagner
83e4559fce mgr/cephadm: PEP8tify schedule.py
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-25 15:30:51 +02:00
Sebastian Wagner
f82058f06b
Merge pull request #36738 from ShraddhaAg/cephadm-archive-logs
qa/tasks/cephadm.py: add ceph logs directory in job's info.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-25 15:18:58 +02:00
Avan Thakkar
330c75cb51 mgr/dashboard: redirect to original URL after successful login
Fixes: https://tracker.ceph.com/issues/45067

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b5359a155c6ff7e92ab6c6ed16f7ff9fc76c76ea)
2020-08-25 17:37:24 +05:30
Kefu Chai
e307b9aefb crimson/osd: drop misdirected ops
see also `PrimaryLogPG::do_op()`, we should ignore the ops hitting us if
we are not supposed to serve them. this happens when the client is using
a stale osdmap.

Fixes: https://tracker.ceph.com/issues/47031
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-25 18:40:07 +08:00
Kefu Chai
1afc907d8c
Merge pull request #36778 from tchaikov/wip-cmake-fio
cmake: silence "detached HEAD" warning

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-08-25 17:16:22 +08:00
Kefu Chai
47b76ce24e cmake: silence "detached HEAD" warning
git complains when checking out a tag in "detached HEAD", like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them
...

but this does not help, as, in general, we don't hack fio in Ceph,
so disable this warning. and also clone the repo in shallow mode
for the same reason -- we don't care about the whole history of
fio repo. we just use it for testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-25 12:38:03 +08:00
pcuzner
5fbfeaadba
Merge pull request #36336 from pcuzner/ceph-volume-lsm
ceph-volume: add libstoragemgmt support
2020-08-25 11:23:28 +12:00
Paul Cuzner
225cbd8d79 cephadm: remove py2 from test_adoption
Since cephadm is py3 based, and py2 is EOL this patch
removes the py2 test iteration from test_adoption.sh

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-08-25 10:51:26 +12:00
Patrick Donnelly
d83dc481dc
Merge PR #36684 into master
* refs/pull/36684/head:
	qa/tasks/nfs: Test mounting of export created with nfs command
	qa/tasks/nfs: Add helper method to check nfs cluster status
	qa/tasks/nfs: Cleanup created filesystem
	qa/tasks/nfs: Remove unused port status function and 'stdin' keyword argument

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 13:18:13 -07:00
Jason Dillaman
d7dfb8e37c librbd: support reverting to legacy blacklist op via neorados API
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-08-24 19:53:08 +00:00
Ernesto Puerta
6f8c30b1bc mgr/dashboard: remove racially insensitive terms
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
2227121391 librbd: RESTORE_SOURCE_WHITELIST -> ALLOWED_RESTORE_SOURCES
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
be32269fd3 journal, librbd: whitelisted_laggy_clients -> ignored_laggy_clients
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Patrick Donnelly
b5bff49a58 *: convert more blacklist to blocklist
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 19:53:08 +00:00
Patrick Donnelly
1690d3814e librbd: convert WHITELIST to ALLOWLIST
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 19:53:08 +00:00
Patrick Donnelly
c4f5de0058 *: convert EBLACKLISTED to EBLOCKLISTED
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
f466294d94 test/librados_test_stub/TestMemIoCtxImpl.cc: blacklist -> blocklist
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
c0a358670a qa/suites/rados/thrash: log-whitelist -> log-ignorelist
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
01fb7e7f7b qa/suites/krbd/thrash: log-whitelist -> log-ignorelist
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
a8c7efab2d qa/suites/multimds: log-whitelist -> log-ignorelist
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
240280aed9 test/librbd/managed_lock/test_mock_BreakRequest.cc: blacklist -> blocklist
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Neha Ojha
0d5770526f test/librados_test_stub/TestRadosClient.cc: blacklist -> blocklist
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Sage Weil
2e27a54c80 mon/ConfigMonitor: handle renamed config options
Signed-off-by: Sage Weil <sage@newdream.net>
2020-08-24 19:53:08 +00:00