Commit Graph

114404 Commits

Author SHA1 Message Date
Venky Shankar
29a6f72131 spec: include snap_schedule in ceph spec file
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
26b94fe21a snap_schedule: add initial interface and basic CRUD ops
Storage of schedules is in an in-memory sqlite DB that gets serialized
to an rados object.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
2ec5a2387e pybind/mgr/mgr_util: CephFSClient - add helper methods
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
ee9dea6cbf pybing/mgr/mgr_module: allow keyword arguments
If a mgr module encounters an argument values containing a '=' parsing
switches to kwargs style parsing independent of arg position. If a
non-kw argument is encountered after the first kw argument an EINVAL
error is returned.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Sebastian Wagner
f5db639512
Merge pull request #36816 from sebastian-philipp/cephadm-pep8-osd
mgr/cephadm: PEP8tify services/osd.py

Reviewed-by: Adam King <adking@redhat.com>
2020-08-27 15:38:43 +02:00
Kefu Chai
43a4497491
Merge pull request #36830 from tchaikov/wip-crimson-recovery-cleanup
crimson/osd: recovery related cleanup

Reviewed-by: Xuehan Xu <xxhdx1985126@163.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-27 21:31:24 +08:00
Sebastian Wagner
ebb6db7719
Merge pull request #36841 from matthewoliver/cephadm_iscsi_dev_volumemount
cephadm: Give better access to the /dev in the iscsi container

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-27 15:23:58 +02:00
Sebastian Wagner
48834128ab
Merge pull request #36811 from sebastian-philipp/mgr-always-mock-rados
pybind/mgr: pytest: Unconditionally mock the rados objects

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 14:50:46 +02:00
Daniel Gryniewicz
9d14078d6c
Merge pull request #36844 from BryceCao/wip-correct-log-output
rgw: correct log output when decode bucket CORS attr
2020-08-27 08:30:00 -04:00
Laura Paduano
836e1d4202
Merge pull request #36736 from ceph/47052-fix-base-uri-api-tests
mgr/dashboard: fix API tests: Invalid URL: None/api/auth

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-08-27 14:27:08 +02:00
Sebastian Wagner
a1c3f3b7cf
Merge pull request #36818 from sebastian-philipp/orch-pep8
mgr/orchestrator: PEP8tify

Reviewed-by: Adam King <adking@redhat.com>
2020-08-27 14:03:10 +02:00
Lenz Grimmer
6040a45cab
Merge pull request #36590 from votdev/issue_46903_prevent_select_events
mgr/dashboard: Datatable catches select events from other datatables

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-08-27 12:33:01 +02:00
Lenz Grimmer
ec66fe7a42
Merge pull request #36599 from votdev/issue_46915_login_autocomplete
mgr/dashboard: Disable autocomplete on user form

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-08-27 12:15:17 +02:00
Sebastian Wagner
773b4c64fa
Merge pull request #36799 from mgfritch/cephadm-iscsi-check-mon-command
mgr/cephadm: use `check_mon_command` during iscsi config
2020-08-27 12:03:56 +02:00
Sebastian Wagner
63c7e27072
Merge pull request #36797 from mgfritch/cephadm-infer-image-no-tag
cephadm: infer latest image with a valid tag

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-27 12:03:23 +02:00
Sebastian Wagner
3a0c965d95
Merge pull request #36752 from mgfritch/cephadm-nfs-redux
mgr/cephadm: clean-up nfs service

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-27 12:01:52 +02:00
Sebastian Wagner
a80eafcf7a
Merge pull request #36330 from Devp00l/wip-46551
cephadm: Update error message on host failure

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-27 11:59:36 +02:00
cao.leilc
07cf0e2d24 rgw: correct log output when decode bucket CORS attr
Signed-off-by: caolei <cao.leilc@inspur.com>
2020-08-27 14:32:38 +08:00
Kefu Chai
4e67b77fb5
Merge pull request #36842 from cyx1231st/wip-fix-crimson-heartbeat
crimson/osd: fix unexpected connection markdown in heartbeat

Reviewed-by: Xuehan Xu <xxhdx1985126@163.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-27 14:25:06 +08:00
Yingxin Cheng
b7547d1cbb crimson/net: cleanup, reset event is synchronously called
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-08-27 12:55:07 +08:00
Kefu Chai
031d363ca3
Merge pull request #36807 from badone/wip-tracker-47002-legacy_vals-keyring
common/config: Make sure legacy_values are updated

Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-27 12:40:58 +08:00
Kefu Chai
3b1ab0313f
Merge pull request #36535 from hualongfeng/fixed_unused_bluefs
os/bluestore: do not update used_blocks with bluefs_extents if bluefs is not available

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-27 11:41:46 +08:00
Yingxin Cheng
bfadb8282c crimson/osd: fix unexpected connection markdown in heartbeat
Pass reference when log Heartbeat::Connection instance, or the
destructor will be called incorrectly, and the conn be marked down
unexpectedly.

Also, the replacing conn is actually connected during replacement-reset
event.

Fixes: https://tracker.ceph.com/issues/47124

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-08-27 11:05:09 +08:00
Brad Hubbard
076c4d752c common/config: Make sure legacy_values are updated
Must be updated after final change to 'values'

Fixes: https://tracker.ceph.com/issues/47002

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-08-27 12:45:26 +10:00
Patrick Donnelly
f522cbf0b4
Merge PR #36827 into master
* refs/pull/36827/head:
	qa/suites/smoke: remove packages on test completion

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-08-26 19:45:07 -07:00
Matthew Oliver
40c0d68212 cephadm: Give better access to the /dev in the iscsi container
In testing it seems the main iscsi container's /dev related volume mount to just
/dev/log is too narrow. And in certain circumstances it needs to access
to see /dev/rbd* devices. Like if using krdb.

This patch volume mounts /dev rather then /dev/log in the main
container, but since this aligns with what we need in the tcmu-runner
container it actaully ends up simplifying the code as well.

Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-08-27 12:44:40 +10:00
Kefu Chai
22b4d3a0b3
Merge pull request #36801 from yuriw/wip-yuriw-crontab-master
qa/tests: clients' upgrades for older releases must use "-t py2"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-08-27 10:14:33 +08:00
Yuri Weinstein
ed804d284c qa/tests: clients' upgrades for older releases must use "-t py2"
Fixes: https://tracker.ceph.com/issues/47120
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-08-26 16:17:51 -07:00
Patrick Donnelly
d2a3fcc728
Merge PR #36773 into master
* refs/pull/36773/head:
	mgr/volumes: Prevent subvolume recreate if trash is not-empty
	mgr/volumes: Disallow subvolume group level snapshots
	mgr/volumes: Add test case to ensure subvolume is marked
	mgr/volumes: handle idempotent subvolume marks
	mgr/volumes: Tests amended and added to ensure subvolume trash functionality
	mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
	mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
	mgr/volumes: maintain per subvolume trash directory
	mgr/volumes: make subvolume_v2::_is_retained() object property

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-26 10:46:07 -07:00
Shyamsundar Ranganathan
00e4ab054f mgr/volumes: Prevent subvolume recreate if trash is not-empty
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:33 -04:00
Shyamsundar Ranganathan
f97e57c28c mgr/volumes: Disallow subvolume group level snapshots
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:26 -04:00
Shyamsundar Ranganathan
3527711a4a mgr/volumes: Add test case to ensure subvolume is marked
with the vxattr ceph.dir.subvolume set to true.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:21 -04:00
Venky Shankar
2f5eed200a mgr/volumes: handle idempotent subvolume marks
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 13:41:14 -04:00
Shyamsundar Ranganathan
88cffe6ea9 mgr/volumes: Tests amended and added to ensure subvolume trash functionality
Amended a few test cases to ensure created subvolumes and snaps
are removed, and trash stays empty at the end of the test.

Further added one test case for create errors in a retained
v2 subvolume, to ensure metadata is sane, and created incarnation
is not present.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:08 -04:00
Shyamsundar Ranganathan
fe0c0bd5ea mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
This enables snapshot scaling of subvolumes, and hence prevents
renaming, hardlinking etc. outside the subvolume.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:03 -04:00
Shyamsundar Ranganathan
20c8a62939 mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
For v2 subvolumes without any snapshots or pending purges
move the subvolume to trash, for the purge operation.

When removing only incarnations, leverage symlink based purge
job creation, to avoid renaming the incarnation out of the
subvolume.

For create errors, where the subvolume was retained, just remove
the created incarnation directory.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:40:57 -04:00
Venky Shankar
aae7a70ed2 mgr/volumes: maintain per subvolume trash directory
PR https://github.com/ceph/ceph/pull/36472 introduces changes
that disallow nested nested snapshots in a subtree (subvolume)
and renames across subvolumes. This effect asynchronous purge
in mgr/volumes as subvolume are moved to a trash directory for
asynchronous deletion by purge threads.

To workaround this, start maintaining a subvolume specific
trash directory. Use the trash directory as an index to the
subvolume specific trash directory entry.

This changes subvolume deletion logic which currently relies
on `--retain-snapshots` flag to decide if the subvolume user
directory should get purged or the subvolume base directory
itself. Deleting a subvolume moves the user facing directory
to its specific trash directory. Purge threads take care of
deleting user facing directories (in trash) and the subvolume
base directory if required (when certain conditions are met).

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 13:40:51 -04:00
Venky Shankar
9a76c01536 mgr/volumes: make subvolume_v2::_is_retained() object property
... and drop the leading underscore.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 13:40:11 -04:00
Kefu Chai
7736bddc53 crimson/osd: use map::at() for better readability
avoid using operator[] for find-or-create, to ensure that we fail if the
indexed element is missing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-27 01:38:47 +08:00
Kefu Chai
2a5f3b7ee7 messages,osd,crimson: MOSDPGPull::set_pulls(vector<>&&)
this allows us to pass a lvalue reference to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-27 01:32:03 +08:00
Kefu Chai
f39d0a1d0d crimson/osd: extract load_obc_for_recovery() out
extract ReplicatedRecoveryBackend::load_obc_for_recovery()
from ReplicatedRecoveryBackend::recover_object() for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-27 01:32:03 +08:00
Kefu Chai
7aff9cbc37 crimson/osd: extract maybe_pull_missing_obj() out
extract ReplicatedRecoveryBackend::maybe_pull_missing_obj() from
ReplicatedRecoveryBackend::recover_object() for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-27 01:32:03 +08:00
Patrick Donnelly
55c219e9ed
qa/suites/smoke: remove packages on test completion
A new teuthology PR [1] skips package cleanup by default. Use the
lightweight smoke suite to ensure that package cleanup continues to
work.

[1] https://github.com/ceph/teuthology/pull/1553

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-26 08:43:20 -07:00
Patrick Donnelly
da9f387e6a
Merge PR #36805 into master
* refs/pull/36805/head:
	pybind/mgr/volumes: add global lock debug

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-08-26 08:27:43 -07:00
Mykola Golub
f34d1d3ac0
Merge pull request #36774 from dillaman/wip-46669-2
librbd: drop deferred AsioEngine destruction

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-08-26 18:22:34 +03:00
Lenz Grimmer
0c2d3940c4
Merge pull request #36582 from rhcs-dashboard/fix-redirected-URL
mgr/dashboard: redirect to original URL after successful login

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-08-26 16:25:56 +02:00
Patrick Donnelly
d399d4c05b
pybind/mgr/volumes: add global lock debug
Fixes: https://tracker.ceph.com/issues/47149
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-26 07:10:58 -07:00
Sebastian Wagner
1a3d63761f
Merge pull request #36817 from sebastian-philipp/cephadm-pep8-tests
mgr/cephadm: PEP8tify some tests

Reviewed-by: Adam King <adking@redhat.com>
2020-08-26 14:27:57 +02:00
Sebastian Wagner
8954dfba07
Merge pull request #36315 from p-se/fix-monitoring-images-ceph-salt
cephadm: deploying of monitoring images partially broken

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2020-08-26 13:28:00 +02:00
Sebastian Wagner
27d20304dc
Merge pull request #36571 from pcuzner/cephadm-tox-update
cephadm: remove py2 from tox tests

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2020-08-26 12:56:26 +02:00