Commit Graph

114567 Commits

Author SHA1 Message Date
Sebastian Wagner
26f231f60a
Merge pull request #36572 from dmick/wip-cephadm-doc
doc/mgr: Clean up orchestrator.rst (spell/punct, awkward wording)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-27 17:07:33 +02:00
Kefu Chai
7fded72207
Merge pull request #36750 from ifed01/wip-ifed-enable-flexible-bluefs
os/bluestore: enable more flexible bluefs space management by default.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-08-27 22:28:32 +08:00
Kefu Chai
786b87cdb2
Merge pull request #36718 from ShraddhaAg/add-dir-to-archive-in-logs
qa/tasks/ceph.py: add ceph logs directory in job's info.yaml

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-08-27 22:26:01 +08:00
Jason Dillaman
7b1a1109ea librbd: add internal API for plugin libraries
To avoid the need to expose internal librbd symbols as public, expose
a select set of symbols via an internal plugin API for use by the
plugins.

Fixes: https://tracker.ceph.com/issues/46916
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-08-27 10:16:36 -04:00
Jan Fajerski
6491e53fa1 pybind/mgr/snap_schedule: fix imports for snap_schedule vs mgr wide tox runs
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
4823e657f5 pybing/mgr/snap_schedule: normalize dates to utc and ignore microseconds
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
19f918a5a0 pybind/mgr/snap_schedule: don't use as keyword in update statement
While UPDATE <table_name> AS <alias> ... should be valid SQL syntax. It seems
like only some sqlite version support it however.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
b88b5e9664 pybind/mgr/snap_schedule: fix import locations for unit tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
4d29eaea94 pybind/mgr/snap_schedule: datetime.fromisoformat fallbacks
fromisoformat was introduced in pytho3.7. This can be worked around by
either installing
https://github.com/movermeyer/backports.datetime_fromisoformat. If a
pre3.7 python is found and the backport module is not installed,
snap_schedule falls back to parsing a fully specified timestamp
format. This commit adds the necessary code.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
ed61ed9bbc pybing/mgr/snap_schedule: guard minute repeat interval
To enable the use of a minutely repeat interval in the snap schedules,
this adds a guard that checks for the SNAP_SCHED_TEST env variable.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
39cf0d9e73 pybind/mgr/snap_schedule: add type hint for mypy, fix import
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
38a3e2134a pybind/mgr/snap_schedule: integrate with mgr wide tox testing
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
8ebf967c9a pybing/mgr/snap_schedule: add separate retention interface
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
5f0c5f284e pybing/mgr/snap_schedule: add more unit tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
0af14495ff pybind/mgr/snap-schedule: fix meta uniqueness contraint
This adds a simple test for the Schedule.store_schedule. This test
exposed a bug (couldn't add schedule on different paths with the same
repeat and start time). Bug is also fixed.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
5b9c924b65 pybind/mgr/snap_schedule: add unit tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
a48efa43db pybing/mgr/snap_sched: improve SQL statements, logging, metadata
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Venky Shankar
f19114a733 pybind/snap_schedule: return negative error codes
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
07efe450b8 pybind/mgr/snap_schedule: refactor Schedule ctor, make more args optional
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
efc9ca0330 pybing/mgr/snap_schedule: Add active flag to status report
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
aee0981b41 pybind/mgr/snap-schedule: refactor schedule into two modules
This separates the Schedule class from the SnapSchedClient class.
Motivation is cleaner code layout and simpler testing.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
759403a04f pybind/mgr/snap-schedule: add interface to de-/activate schedules
This allows to 1) activate schedules that have previously been
deactivated due to the path no existing and 2) allowing ussers to
explicitly deactivate a schedule for debugging or maintenance purposes.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
6c63b353ea pybind/mgr/snap-schedule: list - print ws separated results
This should make it easy to consume for cli tools.
Also simplifies some internals.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
92f21b1a2a snap_schedule: add metadata fields, refactor Schedule, multiple Schedules per path
This commit adds a bunch of interesting metadata fields to the DB and
refactors the Schedule class so that it concentrates all the DB queries
in on place. Also enables to set multiple schedules per path as long as
the (repeat interval, start time) pairs are unique.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
3f3df87869 snap-schedule: Implement pruning according to retention specfication
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
39cbd13785 pybind/mgr/snap_schedule: take snapshots according to schedule.
Timers are scheduled for the next snapshot to be taken. The next Timer
is canceled in case the snapshot DB is changed. When a Timer triggers it
also schedules the next Timer.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Venky Shankar
29a6f72131 spec: include snap_schedule in ceph spec file
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
26b94fe21a snap_schedule: add initial interface and basic CRUD ops
Storage of schedules is in an in-memory sqlite DB that gets serialized
to an rados object.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
2ec5a2387e pybind/mgr/mgr_util: CephFSClient - add helper methods
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
ee9dea6cbf pybing/mgr/mgr_module: allow keyword arguments
If a mgr module encounters an argument values containing a '=' parsing
switches to kwargs style parsing independent of arg position. If a
non-kw argument is encountered after the first kw argument an EINVAL
error is returned.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Ilya Dryomov
da4ffd834f rbd: rename some MapOptions instances to unmap_options
It is no longer a global variable, so rename the ones that have to do
with options for unmapping (currently just "force").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-08-27 15:53:38 +02:00
Ilya Dryomov
c06338f33c rbd: fix default pool handling for krbd map/unmap
The default pool name does not get passed to the kernel since commit
96f05a7956 ("rbd: delay determination of default pool name").  The
kernel ends up interpreting the image name as the pool name (and the
snapshot name as the image name).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-08-27 15:53:38 +02:00
Sebastian Wagner
f5db639512
Merge pull request #36816 from sebastian-philipp/cephadm-pep8-osd
mgr/cephadm: PEP8tify services/osd.py

Reviewed-by: Adam King <adking@redhat.com>
2020-08-27 15:38:43 +02:00
Kefu Chai
43a4497491
Merge pull request #36830 from tchaikov/wip-crimson-recovery-cleanup
crimson/osd: recovery related cleanup

Reviewed-by: Xuehan Xu <xxhdx1985126@163.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-27 21:31:24 +08:00
Sebastian Wagner
ebb6db7719
Merge pull request #36841 from matthewoliver/cephadm_iscsi_dev_volumemount
cephadm: Give better access to the /dev in the iscsi container

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-27 15:23:58 +02:00
Sebastian Wagner
48834128ab
Merge pull request #36811 from sebastian-philipp/mgr-always-mock-rados
pybind/mgr: pytest: Unconditionally mock the rados objects

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 14:50:46 +02:00
Daniel Gryniewicz
9d14078d6c
Merge pull request #36844 from BryceCao/wip-correct-log-output
rgw: correct log output when decode bucket CORS attr
2020-08-27 08:30:00 -04:00
Laura Paduano
836e1d4202
Merge pull request #36736 from ceph/47052-fix-base-uri-api-tests
mgr/dashboard: fix API tests: Invalid URL: None/api/auth

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-08-27 14:27:08 +02:00
Sebastian Wagner
a1c3f3b7cf
Merge pull request #36818 from sebastian-philipp/orch-pep8
mgr/orchestrator: PEP8tify

Reviewed-by: Adam King <adking@redhat.com>
2020-08-27 14:03:10 +02:00
Lenz Grimmer
6040a45cab
Merge pull request #36590 from votdev/issue_46903_prevent_select_events
mgr/dashboard: Datatable catches select events from other datatables

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-08-27 12:33:01 +02:00
Lenz Grimmer
ec66fe7a42
Merge pull request #36599 from votdev/issue_46915_login_autocomplete
mgr/dashboard: Disable autocomplete on user form

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-08-27 12:15:17 +02:00
Sebastian Wagner
773b4c64fa
Merge pull request #36799 from mgfritch/cephadm-iscsi-check-mon-command
mgr/cephadm: use `check_mon_command` during iscsi config
2020-08-27 12:03:56 +02:00
Sebastian Wagner
63c7e27072
Merge pull request #36797 from mgfritch/cephadm-infer-image-no-tag
cephadm: infer latest image with a valid tag

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-27 12:03:23 +02:00
Sebastian Wagner
3a0c965d95
Merge pull request #36752 from mgfritch/cephadm-nfs-redux
mgr/cephadm: clean-up nfs service

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-27 12:01:52 +02:00
Sebastian Wagner
a80eafcf7a
Merge pull request #36330 from Devp00l/wip-46551
cephadm: Update error message on host failure

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-27 11:59:36 +02:00
Sebastian Wagner
4609210e10 mgr/cephadm: kick serve loop to trigger ceph.conf deployment
Otherwise we'll wait 600s.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-27 11:33:31 +02:00
Sebastian Wagner
d30d5e4746 mgr/cephadm: disable autopep8 for test_{spec,scheduling}.py
The signal to noise ratio is just too bad.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-27 10:20:34 +02:00
cao.leilc
07cf0e2d24 rgw: correct log output when decode bucket CORS attr
Signed-off-by: caolei <cao.leilc@inspur.com>
2020-08-27 14:32:38 +08:00
Kefu Chai
4e67b77fb5
Merge pull request #36842 from cyx1231st/wip-fix-crimson-heartbeat
crimson/osd: fix unexpected connection markdown in heartbeat

Reviewed-by: Xuehan Xu <xxhdx1985126@163.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-27 14:25:06 +08:00
Yingxin Cheng
b7547d1cbb crimson/net: cleanup, reset event is synchronously called
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-08-27 12:55:07 +08:00