Commit Graph

114468 Commits

Author SHA1 Message Date
Patrick Donnelly
ed90c0768b
Merge PR #36701 into master
* refs/pull/36701/head:
	client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-28 17:48:44 -07:00
Patrick Donnelly
ed892ebb8f
Merge PR #36641 into master
* refs/pull/36641/head:
	mds : skip show_cache if level of log is lower than 7

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-08-28 17:48:15 -07:00
Patrick Donnelly
0902bf0e0f
Merge PR #36882 into master
* refs/pull/36882/head:
	qa/tests: Added "--force-priority"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-28 13:31:07 -07:00
Kefu Chai
bb9878d1b0
Merge pull request #36729 from changchengx/zero_malloc
crush: do not malloc for empty bucket and avoid member variables after memset() with 0

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-28 23:21:27 +08:00
Kefu Chai
8c04ee89ba
Merge pull request #36820 from tchaikov/wip-test-multi_stress_watch
test/multi_stress_watch: use condition_variable for level triggering

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-08-28 23:17:05 +08:00
Kefu Chai
8fef81561d
Merge pull request #36832 from neha-ojha/wip-clean-temps
tools/osdmaptool.cc: add ability to clean_temps

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-08-28 23:14:26 +08:00
Yuri Weinstein
0584150501 qa/tests: Added "--force-priority"
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-08-28 07:35:25 -07:00
Ilya Dryomov
46c912978a
Merge pull request #36854 from idryomov/wip-krbd-default-pool-fix
rbd: fix default pool handling for krbd map/unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-08-28 13:04:52 +02:00
Kefu Chai
fb7729c605
Merge pull request #36866 from yison/fix-core-dump-issue-of-sync-read
blk/spdk/NVMEDevice.cc: fix a core dump issue of sync read

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-28 16:47:51 +08:00
Yuan Lu
3b4714bc96 blk/spdk/NVMEDevice.cc: fix a core dump issue of sync read
Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>
2020-08-28 18:32:02 +00:00
Mykola Golub
e56ace7fa1
Merge pull request #36802 from dillaman/wip-46916
librbd: expose internal plugin API to plugin libraries

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-08-28 11:17:08 +03:00
Sebastian Wagner
0867cd4c32
Merge pull request #36792 from sebastian-philipp/cephadm-pep8-inventory
mgr/cephadm: PEP8tify inventory.py

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-08-28 09:24:36 +02:00
Sebastian Wagner
f030255106
Merge pull request #36848 from sebastian-philipp/cephadm-no-pep8-test_scheduling
mgr/cephadm: disable autopep8 for test_{spec,scheduling}.py

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-08-28 09:20:54 +02:00
Venky Shankar
de2128a7db
Merge pull request #29489 from jan--f/cephfs-snap-schedule
pybind/snap_schedule: add initial interface
2020-08-28 09:51:53 +05:30
Kefu Chai
957a5cfbf5
Merge pull request #34566 from X0Ken/master
os/bluestore: Support flock retry

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-08-28 11:01:41 +08:00
Kefu Chai
210000e8f0
Merge pull request #36507 from matthewoliver/msgr_intercepter_step_enum
msgr: interceptor test step enum to ease readability

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-28 10:51:16 +08:00
Kefu Chai
08255a48a7
Merge pull request #36762 from qwren/qat-doc-update
doc: the update for QAT Acceleration

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-28 10:48:52 +08:00
Patrick Donnelly
fffdb6519b
Merge PR #35941 into master
* refs/pull/35941/head:
	doc/mgr: Supplement missing keywords 'crash'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-27 19:11:30 -07:00
Neha Ojha
762966a08a test/cli/osdmaptool/help.t: add clean-temps
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-27 17:02:08 +00:00
Neha Ojha
78224c21d2 doc: add clean-temps to osdmaptool.rst
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-27 17:02:08 +00:00
Neha Ojha
ab54d3821a tools/osdmaptool.cc: add ability to clean_temps
This is particularly useful for debugging purposes when clean_temps()
takes abnormally high amount of time due to flaws in crush rules etc.

Fixes: https://tracker.ceph.com/issues/47159
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-27 17:01:41 +00:00
Sebastian Wagner
26f231f60a
Merge pull request #36572 from dmick/wip-cephadm-doc
doc/mgr: Clean up orchestrator.rst (spell/punct, awkward wording)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-27 17:07:33 +02:00
Kefu Chai
7fded72207
Merge pull request #36750 from ifed01/wip-ifed-enable-flexible-bluefs
os/bluestore: enable more flexible bluefs space management by default.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-08-27 22:28:32 +08:00
Kefu Chai
786b87cdb2
Merge pull request #36718 from ShraddhaAg/add-dir-to-archive-in-logs
qa/tasks/ceph.py: add ceph logs directory in job's info.yaml

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-08-27 22:26:01 +08:00
Jason Dillaman
7b1a1109ea librbd: add internal API for plugin libraries
To avoid the need to expose internal librbd symbols as public, expose
a select set of symbols via an internal plugin API for use by the
plugins.

Fixes: https://tracker.ceph.com/issues/46916
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-08-27 10:16:36 -04:00
Jan Fajerski
6491e53fa1 pybind/mgr/snap_schedule: fix imports for snap_schedule vs mgr wide tox runs
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
4823e657f5 pybing/mgr/snap_schedule: normalize dates to utc and ignore microseconds
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
19f918a5a0 pybind/mgr/snap_schedule: don't use as keyword in update statement
While UPDATE <table_name> AS <alias> ... should be valid SQL syntax. It seems
like only some sqlite version support it however.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
b88b5e9664 pybind/mgr/snap_schedule: fix import locations for unit tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
4d29eaea94 pybind/mgr/snap_schedule: datetime.fromisoformat fallbacks
fromisoformat was introduced in pytho3.7. This can be worked around by
either installing
https://github.com/movermeyer/backports.datetime_fromisoformat. If a
pre3.7 python is found and the backport module is not installed,
snap_schedule falls back to parsing a fully specified timestamp
format. This commit adds the necessary code.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
ed61ed9bbc pybing/mgr/snap_schedule: guard minute repeat interval
To enable the use of a minutely repeat interval in the snap schedules,
this adds a guard that checks for the SNAP_SCHED_TEST env variable.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
39cf0d9e73 pybind/mgr/snap_schedule: add type hint for mypy, fix import
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
38a3e2134a pybind/mgr/snap_schedule: integrate with mgr wide tox testing
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
8ebf967c9a pybing/mgr/snap_schedule: add separate retention interface
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
5f0c5f284e pybing/mgr/snap_schedule: add more unit tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
0af14495ff pybind/mgr/snap-schedule: fix meta uniqueness contraint
This adds a simple test for the Schedule.store_schedule. This test
exposed a bug (couldn't add schedule on different paths with the same
repeat and start time). Bug is also fixed.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
5b9c924b65 pybind/mgr/snap_schedule: add unit tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
a48efa43db pybing/mgr/snap_sched: improve SQL statements, logging, metadata
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Venky Shankar
f19114a733 pybind/snap_schedule: return negative error codes
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
07efe450b8 pybind/mgr/snap_schedule: refactor Schedule ctor, make more args optional
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
efc9ca0330 pybing/mgr/snap_schedule: Add active flag to status report
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
aee0981b41 pybind/mgr/snap-schedule: refactor schedule into two modules
This separates the Schedule class from the SnapSchedClient class.
Motivation is cleaner code layout and simpler testing.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
759403a04f pybind/mgr/snap-schedule: add interface to de-/activate schedules
This allows to 1) activate schedules that have previously been
deactivated due to the path no existing and 2) allowing ussers to
explicitly deactivate a schedule for debugging or maintenance purposes.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
6c63b353ea pybind/mgr/snap-schedule: list - print ws separated results
This should make it easy to consume for cli tools.
Also simplifies some internals.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
92f21b1a2a snap_schedule: add metadata fields, refactor Schedule, multiple Schedules per path
This commit adds a bunch of interesting metadata fields to the DB and
refactors the Schedule class so that it concentrates all the DB queries
in on place. Also enables to set multiple schedules per path as long as
the (repeat interval, start time) pairs are unique.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:46 +02:00
Jan Fajerski
3f3df87869 snap-schedule: Implement pruning according to retention specfication
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
39cbd13785 pybind/mgr/snap_schedule: take snapshots according to schedule.
Timers are scheduled for the next snapshot to be taken. The next Timer
is canceled in case the snapshot DB is changed. When a Timer triggers it
also schedules the next Timer.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Venky Shankar
29a6f72131 spec: include snap_schedule in ceph spec file
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
26b94fe21a snap_schedule: add initial interface and basic CRUD ops
Storage of schedules is in an in-memory sqlite DB that gets serialized
to an rados object.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00
Jan Fajerski
2ec5a2387e pybind/mgr/mgr_util: CephFSClient - add helper methods
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-27 15:55:45 +02:00