Commit Graph

127634 Commits

Author SHA1 Message Date
Omri Zeneva
6b4382aac7 common/tracer: tracer implementation was moved to common/
Signed-off-by: Omri Zeneva <ozeneva@redhat.com>
2021-11-03 01:53:14 +02:00
Neha Ojha
e7778c13da
Merge pull request #43769 from ifed01/wip-ifed-omap-upgrade-fix-notes
PendingReleaseNotes: document OMAP upgrade bug.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-11-02 16:39:03 -07:00
Igor Fedotov
dfacf162af PendingReleaseNotes: document OMAP upgrade bug.
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2021-11-03 00:03:51 +03:00
Sebastian Wagner
0b81879f3c
Merge pull request #43762 from sebastian-philipp/doc-cephadm-ceph-monstore-tool
doc/cephadm: Calling miscellaneous ceph tools

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@newdream.net>
2021-11-02 17:07:42 +01:00
Sage Weil
5f4a1c8368 Merge PR #42762 into master
* refs/pull/42762/head:
	ceph_test_objectstore: skip BlueStoreUnshareBlobTest with SMR
	os/bluestore: debug ExtentMap::update()
	os/bluestore: _txc_create inside of alloc_and_submit_lock
	os/bluestore: fix cleaner race with collection removal
	os/bluestore: add missing ' ' to LruOnodeCacheShare _[un]pin
	os/bluestore: use simpler map<> to track (onode, zone) -> offset
	os/bluestore: avoid casting zoned implementations again
	os/bluestore/ZonedFreelistManager: remove sanity checks
	os/bluestore/ZonedAllocator: fix allocate() search
	os/bluestore: drain transactions on cleaner zone finish
	os/bluestore/ZonedFreelistManager: simplify freelist merge update vs zone reset
	os/bluetore: configurable sleep period for cleaner
	blk/zoned: make discard a no-op
	os/bluestore/ZonedAllocator: count sequential only as 'free'
	os/bluestore: expect smr fields IFF device is smr
	ceph_test_objectstore: Test for fixing write pointer
	ceph_test_objectstore: complain if SMR support not compiled in
	test/objectstore/run_smr_bluestore_test.sh
	os/bluestore/ZonedAllocator: handle alloc/release spanning zones
	os/bluestore: simple cleaner
	os/bluestore: be smarter about picking a zone to clean
	os/bluestore: avoid writes to cleaning zone
	os/bluestore/HybridAllocator: whitespace in debug output
	os/bluestore: give conventional region of SMR to bluefs
	os/bluestore: separate alloc pointer from shared_alloc.a
	test/objectstore/run_smr_bluestore_test.sh
	ceph_test_objectstore: skip tests that don't work on SMR
	os/bluestore: disable cleaner thread until it is implemented
	os/bluestore: fsck verify zone refs
	os/bluestore: include object in zone ref keys
	os/bluestore: refactor object key helpers a bit
	ceph_test_objectstore: skip failing tests on SMR
	os/bluestore: report mismatch write pointer during fsck
	os/bluestore: simplify zone to clean selection
	ceph_test_objectstore: add trivial fsck test
	os/bluestore: fsck smr allocations (verify num_dead_bytes, alloc past write pointer)
	os/bluestore: duplicate zone refs when cloning
	os/bluestore: correct zoned freelist when device write pointers are ahead
	os/bluestore/ZonedFreelistManager: whitespace
	os/bluestore: fix startup vs device write pointers
	blk/zoned: add get_zones() to fetch write pointers
	os/bluestore: use 64 bit values for zone_state_t
	os/bluestore: reimplement zone backrefs
	os/bluestore: fix smr allocator init
	os/bluestore: do not use null freelist with SMR
	blk/zones: implement HMSMRDevice has KernelDevice child
	os/bluestore: fix/simplify zoned_cleaner thread start error handling
	os/bluestore: properly reset zoned allocator on startup
	os/bluestore: force prefer_deferred_size=0 for smr
	os/bluestore: drop SMR 64K min_alloc_size restriction
	os/bluestore/ZonedAllocator: less verbose
	os/bluestore/ZonedAllocator: simplify debug output prefix
	os/bluestore/ZonedAllocator: be consistent with hex debug output
	os/bluestore/ZonedAllocator: whitespace
	blk/zoned: remove dead VDO code
	blk/zoned: add reset_all_zones()
	blk/zoned: print error during init
	os/bluestore: adjust allocator+freelist interfaces for smr params
	os/bluestore: select 'zoned' freelistmanager during mkfs, not mount

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-11-02 11:41:53 -04:00
Sebastian Wagner
4ac06bc9be
doc/cephadm: Calling miscellaneous ceph tools
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-02 15:47:52 +01:00
Yuval Lifshitz
da0dbe5d7d
Merge pull request #43626 from curtbruns/rgw_example
rgw/lua: Example read/write of StorageClass field
2021-11-02 14:16:44 +02:00
Kefu Chai
3f5ff4ca00
Merge pull request #43765 from inspur-wyq/wip-doc-4
doc/rbd/rbd-mirroring.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-11-02 19:32:41 +08:00
Sebastian Wagner
a5cdb21017
Merge pull request #43628 from pcuzner/cephadm-remove-zram-devices
cephadm: exclude zram and cdrom from device list

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-11-02 09:50:35 +01:00
Kefu Chai
9eb085e9a7
Merge pull request #43766 from inspur-wyq/wip-doc2
doc/radosgw/s3-notification-compatibility.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-11-02 10:09:07 +08:00
wangyunqing
2d64232884 doc/radosgw/s3-notification-compatibility.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2021-11-02 09:25:00 +08:00
Neha Ojha
349b6d1b64
Merge pull request #43742 from ljflores/wip-teuthology-subset
doc/dev/developer_guide/testing_integration_tests: update "frequently used options"

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-11-01 18:12:52 -07:00
Laura Flores
05bb4e3f02 doc/dev/developer_guide/testing_integration_tests: update "frequently used options"
The `subset` option is important in Teuthology runs for reducing the number of tests that are triggered. This option is outlined in another part of the Teuthology documentation, but I think it's important to place here as well.

Also, -n (for how many times the job will run) is incorrect; it should be -N.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-11-02 00:37:10 +00:00
Curt Bruns
bbd7f0dbaf rgw/lua: Example read/write of StorageClass field
Admins may setup different pools for RGW objects and
having the StorageClass field mutable allows the steering
of RGW objects to the proper pools.  This Lua example shows
how a user can modify the StorageClass header when
it is empty on a PUT request and steer objects to different
pools based on size (Request.ContentLength).

Signed-off-by: Curt Bruns <curt.e.bruns@gmail.com>
2021-11-01 19:45:38 -04:00
Sage Weil
38b6a8e8d0 Merge PR #43101 into master
* refs/pull/43101/head:
	mgr/rook: implement apply rbd-mirror

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-11-01 15:27:13 -04:00
Neha Ojha
fa74cae025
Merge pull request #43687 from ifed01/wip-ifed-fix-omap-upgrade
os/bluestore: fix invalid omap name conversion when upgrading to per-pg.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
2021-11-01 12:13:28 -07:00
Patrick Donnelly
b8c036f034
Merge PR #43296 into master
* refs/pull/43296/head:
	mds: improve mds_bal_fragment_size_max config option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-11-01 14:48:06 -04:00
Patrick Donnelly
d96ef3d353
Merge PR #40514 into master
* refs/pull/40514/head:
	cephfs-top: add read/write average io sizes support
	cephfs-top: add read/write io speed support
	mds: add read/write io size metric support

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-01 12:36:21 -04:00
Patrick Donnelly
a8b5c4805e
Merge PR #43582 into master
* refs/pull/43582/head:
	mgr/volumes/fs/operations/versions: fix various flake8 issues

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-11-01 12:35:02 -04:00
Patrick Donnelly
e0c19acbf1
Merge PR #43590 into master
* refs/pull/43590/head:
	qa: test that new mounts of same fs function after old mount is evicted
	qa: remove REQUIRE_KCLIENT_REMOTE

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-11-01 12:34:14 -04:00
Patrick Donnelly
76b75e9ecd
Merge PR #43595 into master
* refs/pull/43595/head:
	client: do not defer releasing caps when revoking

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-01 12:33:06 -04:00
Patrick Donnelly
9ed8bc4200
Merge PR #43717 into master
* refs/pull/43717/head:
	script: update ceph-debug-docker for centos:stream

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2021-11-01 11:53:20 -04:00
Casey Bodley
8946085d3d
Merge pull request #43751 from cbodley/wip-rgw-ops-log-cond
rgw: fix debug build of OpsLogFile

Reviewed-by: Cory Snyder <csnyder@iland.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2021-11-01 11:13:26 -04:00
Sebastian Wagner
49a07a3806
Merge pull request #43647 from adk3798/agent-on-off-tests
qa/suites/orch/cephadm: test with agent on/off

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-01 15:43:10 +01:00
Sebastian Wagner
b38686bf9d
Merge pull request #43454 from Daniel-Pivonka/cephadm-daemon-postactions-bydaemon
mgr/cephadm: track daemon deployment post actions by daemon name instead of type

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-01 15:27:26 +01:00
Sebastian Wagner
e21876e277
Merge pull request #42838 from Daniel-Pivonka/cephadm-apply-spec-traceback
cephadm: fix tracebacks that could occur during apply spec

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-01 14:49:02 +01:00
Deepika Upadhyay
ada9794666
Merge pull request #43127 from majianpeng/librbd-pwl-flush-reorder
librbd/cache/pwl: fix reorder when flush cache-data to osd.

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-11-01 12:31:04 +05:30
Kefu Chai
7f0fa84e43
Merge pull request #43749 from tchaikov/wip-seastore-cleanup
crimson/os/seastore: minor cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-11-01 12:39:06 +08:00
Jianpeng Ma
1fc3be2480 librbd/cache/pwl: fix reorder when flush cache-data to osd.
Consider the following workload:
writeA(0, 4096)
writeB(0, 512).
pwl can makre sure writeA persist to cache before writeB.
But when flush to osd, it use async-read to read data from cache and in
the callback function they issue write to osd.
So although we by order issue aio-read(4096), aio-read(512). But we
can't make sure the return order.
If aio-read(512) firstly return, the write order to next layer is
writeB(0, 512)
writeA(0, 4096).
This is wrong from the user point.

To avoid this occur, we should firstly read all data from cache. And
then send write by order.

Fiexs: https://tracker.ceph.com/issues/52511

Tested-by: Feng Hualong <hualong.feng@intel.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2021-11-01 09:25:52 +08:00
zdover23
9b3fdce0ab
Merge pull request #43676 from zdover23/wip-doc-2021-10-26-rados-config-storage_devices
doc/rados: storage-devices.rst rewrite

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-31 15:17:33 +10:00
Casey Bodley
662d29918d rgw: fix debug build of OpsLogFile
can't use ceph::mutex locks with std::condition_variable::wait() in
debug builds. use the ceph::condition_variable wrapper instead

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-30 20:29:26 -04:00
Sage Weil
bd093a848d Merge PR #43734 into master
* refs/pull/43734/head:
	cephadm: --config-json overrides --config or --keyring args

Reviewed-by: Adam King <adking@redhat.com>
2021-10-30 15:51:31 -04:00
Sage Weil
3e22bfe8e2 Merge PR #43743 into master
* refs/pull/43743/head:
	mgr/nfs/export: use tool_exec() instead of private _exec()
	pybind/mgr/mgr_module: add tool_exec() command

Reviewed-by: Sage Weil <sage@redhat.com>
2021-10-30 15:51:04 -04:00
Sebastian Wagner
433463b9e6
Merge pull request #43654 from adk3798/limit-info
mgr/cephadm: inform users if limit set for data devices is not met

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-30 20:47:32 +02:00
Sebastian Wagner
e9636ddd24
Merge pull request #43606 from adk3798/suppress-tls-error
mgr/cephadm: suppress TLSV1_ALERT_DECRYPT_ERROR from cherrypy

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-30 20:46:48 +02:00
Kefu Chai
89d9b739e8 crimson/os/seastore: minor cleanups
* pass string by reference. more consistent this way.
* set a bl using list::append(), simpler this way.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-30 15:51:03 +08:00
Samuel Just
8b7542d27f
Merge pull request #43657 from liu-chunmei/osd_meta_to_file
crimson: osd meta read from/write to file. 

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-29 23:46:24 -07:00
Samuel Just
9d19de5439 test/crimson/seastore/test_transaction_manager_test_state: add mock MDStore
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-29 18:04:11 -07:00
Samuel Just
da2b40586a crimson/os/seastore/seastore: factor out MDStore interface
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-29 18:03:59 -07:00
chunmei-liu
c423abaf0f crimson: check mkfs_done before seastore do mkfs
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-10-29 18:03:55 -07:00
chunmei-liu
784aa2f4f2 crimson: write some other osd metas
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-10-29 18:03:49 -07:00
chunmei-liu
0f0e5993a9 crimson: osd meta data read from and write to files
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-10-29 18:03:42 -07:00
Sage Weil
8dcdde7151
mgr/nfs/export: use tool_exec() instead of private _exec()
Fixes: https://tracker.ceph.com/issues/53096
Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-10-29 14:03:22 -06:00
Sage Weil
e24171f818
pybind/mgr/mgr_module: add tool_exec() command
Run a (ceph) tool with the appropriate keyring and entity name.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-29 14:03:21 -06:00
Casey Bodley
74565072f7
Merge pull request #43625 from alimaredia/wip-marcus-teuthvault-2
qa/rgw: Fix vault token file access.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-29 15:14:03 -04:00
Sage Weil
4328c7408c Merge PR #43736 into master
* refs/pull/43736/head:
	Revert "Merge pull request #41325 from rhcs-dashboard/wip-cephadm_box-master"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-29 14:55:02 -04:00
Ernesto Puerta
f739fc93aa
Merge pull request #43738 from rishabh-d-dave/mgr-dash-pyfakefs
mgr/dashboard: all pyfakefs must be pinned on same version

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: ljflores <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: rishabh-d-dave <NOT@FOUND>
2021-10-29 18:52:44 +02:00
Yuri Weinstein
0f9321552c
Merge pull request #43733 from neha-ojha/wip-fix-octopusx
qa/suites/upgrade/octopus-x: exclude ceph-volume

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2021-10-29 09:45:37 -07:00
Matt Benjamin
852e9e7d7e
Merge pull request #43329 from cfsnyder/wip-48752-generalize-ops-log-socket
rgw: add abstraction for ops log destination and add file logger
2021-10-29 12:19:59 -04:00
Rishabh Dave
24d3a01a30 mgr/dashboard: all pyfakefs must be pinned on same version
Without this patch execution of install-deps.sh fails with an error.

Fixes: https://tracker.ceph.com/issues/53088
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-10-29 20:34:12 +05:30