Commit Graph

130842 Commits

Author SHA1 Message Date
Soumya Koduri
53667dd5eb
Merge pull request #45909 from soumyakoduri/wip-skoduri-dbstore-fixes
rgw/dbstore: Handle prefix/delim while listing objects
2022-04-19 16:34:32 +05:30
Milind Changire
24c13e53de mgr/snap_schedule: add time zone suffix to snapshot dir name
Fixes: https://tracker.ceph.com/issues/54374
Signed-off-by: Milind Changire <mchangir@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-04-19 06:49:18 -04:00
Matan Breizman
80efa9cad6 crimson/os: CyanStore debug subsystem
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-04-19 10:21:08 +00:00
Ilya Dryomov
0f1634a21f run-make-check.sh: enable RBD persistent caches
This was attempted in commit 69a7ed4eab ("run-make-check: enable
WITH_RBD_RWL when WITH_PMEM is true") but never completed.  We soon
bumped the requirement on libpmem, so WITH_SYSTEM_PMDK=ON wouldn't
have worked anyway.

Enable the RWL mode conditionally based on WITH_RBD_RWL variable.
Enable the SSD mode unconditionally as it has no special dependencies
and can be built on any architecture.

Fixes: https://tracker.ceph.com/issues/55285
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-19 11:21:06 +02:00
Ilya Dryomov
59d928a06c test/encoding/check-generated.sh: show diff if binary reencode check fails
Take bf0b161115 ("test/encoding/check-generated.sh: show diff if cmp
fails") a bit further.  Suggesting "cmp $tmp1 $tmp2" isn't very helpful
since cmp would report just the mismatch offset.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-19 11:21:06 +02:00
Ilya Dryomov
91d270b210 librbd/cache/pwl: WriteLogCacheEntry constructor must initialize flags
Initializing the individual bit field members leaves the remaining two
bits uninitialized and that garbage state gets persisted.

In general, using bit fields in a structure where the layout actually
matters is not desirable.  Even with a few single bits, such as here,
their order, strictly speaking, is not guaranteed:

    An implementation may allocate any addressable storage unit large
    enough to hold a bit-field. If enough space remains, a bit-field
    that immediately follows another bit-field in a structure shall be
    packed into adjacent bits of the same unit. If insufficient space
    remains, whether a bit-field that does not fit is put into the next
    unit or overlaps adjacent units is implementation-defined. The
    order of allocation of bit-fields within a unit (high-order to
    low-order or low-order to high-order) is implementation-defined.
    The alignment of the addressable storage unit is unspecified.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-19 11:21:05 +02:00
Ilya Dryomov
2c131f57d6 librbd/cache/pwl: initialize generate_test_instances() objects
... to prevent check-generated.sh failures such as:

**** librbd::cache::pwl::WriteLogPoolRoot test 1 dump_json check failed ****
   ceph-dencoder type librbd::cache::pwl::WriteLogPoolRoot select_test 1 dump_json > /tmp/typ-cAoWrqlHC
   ceph-dencoder type librbd::cache::pwl::WriteLogPoolRoot select_test 1 encode decode dump_json > /tmp/typ-ES5yHpfGL
5c5
<     "flushed_sync_gen": 0,
---
>     "flushed_sync_gen": 255,

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-19 11:21:05 +02:00
Ilya Dryomov
753aa038fd librbd/cache/pwl: fix -Wunused-lambda-capture warnings
Reported by clang on "make check" and "make check arm64" builds.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-19 11:21:05 +02:00
Soumya Koduri
e51de26939 rgw/dbstore: Handle prefix/delim in Bucket::List op
Given a prefix, fetch only those objects matching the prefix.
In addition, skip the entries with "delim" and instead include
those entries in common_prefixes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-04-19 13:23:09 +05:30
Soumya Koduri
61ede70869 rgw/dbstore: DBObject state initialization
Initialize & store 'state' part of DBObject to be
referenced by callers of 'get_obj_state'.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-04-19 13:23:09 +05:30
Ziye Yang
86cb8c75c3 Add the support to use DML library for PMEM device.
The purpose of this patch is to add the initial support to
offload memory/pmem operations by sync usage through hardware path
in DML library.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2022-04-19 14:51:12 +08:00
Liu-Chunmei
453f53ad47
Merge pull request #45866 from liu-chunmei/crimson-longxattr
crimson: check long attr name

Reviewed-by: Samuel Just <sjust@redhat.com>
Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-04-18 20:09:07 -07:00
chunmei-liu
6f21d915b2 crimson/seastore: enlarge omap_leaf_node size
to avoid double split when key size around 2K.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-04-18 19:37:27 -07:00
Laura Flores
01ae097b91
Merge pull request #44784 from Gaoweinan/master-doc-InconsistentFormat-gaoweinan
doc:InconsistentFormat
2022-04-18 13:59:21 -05:00
Ernesto Puerta
d8c0229b90
.github/pr-triage: reorder milestone step
In `master` the milestone step exits and causes remaining tasks not to be run. I previously tried with the `continue-on-error` flag, but it didn't work, so let's try putting that steps at the end.

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2022-04-18 18:50:52 +02:00
Cory Snyder
850c16c246 bluestore: set upper and lower bounds on rocksdb omap iterators
Limits RocksDB omap Seek operations to the relevant key range of the object's omap.
This prevents RocksDB from unnecessarily iterating over delete range tombstones in
irrelevant omap CF shards. Avoids extreme performance degradation commonly caused
by tombstones generated from RGW bucket resharding cleanup. Also prefer CFIteratorImpl
over ShardMergeIteratorImpl when we can determine that all keys within specified
IteratorBounds must be in a single CF.

Fixes: https://tracker.ceph.com/issues/55324
Signed-off-by: Cory Snyder <csnyder@iland.com>
2022-04-18 12:34:02 -04:00
Redouane Kachach
9756e3bf8c
mgr/cephadm: fixing logging output for rm-cluster operation
Fixes: https://tracker.ceph.com/issues/51671

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-04-18 17:10:57 +02:00
Adam King
b6f42e6db3
Merge pull request #45860 from adk3798/alertmanager-skip-verify
mgr/cephadm: allow setting insecure_skip_verify for alertmanager

Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2022-04-18 08:57:14 -04:00
Venky Shankar
73e806b7da
Merge pull request #43879 from rishabh-d-dave/vr-clearer-msg
qa/vstart_runner: print "success" explicitly

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-18 15:27:52 +05:30
Venky Shankar
4070f734ee
Merge pull request #45924 from rishabh-d-dave/qa-cephfs-test_cephfs_shell
qa/cephfs: expand note for test_cephfs_shell.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-18 15:02:55 +05:30
Venky Shankar
fa57a1c180
Merge pull request #45923 from rishabh-d-dave/doc-fix-note
doc/developer_guide: split a note point into two

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-18 13:28:46 +05:30
Rishabh Dave
bc02c3c7e9 qa/cephfs: expand note for test_cephfs_shell.py
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-18 13:15:10 +05:30
Nizamudeen A
d74defbbb3 mgr/dashboard: save mgr logs inside a folder on kcli e2e
Save the mgr container logs of cephadm inside a folder and later on
archive it and get it as an artifact on the cephadm dashboard e2e jobs

Fixes: https://tracker.ceph.com/issues/55247
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-04-18 11:26:40 +05:30
Rishabh Dave
902420cbd4 qa/vstart_runner: print "success" explicitly
This makes it effortless for new vstart_runner.py users to spot that the
test passed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-18 10:39:08 +05:30
Venky Shankar
1ac01c60df
Merge pull request #45816 from kotreshhr/clone-failure-status
mgr/volumes: Show clone failure reason in clone status command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-18 09:43:40 +05:30
Xiubo Li
e9a26c551c mds, client: only send the metrices supported by MDSes
For the old ceph clusters the clients won't send any metrics to
them as default unless they have backported this commit, but there
has one option 'client_collect_and_send_global_metrics' still could
be used to enable it manually.

This will fix the crash bug when upgrading from old ceph clusters,
which will crash the MDSes once they receive unknown metrics.

Fixes: https://tracker.ceph.com/issues/54411
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-18 10:18:31 +08:00
Xiubo Li
beb9f16481 client: add option to enable and force collecting and sending metrics
To be careful to enable this because it may crash the old MDSes while
upgrading.

Fixes: https://tracker.ceph.com/issues/54411
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-18 10:18:31 +08:00
Xiubo Li
f232b020fc mds: notify clients if the session has already opened
If the connection was accidently closed due to the socket issue or
something else the client will try to open the opened sessions, for
now the MDS will just discard the session open request.

But the client will keep waiting the reply from the mds forever.

We need to tell the clients what has happened instead of discard it
directly. And when the client get the session open reply, it can
do what needed.

Fixes: https://tracker.ceph.com/issues/53911
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-18 10:17:11 +08:00
Xiubo Li
0738646739 mds: remove the useless session seq for stale MClientSession
Client side never uses this seq.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-18 10:17:10 +08:00
Xiubo Li
0f9518a3c7 client: skip reopening the opened or is under opening sessions
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-18 10:17:10 +08:00
Rishabh Dave
f5974d3993 doc/developer_guide: split a note point into two
Since both the sentences in the note point aren't strictly related to
each other, it's better to split that note point into two.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-17 22:53:09 +05:30
Adam King
017aa9cfe8 qa/suites/orch/cephadm: stop upgrade tests if failures are seen
Otherwise the tests may run forever. This was already done for
mds upgrade sequence, justadding it in the other two places here

Related to: https://tracker.ceph.com/issues/53939

Signed-off-by: Adam King <adking@redhat.com>
2022-04-17 12:21:44 -04:00
Venky Shankar
144f6fd7e0
Merge pull request #45114 from lxbsz/wip-54362
client: do not release the global snaprealm until unmounting

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-17 15:24:57 +05:30
Venky Shankar
7879e86f50
Merge pull request #45688 from lxbsz/fwd
client: stop forwarding the request when exceeding 256 times

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-17 15:23:15 +05:30
Venky Shankar
baf2f09b5f
Merge pull request #45107 from lxbsz/wip-54345
mds: reset heartbeat when fetching or committing dentries

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-16 20:50:17 +05:30
Laura Flores
c1b0ecc499
Merge pull request #45915 from ljflores/wip-dashboard-cypress-password 2022-04-15 23:58:58 -05:00
Kefu Chai
e2babdfae8 cmake/modules: always use the python3 specified in command line
if another python3 with higher version is found by
find_package(Python3), the cmake's install script would just
install the python modules/extensions into that python3's
dist-package directory, and the packaging script would fail
to find these artifacts when trying to package them.

so we need to ensure that the install directories for python
modeules/extensions are always "versioned" with WITH_PYTHON3
cmake option.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-04-16 09:15:03 +08:00
Laura Flores
cf6feab179 mgr/dashboard/frontend: fix cypress env password
"LOGIN_PASSWORD" should be "LOGIN_PWD". Bug introduced
in e9128c4.

Fixes: https://tracker.ceph.com/issues/55323
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-04-15 22:15:25 +00:00
Adam King
5360de431d
Merge pull request #45765 from m-ildefons/1196785-cephadm-status-trace
cephadm: avoid crashing on expected non-zero exit

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-04-15 11:05:05 -04:00
Adam King
44a13a209f
Merge pull request #45863 from adk3798/mgr-fail-retry
mgr/cephadm: retry mgr fail over in case of transient failure

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-04-15 11:02:55 -04:00
Ilya Dryomov
eca40854e7
Merge pull request #45898 from idryomov/wip-resurrect-mutex-debug
cmake: resurrect mutex debugging in all Debug builds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-04-15 11:09:42 +02:00
chunmei-liu
a91d1fd48e crimson/osd: check obj exists
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-04-14 23:06:22 -07:00
Xiubo Li
5117136583 client: do not release the global snaprealm until unmounting
The global snaprealm would be created and then destroyed immediately
every time when updating it.

Fixes: https://tracker.ceph.com/issues/54362
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-15 10:51:31 +08:00
Ziye Yang
b16296911b blk/pmem: Add the devdax support.
The purpose is to make the pmem device usage more flexible
than the current solution. And prepare for the potential
offloading by hardware engine later.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2022-04-15 02:33:27 +08:00
Adam King
e583d4ef1a mgr/cephadm: allow setting insecure_skip_verify for alertmanager
Add a "secure" parameter to alertmanager spec that will cause it
to deploy alertmanagers with insecure_skip_verify as true or false
depending on the value given for "secure".

NOTE: alertmanager must still be reconfigured after applying a yaml
with this option changed.

Fixes: https://tracker.ceph.com/issues/55272
Fixes: https://tracker.ceph.com/issues/55333

Signed-off-by: Adam King <adking@redhat.com>
2022-04-14 10:57:22 -04:00
Ali Maredia
5c1c754ecd
Merge pull request #45394 from iqbalredkhan/amrojiqbal
cls/rgw : Add missing classes in < #include "cls/rgw/cls_rgw_types.h">

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-04-14 09:35:00 -04:00
Xiubo Li
ad61e1dd1a client: stop forwarding the request when exceeding 256 times
The type of 'num_fwd' in ceph 'MClientRequestForward' is 'int32_t',
while in 'ceph_mds_request_head' the type is '__u8'. So in case
the request bounces between MDSes exceeding 256 times, the client
will get stuck.

In this case it's ususally a bug in MDS and continue bouncing the
request makes no sense.

Fixes: https://tracker.ceph.com/issues/55129
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-04-14 20:13:19 +08:00
Venky Shankar
b4ea90ad82
Merge pull request #45549 from nmshelke/fuse-linux-only
ceph-fuse: restrict already_fuse_mounted function only for linux

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-14 17:38:20 +05:30
Venky Shankar
8316aa95fc
Merge pull request #45636 from joscollin/wip-B54971-rank0-stale-perf-stats-assertion-error
qa: make test_perf_stats_stale_metrics check only the clients created for the tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-14 17:36:42 +05:30
Venky Shankar
3ca23d6791
Merge pull request #45663 from lxbsz/client_cleanup_dl
client: remove expect_null and cleanup the code get_or_create()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-14 17:34:31 +05:30