Commit Graph

131372 Commits

Author SHA1 Message Date
0xavi0
895ee7921c
rgw/dbstore: change default value of dbstore_db_dir to /var/lib/ceph/radosgw
Changes a few NULL to nullptr.

Adds std::filesystem for path building so they're platform independant.

Fixes a bug for DBStoreManager's second constructor not creating the DB.

Adds unit tests to test DB path and prefix.

Fixes: https://tracker.ceph.com/issues/55731

Signed-off-by: 0xavi0 <xavi.garcia@suse.com>
2022-05-31 09:01:56 +02:00
Yingxin
a74fa9a66f
Merge pull request #46271 from xxhdx1985126/wip-backref-new_mapping-opt
crimson/os/seastore: backref related clean ups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-05-26 10:24:37 +08:00
Patrick Donnelly
6606d91ace
Merge commit 'b76b6ea16899861d6c266797b3c11d9f68f924d7' into main
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-05-25 16:02:14 -04:00
David Galloway
15c1109cb5
Merge pull request #46390 from gabriellasroman/inclusive-lang
teuthology-cronjobs: master->main
2022-05-25 15:47:07 -04:00
Gabriella
99c4e068c0
teuthology-cronjobs: master->main
Signed-off-by: Gabriella Roman <gsroman@bu.edu>
2022-05-25 08:01:11 -04:00
Venky Shankar
b76b6ea168
Merge pull request #44486 from batrick/fs-workload-kclient-switches
qa: reduce fs:workload use of random selection

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-25 06:54:53 +05:30
David Galloway
805d2320b3
Merge pull request #46385 from ceph/inclusive-rename-yaml
Rename/re-symlink whitelist_*.yaml
2022-05-24 17:04:50 -04:00
Zack Cerza
0666dd34cf Rename/re-symlink whitelist_*.yaml
whitelist_health.yaml -> ignorelist_health.yaml
whitelist_wrongly_marked_down.yaml -> ignore_wrongly_marked_down.yaml

This was mostly addressed in
2ee9365d0b,
but the rename wasn't done there.

Signed-off-by: Zack Cerza <zack@cerza.org>
2022-05-24 14:14:04 -06:00
Zack Cerza
6fee777d60
Merge pull request #44967 from batrick/nested-subset-doc
doc/dev: document nested subsets in teuthology
2022-05-24 10:23:48 -06:00
Yuri Weinstein
123c57400e
Merge pull request #46099 from NitzanMordhai/wip-nitzan-rados-test-delete-pools-parallel
systest: fix race with pool delete after socket injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-05-24 08:02:26 -07:00
Yuri Weinstein
5442a42857
Merge pull request #45825 from NitzanMordhai/wip-nitzan-watch-notify-reconnect-107
librados/watch_notify: reconnect after socket injection

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2022-05-24 08:01:45 -07:00
Yuri Weinstein
714cdc4e87
Merge pull request #45805 from ifed01/wip-ifed-more-stats
os/bluestore: more debug information from BlueStore

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2022-05-24 08:01:19 -07:00
Yuri Weinstein
bc02a55d3a
Merge pull request #45409 from NUABO/tancz
mon: add proxy to cache tier options

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-24 07:59:36 -07:00
Adam King
7dbddc008c
Merge pull request #46324 from ceph/dgalloway-nodockerio
qa: Pull from quay.io not docker.io

Reviewed-by: Adam King <adking@redhat.com>
2022-05-24 10:27:03 -04:00
kalebskeithley
77e3e0d39d
Merge pull request #46366 from kalebskeithley/rocksdb
rocksdb: build with rocksdb-7.y.z
2022-05-24 07:06:50 -04:00
Rishabh Dave
d4ff8e3a2f
Merge pull request #46185 from dparmar18/fix_rm_error_output
cephfs-shell: fix rm cmd's error output

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-05-24 16:13:48 +05:30
Rishabh Dave
9368f4fe6c
Merge pull request #46160 from dparmar18/fix_tracker_40860
cephfs-shell: fix rmdir cmd

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2022-05-24 16:10:26 +05:30
Ernesto Puerta
0ec4ba41ad
Merge pull request #46249 from rhcs-dashboard/rbd-mirroring-commands
mgr/dashboard: enable/disable/promote/demote/resync mirroring images

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-05-24 12:38:14 +02:00
Xuehan Xu
0e1d43eea0 crimson/os/seastore/segment_cleaner: downgrade the level of unnecessary logs to DEBUG
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-05-24 12:17:47 +08:00
Xuehan Xu
e3f19f0cb9 crimson/os/seastore/backref: rename BackrefManager::batch_insert_from_cache to BackrefManager::merge_cached_backrefs
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-05-24 12:17:47 +08:00
Xuehan Xu
b8f48c22d4 crimson/os/seastore/segment_cleaner: don't repeat backref mapping retrieval when gc transactions get invalidated
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-05-24 12:17:47 +08:00
Xuehan Xu
86433e23e8 crimson/os/seastore: mandate all access to backrefs to go through backref manager
this would avoid other components' unnecessary dependency on Cache

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-05-24 12:17:45 +08:00
Venky Shankar
90ff84d0ac
Merge pull request #45728 from lxbsz/wip-55144
client: stop retrying the request when exceeding 256 times

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-24 07:54:16 +05:30
Venky Shankar
266b73b147
Merge pull request #46034 from mchangir/mgr-snap_schedule-remove-subvolume-support
mgr/snap_schedule: remove subvol interface

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-24 07:13:32 +05:30
Yingxin
08de9222d1
Merge pull request #46351 from cyx1231st/wip-seastore-cleaner-cleanup
crimson/os/seastore/segment_cleaner: minor cleanups

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-05-24 09:34:48 +08:00
Venky Shankar
16d173a0fd
Merge pull request #46314 from rishabh-d-dave/qa-fs-fsid
qa/vstart_runner: store FSID in LocalContext instances

 Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-24 06:56:13 +05:30
Kaleb S. KEITHLEY
eea10281e6 rocksdb: build with rocksdb-7.y.z
RocksDB 7, specifically 7.2.2 has landed in Fedora 37/rawhide.

https://tracker.ceph.com/issues/55730

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2022-05-24 08:33:37 +08:00
Neha Ojha
f0aeb2ec29
Merge pull request #46346 from ljflores/wip-lflores-testing-recovery
qa/suites/rados/thrash-erasure-code-big/thrashers: add `osd max backfills` setting to mapgap and pggrow

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-23 16:26:27 -07:00
Vikhyat Umrao
cd08198132
Merge pull request #46124 from SMIL-Infra/reopen_cluster_logs
mon/LogMonitor: reopen log files on SIGHUP
2022-05-23 12:49:35 -07:00
Ernesto Puerta
09b3210d64
Merge pull request #46233 from rhcs-dashboard/fix-overlapping-form-icons
mgr/dashboard: form field validation icons overlap with other icons

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-05-23 20:30:08 +02:00
Casey Bodley
51969f86a2
Merge pull request #45985 from soumyakoduri/wip-skoduri-dbstore-tests
rgw/qa: Run s3tests on dbstore backend in teuthology

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-23 12:55:41 -04:00
Pere Diaz Bou
8bd89415fe mgr/dashboard: expose image mirroring commands as endpoints
Expose:
  - enable/disable mirroring in image
  - promote/demote (primary and non-primary)
  - resync
  - snapshot mode:
    - mirror image snapshot (manual snapshot)
    - schedule

Fixes: https://tracker.ceph.com/issues/55645
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-05-23 14:20:50 +02:00
Xiubo Li
8a50329d56 client: stop retrying the request when exceeding 256 times
The type of 'retry_attempt' in 'MetaRequest' is 'int', while in
'ceph_mds_request_head' the type of 'num_retry' is '__u8'. So in
case the request retries exceeding 256 times, the MDS will receive
a incorrect retry seq.

In this case it's ususally a bug in MDS and continue retrying the
request makes no sense. For now let's limit it to 256. In future
this could be fixed in ceph code, so avoid using the hardcode here.

Fixes: https://tracker.ceph.com/issues/55144
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-05-23 14:05:25 +08:00
Venky Shankar
48e9ada562
Merge pull request #45844 from lxbsz/wip-55253
client: switch AT_NO_ATTR_SYNC to AT_STATX_DONT_SYNC

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-23 11:27:33 +05:30
Venky Shankar
4cbb268803
Merge pull request #46337 from rishabh-d-dave/missing-arg-exit-code
cephfs-shell: fixes related to Cmd2ArgparseError

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-23 11:05:03 +05:30
Soumya Koduri
a7397100aa rgw/qa: Run s3tests on dbstore backend in teuthology
Add a new test-suite to run s3tests on dbstore backend.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-21 23:49:47 +05:30
Kefu Chai
b539372437
Merge pull request #44264 from tchaikov/wip-mipel
cmake: do not use GCC extension when detecting 16-byte atomic op

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-05-21 20:27:06 +08:00
Kefu Chai
a7c1c50717
Merge pull request #45835 from tchaikov/wip-55189
common/ceph_json: dump bool using f->dump_bool()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-05-21 20:10:30 +08:00
Kefu Chai
73ccf6ead8
Merge pull request #46355 from tchaikov/wip-no-annobin
ceph.spec.in: disable annobin plugin if compile with gcc-toolset

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2022-05-21 17:34:02 +08:00
Kefu Chai
6677658992 common/ceph_json: dump bool using f->dump_bool()
as per https://www.json.org/json-en.html, JSON encodes bool as
"true" or "false", without the quotes. before this change, the quotes
are always added when encoding boolean values.

but this change is not backward compatible.

encode_json()'s bool overload is used by rgw. it uses JSONObj
defined in common/ceph_json.h to decode JSON-encoded structs.
and it does not differentiate bool from str when decoding a boolean
value despite that it could have check the "quoted" member variable
of JSONObj for validating the type of value. so we should be fine.

Fixes: https://tracker.ceph.com/issues/55189
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-05-21 16:32:14 +08:00
Ronen Friedman
6db901185c
Merge pull request #46293 from ronen-fr/wip-rf-sched-test
test/osd: unit-tests for the scrubber scheduler

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-21 10:03:17 +03:00
Adam King
9476907f60
Merge pull request #45467 from phlogistonjohn/jjm-format-exp2
A module and decorator for generically handling format= in python mgr modules

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-05-20 18:36:24 -04:00
Adam King
54cdc1d47e
Merge pull request #45786 from adk3798/staggered-upgrade
mgr/cephadm: staggered upgrade

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-05-20 18:33:36 -04:00
Kefu Chai
5a78fe1764 ceph.spec.in: disable annobin plugin if compile with gcc-toolset
we are seeing FTBFS failures when compiling the packages with seastar
enabled, like:

Package gcc-toolset-9-annobin-9.08-4.el8.x86_64 is already installed.
...
Package redhat-rpm-config-130-1.el8.noarch is already installed.
...
CFLAGS=-O2 -g -pipe -Wall -Werror=format-security  -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
...
-- The CXX compiler identification is GNU 9.2.1
....
    Building CXX object CMakeFiles/cmTC_386fb.dir/testCXXCompiler.cxx.o
    /opt/rh/gcc-toolset-9/root/usr/bin/c++   -O2 -g -pipe -Wall -Werror=format-security  -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -o CMakeFiles/cmTC_386fb.dir/testCXXCompiler.cxx.o -c /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.0-209-g3e01cd41/rpm/el8/BUILD/ceph-17.2.0-209-g3e01cd41/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
    cc1plus: fatal error: inaccessible plugin file /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/gcc-annobin.so expanded from short plugin name gcc-annobin: No such file or directory

the root cause is that redhat-rpm-config expects gcc-annobin:

$ cat /usr/lib/rpm/redhat/redhat-annobin-cc1 *cc1_options: + %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=gcc-annobin}

while this plugin is not installed by gcc-toolset-9-annobin, which installs annobin.so though:

$ rpm -ql gcc-toolset-9-annobin| grep annobin.so /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/annobin.so /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/annobin.so.0 /opt/rh/gcc-toolset-9/root/usr/lib/gcc/x86_64-redhat-linux/9/plugin/annobin.so.0.0.0

but gcc-toolset-8-annobin provides this file. upgrading to
gcc-toolset-11 does not help. see https://centos.pkgs.org/8-stream/centos-appstream-x86_64/gcc-toolset-11-annobin-plugin-gcc-10.23-1.el8.x86_64.rpm.html
so, the intermediate solution would be to disable the plugin, if
we want to use gcc-toolset to build rpm packages.

in this change, _annotated_build is undefined to prevent the compiler
from adding extra information to the binary. in general this change
shuold be safe, without these information, it'd be hard to tell if
the binary is hardened or what ABI version it expects. see
also https://fedoraproject.org/wiki/Changes/Annobin

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-05-20 14:32:18 +08:00
Rishabh Dave
ab19827858 qa/cephfs: remove temporary files
These temporary files don't matter for test execution with teuthology
but they do matter for execution with vstart_runner.py since the test
fails if these files exist already. And tests are often run repeatedly
with vstart_runner.py, unlike with teuthology.

Fixes: https://tracker.ceph.com/issues/55719
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-20 10:58:33 +05:30
Yingxin Cheng
d8f5acb8f6 crimson/os/seastore/cache: ignore maybe unused variables
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-05-20 11:27:02 +08:00
Yingxin Cheng
521a316daa crimson/os/seastore/segment_cleaner: cleanup logs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-05-20 11:27:02 +08:00
Yingxin Cheng
90276e9fae crimson/os/seastore/segment_cleaner: move implementations to cc
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-05-20 11:26:56 +08:00
Samuel Just
c6e5724e1b
Merge pull request #46335 from cyx1231st/wip-seastore-onode-fix-unaligned-access
crimson/onode-staged-tree: fix unaligned reference to shard_pool_t::pool

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-19 19:16:29 -07:00
Samuel Just
b5901f9a42
Merge pull request #46333 from athanatos/sjust/wip-55705
crimson: generalize journal_seq_t to tolerate different paddr_t types

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-05-19 19:14:29 -07:00