Commit Graph

134185 Commits

Author SHA1 Message Date
Samuel Just
31b0c047fd ceph_dedup_tool: fix unnecessarily complicated bool returns
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-19 13:21:19 +09:00
Samuel Just
da52fb63f2 ceph_dedup_tool: SampleDedupWorkerThread can simply inherit from Thread
SampleDedupWorkerThread didn't actually use any of the facilities on
CrawlerThread and indeed left several of them uninitialized.

Also allows us to remove the CrawlerThread constructor which left
several members uninitialized.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-19 13:21:12 +09:00
myoungwon oh
537462a102 qa: remove unused values in deduplication.py
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 13:21:07 +09:00
Sungmin Lee
f81b1b61c9 qa: add validation stage for deduplication.py
To validate sample-dedup actually works, validate() runs
separated thread from sample-dedup and verifies
two following things.
1. check sample-dedup starts properly.
2. check references of all the chunk objects' in chunk tier
   exists in designated base pool.
This routune repeats for max_valication_cnt times while
sample-dedup is running. If it doesn't raise any fail while the loop,
we can pretend sample-dedup works accurately.
If not, assert() will stop this test.

In case that a reference of chunk object doesn't exist in base pool,
validate() gives a second chance after repairing it (chunk-repair op)
to deal with false-positive reference inconsistency.

Signed-off-by: Sungmin Lee <sung_min.lee@samsung.com>
2022-09-19 13:21:00 +09:00
myoungwon oh
8d2cf96ff9 src/tools/ceph_dedup_tool: fixes to daemonize the process correctly
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 13:20:53 +09:00
myoungwon oh
cbe534e0b5 qa: add deduplication test under RGW workload (s3)
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 13:20:46 +09:00
zdover23
62e1390d93
Merge pull request #48157 from zdover23/wip-doc-2022-09-18-rados-network-config-prompt-correction
doc/rados: update prompts in network-config-ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-09-18 08:11:48 +10:00
Zac Dover
e26c1d8574 doc/rados: update prompts in network-config-ref
This PR adds unselectable prompts to network-config-
ref.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-09-18 06:59:47 +10:00
Casey Bodley
da11f77265 rgw: remove unused AioThrottles in rgw_op.cc
throttle creation was moved to RadosStore::get_atomic_writer(),
RadosStore::get_append_writer(), and RadosMultipartUpload::get_writer()

these instances in rgw_op.cc were unused

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-09-17 11:42:11 -04:00
Patrick Donnelly
978b3494c3
qa: turn fail_fs off in case default changes
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-16 16:10:43 -04:00
Ken Dreyer
bacbfccb20 win32: remove libcurl
The Windows client does not use libcurl for anything. Remove it to
simplify the build process.

Note, if we ever add libcurl back on Windows, we should disable unused
protocols to harden the build:

  --disable-ftp --disable-ldap --disable-ldaps --disable-rtsp \
  --disable-dict --disable-telnet --disable-tftp --disable-pop3 \
  --disable-imap --disable-smb --disable-smtp --disable-gopher \
  --disable-mqtt --disable-manual --disable-ntlm

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-16 14:30:39 -04:00
Ken Dreyer
1b4db7c477 win32: install wget when building on ubuntu
The build process requires wget. Install this on Ubuntu platforms.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-16 13:52:24 -04:00
Ken Dreyer
100990cfa6 win32: install deps with DEBIAN_FRONTEND=noninteractive
This simplifies the installation process when I run it in an interactive
terminal.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-16 13:51:58 -04:00
Nizamudeen A
e4638a14ec
Merge pull request #47849 from torchiaf/fix/37327
mgr/dashboard: Add details to the modal which displays the `safe-to-d…

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2022-09-16 20:49:48 +05:30
Nizamudeen A
ab52b8cc71
Merge pull request #47958 from rhcs-dashboard/edit-host-labels-fix
mgr/dashboard: fix Expected to find element: `cd-modal .badge but never found it

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2022-09-16 20:47:50 +05:30
Ken Dreyer
bff19ba720 win32: skip installing openssl man pages
This speeds up the build a bit and makes the logs easier to read.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-16 10:49:13 -04:00
Adam King
d987df1c80
Merge pull request #45148 from guits/revert-ceph-crash-changes
Revert "src/ceph-crash.in: remove unused frame in handler()"

Reviewed-by: Teoman ONAY <tonay@redhat.com>
2022-09-16 10:27:10 -04:00
Casey Bodley
e59d743b8b
Merge pull request #47965 from cbodley/wip-rgw-drain-cb-capture-error
Revert "rgw: add coroutine drain callback capture_error()"

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2022-09-16 10:02:14 -04:00
Francesco Torchia
0d6100bbf9
mgr/dashboard: Add details to the modal which displays the safe-to-destroy result
- Add warnings type information in the case of the OSDs are not safe to destroy
- Add info type information in the case of the OSDs are safe to destroy

Fixes: https://tracker.ceph.com/issues/37327
Signed-off-by: Francesco Torchia <francesco.torchia@suse.com>
2022-09-16 14:59:11 +02:00
Yingxin
96e9f4e2b0
Merge pull request #47878 from xxhdx1985126/wip-pin_set-cpu
crimson/os/seastore/btree: reduce the avg number of queries of pin sets' parent/child lookup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-09-16 09:41:20 +08:00
Adam C. Emerson
f5bb775572 build: Remove -fno-new-ttp-matchingg flag
This was added in the upgrade to C++17. It's no longer needed since
fixing Clang compatibility got rid of non-conforming templates.

It's no longer needed and getting rid of it is a (minor) quality of
life enhancement since it gets rid of a spurious error when using
Clang based build tools (language server, etc.) while compiling with
GCC.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-09-15 20:05:57 -04:00
dparmar18
b54cb863ef qa: mds upgrade suite as per merge-hook scripts
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-09-15 23:29:35 +05:30
Venky Shankar
f0d6c7a8b5
Merge pull request #47090 from lxbsz/wip-56553
client: do not uninline data for read

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-09-15 21:45:00 +05:30
Guillaume Abrioux
a134d00530
Merge pull request #48037 from guits/fix_57085
ceph-volume: fix inventory with device arg
2022-09-15 16:58:14 +02:00
Radoslaw Zarzynski
c6fcdbc992 crimson/osd: initialize PGAdvanceMap::from inside critical section
Fixes: https://tracker.ceph.com/issues/57542
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-09-15 14:43:15 +00:00
Guillaume Abrioux
052edd5f88
Merge pull request #48117 from guits/fix-cv-func-test-filestore-create
ceph-volume/tests: fix lvm centos8-filestore-create job
2022-09-15 16:43:11 +02:00
Casey Bodley
2b888af746
Merge pull request #48054 from yuvalif/wip-yuval-fix-56572
tests: remove pubsub tests from multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kalpesh Pandya <kapandya@redhat.com>
2022-09-15 10:04:39 -04:00
Venky Shankar
7157008318
Merge pull request #47516 from nmshelke/fix-56632
qa: ignore disk quota exceeded failure in test

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-09-15 14:55:26 +05:30
Guillaume Abrioux
890e1687d0 ceph-volume/tests: fix lvm centos8-filestore-create job
/dev/vdc1 can't be zapped if it still holds an lv mapper.
let's use --destroy in the lvm zap command in order to remove
the held lv mapper before zapping the partition and recreate the partition after.

Fixes: https://tracker.ceph.com/issues/57553

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-09-15 11:07:33 +02:00
Xuehan Xu
ad2216e74b crimson/os/seastore/btree: reduce the avg number of queries of pin sets' parent/child lookup
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-09-15 15:08:21 +08:00
Yingxin
2c21cc1b0b
Merge pull request #47966 from cyx1231st/wip-seastore-move-gcprocess
crimson/os/seastore: move AsyncCleaner::GCProcess to EPM::BackgroundProcess

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-09-15 14:02:41 +08:00
J. Eric Ivancich
28d04e865c rgw: fix bool/int logic error when calling get_obj_head_ioctx
get_obj_head_ioctx returns an int where 0 indicates success. When
called in RGWRados::check_disk_state the result was treated as a
booleann with inverted logic. This fixes that error.

This was already fixed in PR #44616 as part of a larger commit, but
this PR unifies the codebase, so various backports are consistent.

This also adds additional logging.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2022-09-14 22:57:35 -04:00
Yingxin
93ea07ebaa
Merge pull request #48105 from athanatos/sjust/wip-57530
crimson/.../node.cc: fix Node initialization argument passing

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-09-15 09:26:18 +08:00
Samuel Just
420ff0b475 crimson/.../node.cc: fix Node initialization argument passing
Argument evaluation order is unspecified, clang seems to be choosing
to execute the std::unique_ptr move constructor before the call to
get(), causing the first argument to be passed as null.

https://tracker.ceph.com/issues/57530
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-14 16:33:32 -07:00
Laura Flores
f99b140983
Merge pull request #48002 from nSedrickm/accessibility-documentation
doc/dashboard: update documentation with accessibility development pr…
2022-09-14 16:43:34 -05:00
Pere Diaz Bou
cb7c58f5b5
Merge pull request #46723 from nSedrickm/dashboard-health-accessibility-update
mgr/dashboard: Improve accessibility for dashboard health page 


Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-09-14 22:02:58 +02:00
Neha Ojha
58b749a60b
Merge pull request #47668 from NitzanMordhai/wip-nitzan-test-rados-watch-notify-callbacks
pybind/rados: notify callback reconnect

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-09-14 12:11:39 -07:00
Adam King
00074425c5
Merge pull request #47937 from adk3798/tp-add-rm-settings-docs
doc/cephadm: fix tuned-profile add/rm-setting syntax example

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2022-09-14 13:36:45 -04:00
Casey Bodley
0b99852e3b
Merge pull request #47709 from zalsader/add-daos
rgw: add DAOS SAL implementation

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-09-14 13:12:03 -04:00
Yuval Lifshitz
83d67516c1
Merge pull request #48066 from yuvalif/wip-yuval-fix-57518
rgw/lua: create lua background manager before the frontends

Reviwed-by: dang, cbodley, mattbenjamin
2022-09-14 18:00:37 +03:00
Casey Bodley
6dc092c8bd Revert "rgw: add coroutine drain callback capture_error()"
This reverts commit d75ea8757d.

Fixes: https://tracker.ceph.com/issues/57400

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-09-14 10:32:17 -04:00
Laura Flores
09e777ce1b
Merge pull request #43655 from liewegas/auth-rotate
mon,auth,cephadm: support auth key rotation
2022-09-14 09:18:46 -05:00
Adam King
2705b81005
Merge pull request #47967 from yangxiaolaing/cephadm_doc
doc/cephadm: Add creation method of OSD

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2022-09-14 08:37:46 -04:00
luo rixin
afdb5e4271 os/bluestore: use direct write in BlueStore::_write_bdev_label
On AArch64 with kernel page size 64K, it occurs occasionally
"OSD::init(): unable to read osd superblock" when deploying osd.
As bluestore use direct write to write the superblock at 0x2000~1000
and BlueStore::_write_bdev_label use buffer write to write label at
0x0~1000, The OS flush the buffer write algined to page size, it will
overwrite the superblock(0x2000~1000). Use driect write to avoid
overwriting the superblock.

Fixes: https://tracker.ceph.com/issues/57537
Signed-off-by: luo rixin <luorixin@huawei.com>
2022-09-14 20:26:32 +08:00
Radoslaw Zarzynski
70bb441f9d crimson/osd: improve debugs in PGAdvanceMap
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-09-14 11:37:38 +00:00
Radoslaw Zarzynski
a1dc9a28e1 crimson/osd: PGAdvanceMap ops complete in order
For the analysis refer to the bug tracker.

Dropping the `handle.exit()` looks like the simplest solution,
far simpler than e.g. introducing extra future-promise.

Fixes: https://tracker.ceph.com/issues/57536
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-09-14 11:37:38 +00:00
John Mulligan
b2bd88f7ca cephadm: use build.py in cmake configuration
This change replaces the call to build.sh with a call to build.py.

It also makes the build.py script a dependency to the cmake custom to
ensure we rebuild the cephadm binary if build.py changes.  Since, if one
changes the build script used to make the binary we should rebuild the
output as the new script may indeed change the intended output.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00
John Mulligan
6ba6981148 cephadm: add a pythonic build.py for constructing the cephadm binary
As discussed in a ceph orch meeting, the build.sh script was deemed
"unpythonic". This script is a first attempt to do it more pythonic
with fewer explicit version checks.

This change minimizes the existing build.sh to simply call build.py.
We can completely eliminate build.sh at a later time if desired.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00
Michael Fritch
95ead7401f qa/tasks/cephadm: conditionally pull cephadm binary from chacra
When a specific (older) version of cephadm is needed for a test, pull
it from chacra. If we need to bootstrap an older version of ceph, we
pull an older "single-script" binary using the older method.

Co-authored-by: Michael Fritch <mfritch@suse.com>
Co-authored-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00
Michael Fritch
72181fc14c qa/tasks: add chacra util
to query shaman and pull a binary from chacra

Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00