Commit Graph

2634 Commits

Author SHA1 Message Date
Adam King
7c825da44a qa/cephadm: remove fsid dir before bootstrap in test_cephadm.sh
The shell commands we test beforehand can create the
/var/lib/ceph/00000000-0000-0000-0000-0000deadbeef directory
and that directory being present will block bootstrap as
it will think a cluster with this fsid alreayd exists

Fixes: https://tracker.ceph.com/issues/57290

Signed-off-by: Adam King <adking@redhat.com>
2022-08-24 15:13:15 -04:00
Laura Flores
4cf097d7e0
Merge pull request #47642 from ljflores/wip-librados-fix
qa/workunits/rados: pull librados files from "main" instead of "master"
2022-08-23 10:32:31 -05:00
Venky Shankar
a9a7d50d20
Merge pull request #47542 from batrick/i56140
tools/cephfs: add basic detection/cleanup tool for dentry first damage

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2022-08-23 16:22:29 +05:30
Venky Shankar
78871e59c0
Merge pull request #46599 from kotreshhr/rename_config_on_flush
mgr/volumes: Better handle config file on osd full scenario

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-08-23 13:25:02 +05:30
Venky Shankar
5e1625fa7c
Merge pull request #47063 from lxbsz/wip-56529
mds/client: fail the request if the peer MDS doesn't support getvxattr op

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2022-08-23 13:22:09 +05:30
Patrick Donnelly
e9730461a2
qa/workunites/fs: add first damage test script
Not part of the QA test suites because we don't want to install the
associated script.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-08-18 13:57:38 -04:00
Laura Flores
d6b46a2c77 qa/workunits/rados: pull librados files from "main" instead of "master"
Fixes: https://tracker.ceph.com/issues/57122
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-08-16 18:07:51 +00:00
Kotresh HR
0687f78650 mgr/volumes: Allow forceful snapshot removal on osd full
When the osd is full, if the snapshot has metadata set, it
can't be removed as user metadata can't be removed when osd
is full. This patch provides a way to remove the snapshot
with 'force' option while keeping the corresponding metadata
which gets removed on subvolume discover when it finds space.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 17:08:47 +05:30
Kotresh HR
a64f049614 qa: Add subvolume clone and snapshot rm tests when osd is full
Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 17:08:36 +05:30
Xiubo Li
91fcdbb58e test/libcephfs: add newops test case
Fixes: https://tracker.ceph.com/issues/56529
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-08-16 08:30:52 +08:00
Kefu Chai
8761314777
Merge pull request #46101 from myoungwon/wip-dedup-tool-update
tool: modify object-dedup command to run on manifest object

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-08-10 00:17:00 +08:00
myoungwon oh
ea749df517 osd/,rados/: modify TIER_FLUSH to implicitly make an object a manifest object
Existing object-dedup command always creates temporary object and
perform set-chunk to make the input object manifest---this is because
current tier-flush, which is called by object-dedup, works only if
the target object is manifest, otherwise it just return 0.

So, object-dedup may not work if the target is already manifest.
Moreover, it causes unnecessary overhead if the target object is not manifest.

To solve these, this commit makes existing tier-flush to
change object's state to manifest implicitly.

With this change, the only required operation when calling object-dedup
is a tier-flush and object-dedup can work with the manifest object.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-07-28 10:22:11 +09:00
Adam King
9b08edf939 qa/workunits/cephadm: update test_repos master -> main
Missed this one, it seems the test was still passing until
recently but I guess the "master" builds are now gone

Signed-off-by: Adam King <adking@redhat.com>
2022-07-25 16:46:39 -04:00
Ilya Dryomov
b2137e2058 rbd: don't default empty pool name unless namespace is specified
Commit 96f05a7956 ("rbd: delay determination of default pool name")
broke "rbd perf image iostat" and "rbd perf image iotop" GLOBAL_POOL_KEY
support (the ability to blend all rbd pools together into a single
view).

Fixes: https://tracker.ceph.com/issues/56561
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-07-16 10:04:02 +02:00
Yuri Weinstein
f9bd8922d7
Merge pull request #46114 from myoungwon/wip-dedup-tool-object-dedup-snapshot
tool/ceph-dedup-tool: add performing dedup option on cloned object

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-07-13 07:32:32 -07:00
Nikhilkumar Shelke
217c1d2bb2 qa/workunits/cephtool: check_response didn't find erasure-code string
1. If data or metadata pool is already in-use by filesystem
then it is not allowed to reuse the same pool for another
filesystems.

2. Test is failing because above(1) restrictions/checks comes
before checking erasure-code pools. Hence test is failing
and not finding expected error string in output.

3. Proposed fix checks newly added error string instead of
'erasure-code'.

4. Also adding new tests to verify string 'erasure-code'
by passing --force option so that check for pools reuse(1)
will be skipped and check for 'erasure-code' will be hit.

Fixes: https://tracker.ceph.com/issues/56384
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-06-29 12:28:55 +05:30
myoungwon oh
565320f04d tool/ceph-dedup-tool: add performing dedup option on cloned object
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-06-22 14:47:23 +09:00
Ilya Dryomov
7fb4fdbed0 mgr/rbd_support: always rescan image mirror snapshots on refresh
Establishing a watch on rbd_mirroring object and skipping rescanning
image mirror snapshots on periodic refresh unless rbd_mirroring object
gets notified in the interim is flawed.  rbd_mirroring object is
notified when mirroring is enabled or disabled on some image (including
when the image is removed), but it is not notified when images are
promoted or demoted.  However, load_pool_images() discards images that
are not primary at the time of the scan.  If the image is promoted
later, no snapshots are created even if the schedule is in place.  This
happens regardless of whether the schedule is added before or after the
promotion.

This effectively reverts commit 69259c8d37 ("mgr/rbd_support: make
mirror_snapshot_schedule rescan only updated pools").  An alternative
fix could be to stop discarding non-primary images (i.e. drop

    if not info['primary']:
        continue

check added in commit d39eb283c5 ("mgr/rbd_support: mirror snapshot
schedule should skip non-primary images")), but that would clutter the
queue and therefore "rbd mirror snapshot schedule status" output with
bogus entries.  Performing a rescan roughly every 60 seconds should be
manageable: currently it amounts to a single mirror_image_status_list
request, followed by mirror_image_get, get_snapcontext and snapshot_get
requests for each snapshot-based mirroring enabled image and concluded
by a single dir_list request.  Among these, per-image get_snapcontext
and snapshot_get requests are necessary for determining primaryness.

Fixes: https://tracker.ceph.com/issues/53914
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-21 12:46:49 +02:00
Adam King
06f265d057
Merge pull request #46506 from asm0deuz/mod_iscsi_systemd
Modify how Iscsi tcmu-runner container is started within systemd

Reviewed-by: Adam King <adking@redhat.com>
2022-06-14 09:56:49 -04:00
Venky Shankar
27f4729256
Merge pull request #45556 from mchangir/qa-add-subvolume-option-flavors
qa: add subvolume option flavors

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-13 12:29:43 +05:30
Venky Shankar
67371c1ab4
Merge pull request #46332 from lxbsz/qa-snap
qa: enlarge the tag number and test more for the snapshot

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-06-13 12:25:44 +05:30
Adam King
cf2fdca093
Merge pull request #46539 from adk3798/master-main-cleanup2
mgr/cephadm: some master -> main cleanup

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-06-07 20:51:06 -04:00
Venky Shankar
45c9fd6187
Merge pull request #46078 from kotreshhr/fuse-directory-dacs-issue
client/fuse: Fix directory DACs overriding for root

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-06-07 22:45:05 +05:30
Teoman ONAY
79c51938e8 Modify how Iscsi tcmu-runner container is started within systemd
Modify Iscsi tcmu-runner container to be run demonized in the same
systemd slice as all other ceph processes

Signed-off-by: Teoman ONAY <tonay@redhat.com>
2022-06-07 14:42:50 +02:00
Adam King
7c6c715aa5 mgr/cephadm: some master -> main cleanup
Signed-off-by: Adam King <adking@redhat.com>
2022-06-06 15:18:08 -04:00
Casey Bodley
47913cb7f1 test/rgw: fix test case for empty-OLH-name cleanup
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-31 17:29:18 -04:00
Casey Bodley
ede6a85385 qa/rgw: fix flake8 errors in test_rgw_reshard.py
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:34 -04:00
Casey Bodley
87efd66db0 test/rgw: test_rgw_reshard.py injects ECANCELED on set_target_layout/commit_target_layout
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:33 -04:00
Casey Bodley
c3ccb1adb4 test/rgw: test_bucket_reshard verifies that ACLs are preserved
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:33 -04:00
Casey Bodley
6e6f59b945 qa/rgw: disable coredumps for reshard fault injection
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:31 -04:00
Casey Bodley
2a51bc6ce1 test/rgw: add test_bucket_reshard() for fault injection testing
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:31 -04:00
Casey Bodley
8c2bac38f8 test/rgw: test_rgw_reshard.py exec_cmd() can return error code
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:31 -04:00
Shilpa Jagannath
eb304f4bc5 rgw: reshard tests with fault injection
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2022-05-27 15:47:31 -04:00
Yuri Weinstein
bc02a55d3a
Merge pull request #45409 from NUABO/tancz
mon: add proxy to cache tier options

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-24 07:59:36 -07:00
Xiubo Li
4fbf4c4f58 qa: enlarge the tag number and test more for the snapshot
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-05-19 11:08:10 +08:00
Kotresh HR
2e1f43c99b client/fuse: Fix directory DACs overriding for root
DACs are overridable for directories. For files,
Read/write DACs are always overridable but executable
DACs are overridable when there is at least one exec bit
set.

The files and directory DACS overriding were handled the
same way for root which is incorrect. This patch fixes
DACs overriding as described above for the root.

Fixes: https://tracker.ceph.com/issues/55313
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-05-17 12:15:41 +05:30
tancz1
77ad5a810c mon: add proxy to cache tier options
When we set the proxy mode to remove a writeback cache according to
the ceph official documentation an error occurred:

[root@controller-1 root]# ceph osd tier cache-mode cachepool proxy
Invalid command: proxy not in writeback|readproxy|readonly|none
osd tier cache-mode writeback|readproxy|readonly|none [--yes-i-really-mean-it]:
specify the caching mode for cache tier

According to the description of the official website document: since
a writeback cache may have modified data, you must take steps to ensure
that you do not lose any recent changes to objects in the cache before
you disable and remove it. Change the cache mode to proxy so that new and
modified objects will flush to the backing storage pool.

Fixes: https://tracker.ceph.com/issues/54576
Signed-off-by: tan changzhi <544463199@qq.com>
2022-04-29 10:40:40 +08:00
Neha Ojha
cf78cf4980
Merge pull request #46010 from gregsfortytwo/wip-55419-blocklist-test
test: Fix a blocklist API test

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-04-26 10:59:07 -07:00
Greg Farnum
63db714890 test: use the same address input format as output will generate
Otherwise, our grep fails!

Fixes: https://tracker.ceph.com/issues/55419

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2022-04-23 00:00:59 +00:00
Gregory Farnum
6536d0c1af
Merge pull request #44151 from gregsfortytwo/wip-cidr-blocklist
Implement CIDR blocklisting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-04-21 08:31:16 -07:00
Ionut Balutoiu
8054199787 qa: add basic Ceph on Windows integration test
Due to lack of Windows support in the Teuthology, the test case adopts
the following workaround:

* Deploy baremetal machine with `ubuntu_latest.yaml` and
  configure it with libvirt KVM.
* Create a libvirt VM and provision it with Windows Server 2019, using
  the official ISO from Microsoft.
* Configure SSH in the Windows VM, and run the tests remotely via SSH.

The implementation of the test case consists of workunit scripts.

`qa/workunits/windows/test_rbd_wnbd.py` is the main Python script
to test Ceph on Windows basic functionality. This is executed in the
libvirt VM configured with Windows Server 2019.

Co-authored-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-authored-by: Daniel Vincze <dvincze@cloudbasesolutions.com>

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2022-04-19 17:21:52 +03:00
Adam King
5360de431d
Merge pull request #45765 from m-ildefons/1196785-cephadm-status-trace
cephadm: avoid crashing on expected non-zero exit

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-04-15 11:05:05 -04:00
Moritz Röhrich
a02be6f22f
cephadm: avoid crashing on expected non-zero exit
- Avoid crashing when a call out to an external program expectedly does
  not return exit status zero.

There are programs that communicate other information than error/no
error through exit status. E.g. `systemctl status` will return different
exit codes depending on the actual status of the units in question.
In cases where this is expected crashing with a RuntimeError exception
is inappropriate and should be avoided.

Fixes: https://tracker.ceph.com/issues/55117
Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
2022-04-13 17:22:22 +02:00
Greg Farnum
1ca89ca76d test: check range blocklist in cephtool/test.sh
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2022-04-13 00:14:16 +00:00
Greg Farnum
8246892172 test: fix negative blocklist tests
These tests are supposed to be validating we don't accept invalid IPs,
but they left out the "add" subcommand so they're all failing on that!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2022-04-13 00:14:16 +00:00
Ilya Dryomov
88db8db1b4
Merge pull request #45798 from adk3798/iscsi-only-pid-limit
cephadm: only apply unlimited pids-limit to iscsi and rgw

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 16:02:50 +02:00
Ilya Dryomov
a23b9c9938 qa: test_iscsi_pids_limit.sh: increase sleep time
It could take longer than 30 seconds to fork off 40000 processes on
a busy system.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 13:15:22 +02:00
Milind Changire
8d9bf97a58 qa: replace subvolume.sh with test_subvolume.py
qa/workunits/fs/misc/subvolume.sh is getting in the way of fs:workload
testing with subvolumes. Hence moved this script to a python test.

Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-04-10 21:35:23 +05:30
Casey Bodley
1ab04ddc38
Merge pull request #45304 from dvanders/cleanup_46456
rgw_reshard: drop olh entries with empty name

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-04-08 09:05:33 -04:00
Dan van der Ster
16511fd15f workunits/rgw: reshard cleans empty name entries
Check that reshard prunes olh entries with empty name.

Fixes: https://tracker.ceph.com/issues/54500
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2022-04-07 18:12:06 +02:00