Commit Graph

129450 Commits

Author SHA1 Message Date
Casey Bodley
1d25e87ed3
Merge pull request #44893 from cfsnyder/wip-54130-oplog-segfault
rgw: fix segfault in OpsLogRados::log when realm is reloaded

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-05 08:54:57 -05:00
Venky Shankar
0e37e6a7be
Merge pull request #44851 from batrick/i54081
mds: add inline flag to MDS bootstrap compat in MDSMap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-05 18:04:29 +05:30
Samuel Just
83200ce754
Merge pull request #44887 from josephsawaya/vstart-dev
src/vstart.sh: Add --no-restart to vstart & add mkfs logs to osd.stdout

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-04 21:26:11 -08:00
Joseph Sawaya
fc3b6273f3 src/vstart.sh: Add --no-restart to vstart & add mkfs logs to osd.stdout
The output of mkfs wasn't also being included in the OSD's log before this which
can make it more difficult to debug issues with mkfs.

ceph-run restarting every 5 seconds can make it difficult to read the osd's stdout.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2022-02-04 16:07:35 -05:00
Neha Ojha
8a8945e640
Merge pull request #44868 from neha-ojha/wip-move-to-stream
qa/distros: remove centos8

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-04 11:56:08 -08:00
Casey Bodley
f692731e2a
Merge pull request #44866 from yuvalif/wip-yuval-fix-datalog-list
rgw/admin: fix radosgw-admin datalog list max-entries issue

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-02-04 14:55:07 -05:00
Casey Bodley
ca8e0a64b1
Merge pull request #44386 from ivancich/wip-reshard-tentative-goal
rgw: in bucket reshard list, clarify new num shards is tentative

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2022-02-04 14:54:36 -05:00
Casey Bodley
235b51b786
Merge pull request #42330 from SWJTU-ZhangLei/wip-rgw-postobj
rgw: RGWPostObj::execute() may lost data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-04 14:50:45 -05:00
Casey Bodley
425100334b
Merge pull request #44743 from linuxbox2/wip-rgwlc-lcgetv
rgwlc: fix compat-decoding of cls_rgw_lc_get_entry_ret

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-04 14:50:27 -05:00
Casey Bodley
70630b9644
Merge pull request #44137 from ivancich/wip-fix-reshard-cancel
rgw: "reshard cancel" errors with "invalid argument"

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-04 14:50:10 -05:00
Casey Bodley
1762305bb4
Merge pull request #44892 from trociny/wip-54119
rgw: check bucket shard init status in RGWRadosBILogTrimCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-04 14:49:47 -05:00
Neha Ojha
e84992348e
Merge pull request #44563 from benhanokh/NCB_new_alloc_map
BlueStore:NCB:Bug-Fix for recovery code with shared blobs

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
2022-02-04 09:31:16 -08:00
Ernesto Puerta
6a4b1e148d
Merge pull request #44796 from pcuzner/remove-old-mib
monitoring: remove old MIB

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-02-04 17:42:08 +01:00
Ernesto Puerta
63f5fa45fe
Merge pull request #44857 from rhcs-dashboard/test-doc-dash-cephadm
doc: update kcli test env documentation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-02-04 17:35:54 +01:00
Ernesto Puerta
84287bdbb6
Merge pull request #44059 from MrFreezeex/mixins
mgr/dashboard: monitoring: refactor into ceph-mixin

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2022-02-04 17:28:44 +01:00
Daniel Gryniewicz
a7e8d68d5e
Merge pull request #44862 from andriytk/fix-put-memleak2
rgw_sal_motr: fix possible memleak on put
2022-02-04 11:24:58 -05:00
Casey Bodley
a2d2ed2d8e
Merge pull request #44348 from soumyakoduri/wip-skoduri-lc-teuthology
rgw/qa: Add test suite for lifecycle cases

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-04 11:04:13 -05:00
Gabriel BenHanokh
8868894491 os/BlueStore: NCB fixes recovery code with shared blobs
Replaces the BitmapAllocator used by NCB Recovery code with a dedicated SimpleBitmap.
The SimpleBitmap allows for bits to be set multiple times without any adverse effect.
This is needed beacuse shared-blobs will report the same allocation multiple times.

Fixes: https://tracker.ceph.com/issues/53678

Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
2022-02-04 07:35:31 +02:00
Neha Ojha
c2a3676b29
Merge pull request #44003 from zdover23/wip-doc-2021-11-17-start-documenting-ceph-python-pip-update
doc/start: update python-pip to python3-pip

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-02-03 15:36:22 -08:00
Neha Ojha
630b43d851
Merge pull request #44560 from jdurgin/wip-edit-on-github
doc/conf.py: use links to the 'edit' page for edit on github

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-02-03 15:28:16 -08:00
Samuel Just
2b955b9e3b
Merge pull request #44888 from josephsawaya/zns-dev
crimson/os/seastore: change segment_off_t to seastore_off_t in ZNSSegmentManager

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-03 15:00:47 -08:00
Patrick Donnelly
4fbd42ab5b
Merge PR #44691 into master
* refs/pull/44691/head:
	doc: document PI .qa symlinks
	qa: fix or add missing .qa links
	qa: move .qa terminator

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-02-03 16:40:07 -05:00
Cory Snyder
0713f65355 rgw: fix segfault in OpsLogRados::log when realm is reloaded
We weren't previously handling the deallocation of the store when
a realm was reloaded. Now passing a const reference to the pointer.

Fixes: https://tracker.ceph.com/issues/54130
Signed-off-by: Cory Snyder <csnyder@iland.com>
2022-02-03 21:31:17 +00:00
Andriy Tkachuk
9d1488f2b3 rgw_sal_motr: fix possible memleak on put
Currently, the MotrAtomicWriter::cleanup() is called from
MotrAtomicWriter::commit(), which may not be called at all
by rgw in case of md5 checksum failure.

Solution: call cleanup() from process() when data is zero.
rgw calls Writer::process(data, off) with zero data at the
end of the loop to allow writes to flush the data. From:
src/rgw/rgw_op.cc:RGWPutObj::execute():

      op_ret = filter->process(std::move(data), ofs);
      ...
      ofs += len;
    } while (len > 0);

    // flush any data in filters
    op_ret = filter->process({}, ofs);

Signed-off-by: Andriy Tkachuk <andriy.tkachuk@seagate.com>
Reviewed-by: Sining Wu <sining.wu@seagate.com>
2022-02-03 20:44:04 +00:00
Andriy Tkachuk
a9a2955297 rgw_sal_motr: fix cosmetic issues
- rename acc_bl to acc_data
- drop DEBUG prefix from logs

Signed-off-by: Andriy Tkachuk <andriy.tkachuk@seagate.com>
2022-02-03 20:42:17 +00:00
Neha Ojha
75c4960abf
Merge pull request #44878 from aclamk/fix-git-id-aclamk
github: fix id and mail

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-02-03 09:39:13 -08:00
Joseph Sawaya
606939cbf2 crimson/os/seastore: change segment_off_t to seastore_off_t
Signed-off-by: Joseph Sawaya <josephsawaya938@gmail.com>
2022-02-03 11:49:25 -05:00
Patrick Donnelly
d341c5b773
mds: add inline feature to MDS bootstrap incompat
File systems that had inline data enabled at some point would have this
bit in the CompatSet "incompat" set. This would conflict during upgrade
with the default v16.2.4 CompatSet assigned to existing (16.2.4-) MDS.
Subsequently, this would cause an assertion in FSMap::sanity during
pending map creation.

This bit will get added anyway during the upgrade process so might as
well add it to the MDS CompatSet during bootstrap.

Fixes: https://tracker.ceph.com/issues/54081
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-03 11:02:02 -05:00
Patrick Donnelly
e2c461e02a
mds: throw some feature definitions in static memory
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-03 11:02:02 -05:00
Patrick Donnelly
445cdd4120
qa: test inline compat set on older MDSMap
Reproduced here:

    /ceph/teuthology-archive/pdonnell-2022-01-31_19:13:02-fs:upgrade-master-distro-default-smithi/6651572/teuthology.log

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-03 11:01:59 -05:00
Patrick Donnelly
547b21c0de
doc: document PI .qa symlinks
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-03 10:08:30 -05:00
Patrick Donnelly
1f714da814
qa: fix or add missing .qa links
Using this command:

    find qa/suites/ -type d -execdir ln -sfT ../.qa/ {}/.qa \;

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-03 10:08:30 -05:00
Patrick Donnelly
ec6cd23b20
qa: move .qa terminator
So links can be elsewhere in the qa suite (not used yet) and to simplify
a find command in a follow-up commit.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-03 10:08:30 -05:00
Adam Kupczyk
caa5a8ae62 github: fix id and mail
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2022-02-03 15:43:41 +01:00
Arthur Outhenin-Chalandre
8ff1e6b399
monitoring: build jsonnet/jb only for testing
Build jsonnet and jb in the testso that we can build ceph without
internet access and still be able to run the test needed for monitoring
using jsonnet tools.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2022-02-03 13:08:37 +01:00
Arthur Outhenin-Chalandre
ecaf9070ae
spec: debian: monitoring: build jsonnet from source to use 0.18.0
As this new version is recently released it's still not in every distro
we use. We now build jsonnet from source so that we can use this new
version of jsonnet. This commit could be reverted later on when the new
version would be available everywhere.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2022-02-03 13:08:36 +01:00
Arthur Outhenin-Chalandre
98236e3a1d
mgr/dashboard: monitoring: refactor into ceph-mixin
Mixin is a way to bundle dashboards, prometheus rules and alerts into
jsonnet package. Shifting to mixin will allow easier integration with
monitoring automation that some users may use.

This commit moves `/monitoring/grafana/dashboards` and
`/monitoring/prometheus` to `/monitoring/ceph-mixin`. Prometheus alerts
was also converted to Jsonnet using an automated way (from yaml to json
to jsonnet). This commit minimises any change made to the generated files
and should not change neithers the dashboards nor the Prometheus alerts.

In the future some configuration will also be added to jsonnet to add
more functionalities to the dashboards or alerts (i.e.: multi cluster).

Fixes: https://tracker.ceph.com/issues/53374
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2022-02-03 13:08:20 +01:00
Arthur Outhenin-Chalandre
e102620394
spec: debian: add golang as build dependency
Add golang as a build dependency to build golang project in the test
for monitoring/ceph-mixin.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2022-02-03 09:49:55 +01:00
Mykola Golub
02c6a56f49 rgw: check bucket shard init status in RGWRadosBILogTrimCR
to avoid using not properly initialized bs.bucket_obj.

Fixes: https://tracker.ceph.com/issues/54119
Signed-off-by: Mykola Golub <mgolub@suse.com>
2022-02-03 08:16:20 +00:00
Pere Diaz Bou
6c0c75fa17
Merge pull request #44812 from rhcs-dashboard/box-remove-error
cephadm/box: fix remove image tar error
2022-02-03 07:49:46 +01:00
Yingxin
18618bf6d4
Merge pull request #44832 from cyx1231st/wip-crimson-msgr-test-timeout
test/crimson/test_messenger: increase timeout

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-02-03 11:56:38 +08:00
Neha Ojha
8ca5729d21 qa/suites/rados/thrash-old-clients: remove centos_8.3_container_tools_3.0
Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-02-02 23:26:54 +00:00
Neha Ojha
77b0791760 qa/distros/container-hosts: remove centos_8.2 and centos_8.3
Related to https://tracker.ceph.com/issues/54087

Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-02-02 20:03:33 +00:00
Neha Ojha
8f1d1b7c55 qa/distros/podman: remove centos_8.2 and centos_8.3
Related to https://tracker.ceph.com/issues/54087

Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-02-02 20:02:48 +00:00
Soumya Koduri
9dfe5ac714 rgw/qa: Add test suite for lifecycle cases
Execute lifecycle s3-tests in the teuthology test-suite by configuring
required storage classes and 'rgw lc debug interval' option.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-03 00:16:03 +05:30
Ernesto Puerta
c47ace9215
Merge pull request #43707 from BenoitKnecht/ceph-mgr-service-id
mgr: Fix ceph_daemon label in ceph_rgw_* metrics

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-02-02 18:39:57 +01:00
Yingxin Cheng
82af52e8b7 test/crimson/test_messenger: increase timeout
120 seconds are still too short for make check, change to 300.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-02 23:24:49 +08:00
Yuval Lifshitz
bd429ed9be rgw/admin: fix radosgw-admin datalog list max-entries issue
Fixes: https://tracker.ceph.com/issues/54116

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-02-02 16:53:21 +02:00
Ernesto Puerta
8505861220
Merge pull request #44609 from rhcs-dashboard/daemon_action
mgr/dashboard: perform daemon actions

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-02-02 14:06:57 +01:00
Nizamudeen A
8feb2b8fe0 doc: update dashboard kcli test env documentation
Fixes: https://tracker.ceph.com/issues/54105
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-02-02 13:50:00 +05:30