Commit Graph

128634 Commits

Author SHA1 Message Date
Yingxin Cheng
373828e381 crimson/osd: print message when customize parameters
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-27 19:07:30 +08:00
Yingxin Cheng
f508a45b95 crimson/osd: allow seastar application to print its helps
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-27 19:07:30 +08:00
Yingxin Cheng
051224bbac crimson/osd: --help-seastar does no longer belong to unknown_args
Now that app_template handles --help-seastar, and prints out all
app-level usages if --help is included in command line options. There is
no need to add a dedicated --help-seastar by ourselves and translate it
to --help.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-27 19:06:47 +08:00
Neha Ojha
eb5290f690
Merge pull request #44317 from aclamk/aclamk-fix-bluefs-import
Fix ceph-bluestore-tool bluefs-import command

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2021-12-21 11:57:01 -08:00
Yuri Weinstein
e1e2070a01
Merge pull request #44328 from yaarith/gather_crashinfo-indexerror
mgr/telemetry: check that backtrace is not empty when redacting

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2021-12-21 10:04:54 -08:00
Yuri Weinstein
befc5074a4
Merge pull request #44327 from yaarith/gather_device_report-indexerror
mgr/telemetry: catch also IndexError in gather_device_report()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-12-21 10:04:25 -08:00
Yuri Weinstein
302972b0ba
Merge pull request #44314 from locallocal/master
osd: remove unused variable legacy_infos_oid in the function PG::peek_map_epoch

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-12-21 10:03:31 -08:00
Yuri Weinstein
7c326866e8
Merge pull request #43686 from myoungwon/wip-ceph-dedup-tool-make-dedup-obj
tool: add  chunk/object-dedup command

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-12-21 10:03:10 -08:00
Ernesto Puerta
cdc9f742df
Merge pull request #44190 from rhcs-dashboard/grafana-regex
monitoring/grafana: improve grafana unit tests variable substitution

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-12-21 17:58:17 +01:00
Ernesto Puerta
5a22aec48e
Merge pull request #44362 from rhcs-dashboard/kcli-e2e-improvements
mgr/dashboard: cephadm e2e job: improvements 

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-12-21 17:53:41 +01:00
Ernesto Puerta
76eb7a57da
Merge pull request #43838 from rhcs-dashboard/fix-53128-master
mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-12-21 17:33:21 +01:00
Ernesto Puerta
efb062e5c3
Merge pull request #43942 from rhcs-dashboard/host-drain-frontend
mgr/dashboard: Implement drain host functionality in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-12-21 13:33:51 +01:00
Nizamudeen A
524c340562 mgr/dashboard: Implement drain host functionality in dashboard
Fixes: https://tracker.ceph.com/issues/51587
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-12-21 16:23:17 +05:30
Neha Ojha
1f28c06f2a
Merge pull request #42750 from aclamk/wip-aclamk-bluefs-log-incremental
os/bluestore: incremental update mode for bluefs log

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2021-12-20 17:15:13 -08:00
Ernesto Puerta
45b63ebf44
Merge pull request #44368 from ceph/fix-api-tests
mgr/dashboard: run-backend-api-tests.sh: Older setuptools

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2021-12-20 23:43:18 +01:00
Adam Kupczyk
2292a4d85a os/bluestore/bluefs: Add tracking of bluefs log in noop replay mode
Keep updating bluefs log when printing content of bluefs replay log.
Without this modification we only have initial content of log.
Log can be printed by 'ceph-bluestore-tool bluefs-log-dump'.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Adam Kupczyk
ae5c5b7655 os/bluestore/bluefs: Sync BlueFS log with its allocation delta
BlueFS log is the only file that we can append to.
When we append to file we must take into consideration previously commited allocations,
otherwise update will be miscalculated.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Adam Kupczyk
baed63ab12 test/objectstore/bluefs_test: Add test for continuation of previous BlueFS log
Added test that verifies that in update mode we properly pick up delta.
BlueFS log is the only file that can be appended to, but it is done in very indirect way.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Igor Fedotov
821aadaaa9 os/bluefs: allow incremental file metadata updates in bluefs log
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Patrick Donnelly
61de84a522
Merge PR #44315 into master
* refs/pull/44315/head:
	doc/cephfs: mds default cache memory limit is now 4GB

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-12-20 14:41:43 -05:00
David Galloway
4ab2df179b
pybind/mgr/dashboard/run-backend-api-tests: Older setuptools
https://github.com/pypa/setuptools/issues/2353

Signed-off-by: David Galloway <dgallowa@redhat.com>
2021-12-20 18:01:20 +01:00
Matt Benjamin
4ff723061f
Merge pull request #42416 from galsalomon66/wip-test-albin-error-status
RGW/s3select : new-s3select-responds presto-alignments output-serialization
2021-12-20 10:50:44 -05:00
Alfonso Martínez
9aedeaf201 mgr/dashboard: cephadm e2e job: improvements
- Avoid jenkins env. error: Input file /workdir/./src/pybind/mgr/dashboard/ci/cephadm/ceph_cluster.yml not found.Leaving...
- More meaningful log output.

Fixes: https://tracker.ceph.com/issues/53666
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2021-12-20 09:12:37 +01:00
Aashish Sharma
2ba1a5a266 mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements
Ensure "Storage capacity" keeps the "Description : Value" approach ("Number of devices: X" and "Raw Capacity: Y" in different lines).Correct issue with "host by services" host count

Fixes: https://tracker.ceph.com/issues/53128
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-12-20 13:30:57 +05:30
Kefu Chai
242e8f35e8
Merge pull request #44350 from Huber-ming/fix_annotation_misspelling
rgw: fix misspelling of 'effort'

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-19 15:53:45 +08:00
Kefu Chai
9f128e94b4
Merge pull request #44277 from fengchunsong/dpdk-rte-exit
msg/async/dpdk: replacing rte_exit with ceph_assert to avoid exit fai…

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-18 22:19:50 +08:00
galsalomon66
fe0c944777 using fmt for message formatting
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
abccc5d77b fix comments; enum-class; typo; documentation
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
b13b902aa7 typo
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
784c70c496 modification for error-handling and boto3
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
72e82d45ac adding/modifying documentation for output-serialization, s3select-response, syntax support, error-handling
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
9dc21c2575 s3select: point to s3select/master(after merging pr #90)
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
f05cdb71ee in the case Presto is push-down query, it also send output-serialization parameter('&#10;'), that should be handle as '\n'
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:54 +02:00
galsalomon66
7956fb30bf s3tests.yaml points to ceph-master
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:54 +02:00
gal salomon
f128f04c60 RGW: Implement continuation, progress, stats, end s3select response
RGW/S3select: Implement output-serializationi. user may request different CSV defintions
    for output (field delimiter, row delimiter, quote handling.

    RGW/S3select: Implement presto-alignments. presto-application sends
    queries with table-alias,case insensitive, and with no-semicolon at the
    end of statement.

    RGW/s3select: zero object size issue

    Signed-off-by: Albin Antony <aantony@redhat.com>

    Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:54 +02:00
Huber-ming
88571bb94d rgw: fix misspelling of 'effort'
Signed-off-by: Huber-ming <zhangsm01@inspur.com>
2021-12-18 14:43:34 +08:00
Joseph Sawaya
8742173e20
Merge pull request #43139 from josephsawaya/rook-orch-qa
qa/tasks/rook: test reapplication of drive groups stored in mgr
2021-12-17 12:50:18 -05:00
Sage Weil
b55781d412 Merge PR #44337 into master
* refs/pull/44337/head:
	mon: prevent new sessions during shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-12-17 09:59:48 -05:00
Casey Bodley
aa98a82c95
Merge pull request #43872 from Huber-ming/reshard
rgw: init bucket index only if putting bucket instance info succeeds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:42 -05:00
Casey Bodley
34cc746b75
Merge pull request #43817 from joke-lee/UploadPartCopy-src-not-exist
rgw: fix UploadPartCopy error code when src object not exist and src bucket not exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:32 -05:00
Casey Bodley
6e0a171258
Merge pull request #43259 from paulreece42/wip-rgw-xmlfix
rgw: allow CompletedMultipartUpload in XML

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:21 -05:00
Casey Bodley
130e646c4e
Merge pull request #41119 from Huber-ming/ls_mp
rgw: fix error ListMultipart allways returns StorageClass:STANDARD

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:05 -05:00
Sebastian Wagner
834c0c1756
Merge pull request #44336 from sebastian-philipp/cephadm-fix-test-facts
mgr/cephadm: Fix test_facts

Reviewed-by: Adam King <adking@redhat.com>
2021-12-17 11:20:59 +01:00
Samuel Just
498b312b97
Merge pull request #44338 from rzarzynski/wip-crimson-abort_on_LOST_REVERT
crimson/osd: explicitly abort on LOST_REVERT in PG log.

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-12-17 01:58:58 -08:00
Samuel Just
babb8c6841
Merge pull request #44340 from rzarzynski/wip-crimson-backfill-discarding
crimson/osd: honor discarding old backfill messages

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-12-17 01:57:45 -08:00
Samuel Just
d479799bd2
Merge pull request #44345 from xxhdx1985126/wip-onode-root-fix
crimson/os/seastore: reset onode in 'SeaStore::repeat_with_onode' before the transaction gets destroyed

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-12-17 01:55:42 -08:00
Xuehan Xu
2215c8155d crimson/os/seastore: reset onode in 'SeaStore::repeat_with_onode' before the transaction gets destroyed
Onodes hold references to the onode tree extents. And if it's referencing the root extent, that root
extent is cached in the onode trees root_tracker which caches onode tree roots by transaction address.
Than root_tracker entry only gets removed when the onode(or the corresponding "super") is destroyed.
On the other hand, two non-concurrent transactions can occupy the same address. So if an onode gets destroyed
after its transaction is destroyed, there will be a chance that another transaction occupying the same
address get that not-yet-destroyed and may-be-outdated onode.

BTW, Since we already cache extents in transactions, might want to drop onode tree root_tracker later?

Fixes: https://tracker.ceph.com/issues/53651
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-12-17 14:06:09 +08:00
Xuehan Xu
0dd693cc2b crimson/os/seastore/transaction: make sure the transaction's write_set is a subset of its read_set
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-12-17 13:53:16 +08:00
Xuehan Xu
714d8ccfe3 src/vstart.sh: correct shell syntax error
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-12-17 13:31:40 +08:00
Sage Weil
12bf02acc3 Merge PR #44228 into master
* refs/pull/44228/head:
	qa/suites/orch/cephadm/osds: test 'ceph cephadm osd activate'
	mgr/cephadm/services/osd: skip found osds that already have daemons
	mgr/cephadm: allow activation of OSDs that have previously started

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-12-16 23:54:25 -05:00