Commit Graph

131253 Commits

Author SHA1 Message Date
Paul Cuzner
ccb47b06f6 cephadm: Add support for mpath devices to gather-facts
multipath devices typically show as 2 device nodes, so
gather-facts needs to use the serial number of the
device to ensure the disk/flash counts and capacity
calculations are not twice as big as they should be.

In addition to logic changes,
- the device data includes new fields: disk_type (hdd|flash),
  serial, and alt_dev_name. The alt_dev_name shows the
  alternate device name for the same serial number (when a
  serial number is not present, this is just '')
- enclosure information (SES only) is included based
  on device serial number. This maps serial numbers to slots
  and provides a breakdown of the slot states
- the supported block device list now excludes loopback and
  mdraid (/dev/md*) devices

Fixes: https://tracker.ceph.com/issues/55489

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2022-05-13 12:59:18 +12:00
Samuel Just
ecd871a098
Merge pull request #46244 from tchaikov/wip-bump-up-seastar
crimson: bump up seastar

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-12 12:43:55 -07:00
Nizamudeen A
61ed55b8f7
Merge pull request #46227 from rhcs-dashboard/angular-production-fix
mgr/dashboard: avoid overriding the configuration in package.json

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-05-12 20:33:06 +05:30
Adam King
c6e63de4fc cephadm: Revert "mgr/cephadm: fixing call method timeout handling"
This reverts commit 55d42d0c31.

Fixes: https://tracker.ceph.com/issues/55623

Signed-off-by: Adam King <adking@redhat.com>
2022-05-12 10:09:43 -04:00
Kefu Chai
0ef91ba0d5 seastar: bump up seastar submodule to recent master
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-05-12 21:30:50 +08:00
Kefu Chai
29ed445646 crimson: replace DERIVE metrics with COUNTER metrics
in a recent change in Seastar, DERIVE metric was dropped in favor
of COUNTER.

in this change, all DERIVE metrics are replaced with COUNTER metrics.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-05-12 21:30:50 +08:00
Kefu Chai
56875e9c26
Merge pull request #46239 from tchaikov/wip-crimson-nbd
crimson/tools/store_nbd: s/get_paddr/get_val/

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-05-12 20:21:55 +08:00
Ernesto Puerta
62dbb254a6
Merge pull request #46231 from rhcs-dashboard/update-id-email
githubmap: Email and Id updated

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-05-12 14:08:50 +02:00
Venky Shankar
153634f1de
Merge pull request #45950 from rishabh-d-dave/qa-cephfs-test_acls
qa/cephfs: minor fixes for xfstests_dev.py and test_acls.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:21:14 +05:30
Venky Shankar
29793ca78c
Merge pull request #45829 from dparmar18/fix_put
cephfs-shell: fix put and get cmd

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:16:14 +05:30
Venky Shankar
44a0199114
Merge pull request #45834 from ajarr/wip-53996
qa/suites/fs/upgrade: change octopus installations and references

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:14:53 +05:30
Venky Shankar
18681c5d51
Merge pull request #46161 from joscollin/wip-B55516-extra-data
qa: fix is_addr_blocklisted() to get blocklisted clients from 'osd dump' 

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:12:57 +05:30
Venky Shankar
806551ad95
Merge pull request #46192 from rishabh-d-dave/qa-cephfs-mount-omit_sudo-no-effect
qa/cephfs: omit_sudo must be passed to underlying method...

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:11:01 +05:30
Kefu Chai
9b66089601
Merge pull request #45185 from a16bitsysop/quincy-32bit
src/os/bluestore/BlueStore.cc: Use static_cast<uint64_t> fix for 32bit

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-05-12 17:24:07 +08:00
Kefu Chai
bf23e257af crimson/tools/store_nbd: s/get_paddr/get_val/
it is a follow-up change of ec89b17f2a
which breaks the build of the nbd tool.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-05-12 17:04:45 +08:00
Duncan Bellamy
6190f90a5d src/os/bluestore/BlueStore.cc: Use static_cast<uint64_t> to fix 32bit compilation
Fixes: https://tracker.ceph.com/issues/54415

Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>
2022-05-12 13:36:44 +08:00
Venky Shankar
70ef761997 doc: add setting/unsetting mon_mds_skip_sanity config for manual ceph-mds upgrade
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 10:55:02 +05:30
Samuel Just
6f78f2f42e
Merge pull request #46136 from athanatos/sjust/wip-crimson-operation-refactor-for-review
crimson: refactor osd operation pipeline

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-05-11 19:06:53 -07:00
Samuel Just
12d0955b5f
Merge pull request #45390 from xxhdx1985126/wip-backref2
crimson/os/seastore: add back-reference

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-05-11 13:51:13 -07:00
Casey Bodley
98ddd40859 qa/rgw: use 'with-sse-s3' override for s3tests
don't rely on the ceph manager task to parse a config file. each rgw
could be using a different config. instead, revert to an s3tests
override called 'with-sse-s3'

this way, the only job that enables sse-s3, vault_transit.yaml, contains
both the 'rgw crypt sse s3' configurables, and the flag to enable the
associated test cases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 14:49:51 -04:00
Casey Bodley
864f457a84 Revert "qa/rgw - run sse-s3 test cases only if configured or requested"
This reverts commit 9235cf95c1.

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 14:45:23 -04:00
Soumya Koduri
98087586c4
Merge pull request #46126 from soumyakoduri/wip-skoduri-dbstore-vstart
rgw/dbstore: Support user creation via `radosgw-admin`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 23:31:29 +05:30
Sarthak0702
7f05a9089a githubmap: Email and Id upadted
Signed-off-by: Sarthak0702 <sarthak.dev.0702@gmail.com>
2022-05-11 23:24:46 +05:30
Casey Bodley
f0e30f67ab
Merge pull request #45064 from s-t-e-v-e-n-k/no-direct-use-of-nose
test: No direct use of nose

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 12:28:48 -04:00
Yuri Weinstein
3bc0354bc4
Merge pull request #45762 from NitzanMordhai/wip-nitzan-test-cls-lock-expiered-locks
cls_lock: check expired lock before unlock

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2022-05-11 07:34:42 -07:00
Yuri Weinstein
6322d1aaae
Merge pull request #45529 from NitzanMordhai/wip-nitzan-pglog-dups-not-trimmed
osd/PGLog.cc: Trim duplicates by number of entries

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-05-11 07:34:12 -07:00
Yuri Weinstein
7268e4574c
Merge pull request #44839 from NitzanMordhai/wip-nitzan-mon-ok-to-stop
mon/Monitor: Validate monitor ids during 'mon ok-to-stop' cmd

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-11 07:32:49 -07:00
Nizamudeen A
48fe886bb5 mgr/dashboard: avoid overriding the configuration in package.json
the warning in npm run build is `Option "configuration" was already specified with value "en-US". The
new value "production" will override it.`

This causes the dist for production env to be created in dist/ whereas
it should be dist/en-US (or whatever npm_config_locale. That prevented
from dashboard to load and cause the cephadm e2e checks to fail.

https://jenkins.ceph.com/job/ceph-dashboard-cephadm-e2e-nightly-master/185/consoleFull#57532618373a8703b-5adb-41c5-84a0-8cf4e065ba3d

Related to https://github.com/ceph/ceph/pull/46143

Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-05-11 19:41:33 +05:30
dparmar18
1e3fb3c874 qa: testcase test_get_without_target_name fixed
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:16:29 +05:30
dparmar18
db8ff8b809 doc/man/cephfs-shell.rst: get cmd must accept both path mandatorily
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:16:28 +05:30
dparmar18
689927b0e5 cephfs-shell: get cmd must get both path and should validate them
Description:
- While using `get` command, `local_path` parameter is optional. Changing it
  to mandatory.
  - Rationale: Till now, there used to be a default path of `local_path` as
               `default='.'` but wasn't mentioned anywhere. It led to confusion.
               On top of it, considering get command to be a ssh inspired utlity,
               or any other CLI tool that copies file between filesystems, source
               and destination path are always mandatory. Therefore in order to
               simulate this behavior in cephfs-shell`s command(s), my opinion is
               to make get command accept both the paths.

- Added checks to make sure:
	1) File does exist at `remote_path`
	2) File with the same name doesn't exist in `local_path`
	3) Removed code that would run through the directory and if it finds
	   nothing in `root_src_dir`, then it will try to do:
	   `os.makedirs(root_dst_dir + b'/' + root_src_dir)`, but it will
	   never be empty as 1) takes care of it.

Fixes: https://tracker.ceph.com/issues/55216

Signed-off-by: dparmar18 <dparmar@redhat.com>
2022-05-11 17:16:28 +05:30
dparmar18
aa9938aacb qa: added testcase to check get cmd doesn't create subdirs on its own
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:16:28 +05:30
dparmar18
4da419ed24 qa: path in test_get_with_target_name() fixed
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:16:28 +05:30
dparmar18
64df8c2cdf cephfs-shell: get command should never create a directory on its own
Descrption: When using the get command on a single file, it would append the remote path to
            source path and create directories that shouldn't be created. For instance,
            file 'foo.txt' resides at `/dir1/dir2/` and `get` command is used to copy it to
            /tmp/foo then it would do `/tmp/dir1/dir2/foo.txt` which is not the expected
            behavior. Therefore this PR intends to correct this behavior.

Fixes: https://tracker.ceph.com/issues/55112

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:16:28 +05:30
dparmar18
31ce2eff60 qa: removed test_without_target_dir
Rationale: get and put now demand both the paths mandatorily.
           Also testing of get and put without target paths
           have been take of in other tests in class TestGetAndPut().

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:11:27 +05:30
dparmar18
70e21a92df doc/man/cephfs-shell.rst: put cmd must accept both path mandatorily
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:10:36 +05:30
dparmar18
e52fb12c39 qa: added tests to make sure put accepts both paths and validate local path
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-11 17:10:36 +05:30
dparmar18
cfeab92ec8 cephfs-shell: put cmd must accept both paths and validate local_path
Description:

- `put` command didn't display any error when file at local_path was not
   found. This PR intends to add that check.
   - Rationale: Till now, there used to be a default path of `remote_path` as
                `default='.'` but wasn't mentioned anywhere. It could lead to confusion.
                On top of it, considering put command to be a ssh inspired utlity,
                or any other CLI tool that copies file between filesystems, source
                and destination path are always mandatory. Therefore in order to
                simulate this behavior in cephfs-shell`s command(s), my opinion is
                to make put command accept both the paths.

Fixes: https://tracker.ceph.com/issues/55242

Signed-off-by: dparmar18 <dparmar@redhat.com>
2022-05-11 17:10:36 +05:30
Volker Theile
4281dc1bbc cephadm: prometheus: The generatorURL in alerts is only using hostname
Prometheus is currently using only the hostname in the 'generatorURL' of an alert which causes issues when clicking on the URL in the Ceph Dashboard or somewhere else, because in most cases the hostname of the node that is running the Prometheus container is not resolvable.

To fix that the command line argument '--web.external-url' must be appended in the systemd unit file of the Prometheus container, e.g. '--web.external-url http://foo.bar:9095' whereas a FQDN hostname is used.

Fixes: https://tracker.ceph.com/issues/55595

Signed-off-by: Volker Theile <vtheile@suse.com>
2022-05-11 12:42:18 +02:00
Ronen Friedman
a1c5052cc7 osd: cleanup object_stat_sum_t initialization
Fix the confusing combination of default-member-initializers
for some members and direct initialization of others in
the ctor.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-05-11 09:50:52 +00:00
Omri Zeneva
6ec1565b14 rgw: support conditional tracing using lua script
before lua script is being executed, we keep the tracer runtime configuration value, and then decides whether to trace or not the request based on the value that maybe changed during lua exeuction, so we can disable/enable tracing for request even if the tracer is in the opposite state at the same time

Signed-off-by: Omri Zeneva <ozeneva@redhat.com>
2022-05-11 11:10:02 +03:00
Ronen Friedman
ce7e7e512e test: fixing compilation issues with kv_store_bench
Existing code relied on an include path that had
specified 'using std'.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-05-11 06:28:15 +00:00
Ronen Friedman
b7d2b25232 test/osd/scrub: scrubber backend test files
introducing the scrubber_generators to create
scrubber test data, and the scrubber_test_datasets
for pre-prepared test configurations.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-05-11 06:28:15 +00:00
Ronen Friedman
2852ea77cb test/scrub/osd: scrubber-related interface changes
required to facilitate testing

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-05-11 06:12:28 +00:00
Venky Shankar
6f459d8bf2
Merge pull request #46164 from rishabh-d-dave/vstart-minor-fixes
qa/vstart_runner: minor bug fix and minor adjustments

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-11 11:26:09 +05:30
Samuel Just
d22869b24f
Merge pull request #46217 from Matan-B/wip-matanb-c-shard_services
crimson/osd: Moving shard_services to PGBackend

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-10 21:50:27 -07:00
Kefu Chai
7f022510a7
Merge pull request #46208 from rzarzynski/wip-crimson-main-using-cleanups
crimson/osd: drop "using ..." from main.cc

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-05-11 10:41:06 +08:00
Venky Shankar
5e1ec0cb49
Merge pull request #46110 from dparmar18/fix_onecmd
cephfs-shell: make onecmd() print proper error msg

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 22:58:41 +05:30
Adam King
acf17ed457 mgr/cephadm: don't use cached conns for hosts not in cluster
Fixes: https://tracker.ceph.com/issues/55596

Signed-off-by: Adam King <adking@redhat.com>
2022-05-10 13:27:44 -04:00
Venky Shankar
8cfef7194c
Merge pull request #46047 from dparmar18/qa_testcephfs_shell_pep8_fixes
qa/cephfs/test_cephfs_shell: Fixed pep8 formatting violations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 22:57:16 +05:30