Commit Graph

136605 Commits

Author SHA1 Message Date
Zac Dover
235d2e8b67 doc/glossary: add "application" to the glossary
Add an admittedly-stub definition of "application" to the glossary.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-26 04:50:48 +10:00
Rishabh Dave
b88e917461 qa/cephfs: upgrade xfstests_dev.py for FUSE mounts
Add support for testing CephFS mounted with FUSE using xfstests-dev.

Fixes: https://tracker.ceph.com/issues/55354
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-25 16:03:45 +05:30
myoungwon oh
7d75557d3b test/crimson/seastore/rbm: use normal initialization path instead of open and set_block_size for test
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 13:19:20 +09:00
myoungwon oh
9a84b36181 crimson/os/seastore/rbm: add do_mount() and use it in derived classes
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 13:19:04 +09:00
myoungwon oh
4dfe1a7d16 crimson/os/seastore/rbm: only set block_size at mkfs time
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 13:18:57 +09:00
myoungwon oh
9d01a7cec6 crimson/os/seastore/rbm: move RBMDevice related functions to rbm_device.cc
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:10:28 +09:00
myoungwon oh
c1fb998eca crimson/os/seastore/rbm: drop set_device_id() in RBMDevice
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:10:21 +09:00
myoungwon oh
bc3ab64c05 crimson/os/seastore/rbm: remove read_rbm_header in BlockRBManager::open(), and use mount to load superblock
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:10:15 +09:00
myoungwon oh
60e4407617 crimson/os/seastore/rbm: reset stream_index_to_open in NVMeBlockDevice at close()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:10:07 +09:00
myoungwon oh
70d0145c35 crimson/os/seastore/rbm: use do_with() to prevent from freeing allocated memory
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:10:00 +09:00
myoungwon oh
c0305cedf2 crimson/os/seastore/rbm: add mount() to NVMeBlockDevice
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:09:53 +09:00
myoungwon oh
1e9de00aa8 crimson/os/seastore/rbm: In mkfs, add open() before write_rbm_header()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:09:45 +09:00
myoungwon oh
8e5a3f229d crimson/os/seastore/rbm: generalize shared logic and set cbjournal_size in mkfs()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:09:32 +09:00
myoungwon oh
1bd4acc318 crimson/os/seastore/rbm: do not set block_size in open()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:08:35 +09:00
myoungwon oh
1bea8348de crimson/os/seastore/rbm: fix stat_deivce() to return correct device size
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:08:29 +09:00
myoungwon oh
2e4ab7fdba crimson/os/seastore: create random_block_device using make_device at startup time
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-25 12:08:20 +09:00
Rishabh Dave
d8749faf21
Merge pull request #50142 from rishabh-d-dave/fs-xfstests_dev-distro
qa/cephfs: add 'rhel' to family of RH OS in xfstest_dev.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-02-24 22:51:20 +05:30
zdover23
f9d812a562
Merge pull request #50250 from tnyeanderson/fix-service-spec-link
doc/cephadm/host-management: add service spec link

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-02-24 18:33:52 +10:00
Yingxin
fce4edc834
Merge pull request #50191 from xxhdx1985126/wip-weak-backref-retrieval-trans-issue-fix
crimson/os/seastore: make weak transactions conflict with others on ROOT

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-24 14:35:36 +08:00
thomas
bb5837f049 doc/cephadm/host-management: add service spec link
The old "(below)" text is not accurate, the service spec definition is
not in the same file at this point in time. This commit adds a ref link
to the actual service specification section.

Signed-off-by: thomas <me@thomasanderson.cloud>
2023-02-24 01:10:23 -05:00
Samuel Just
acedd169e4
Merge pull request #48516 from athanatos/sjust/wip-57801
crimson,mon: add guards to avoid accidental crimson deployment and to avoid usage of unsupported features with crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2023-02-23 18:39:27 -08:00
zdover23
5bdcee8bfe
Merge pull request #50246 from zdover23/wip-doc-2023-02-24-glossary-bucket-2
doc/glossary: add AWS links to bucket info

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-02-24 12:15:54 +10:00
Zac Dover
d16ed970e9 doc/glossary: add AWS/OpenStack bucket info
Add links to AWS's documentation of buckets, in accordance with Casey
Bodley's suggestions here:
https://github.com/ceph/ceph/pull/50221#discussion_r1115900879

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-24 11:57:58 +10:00
Ilya Dryomov
673373f829
Merge pull request #50202 from rzarzynski/wip-librbd-drop-cxx11-from-example
examples/librbd: drop -std=c++11 from CXXFLAGS

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-23 18:25:14 +01:00
Ali Maredia
56e2a1006f common: Add labeled perf counters
Add the ability to dump labeled perf counters
for a daemon. Labeled perf counters are stored
in a CephContext's PerfCountersCollection.

Labeled and unlabeled perf counters are dumped
to the admin socket via `counters dump` command.

The schema for labeled and unlabeled perf
counters are dumped to the admin socket via
`counters schema` command.

This commit includes docs and additional unit tests

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2023-02-23 12:05:36 -05:00
Casey Bodley
f427bb2d2c common: add abstraction for label-aware perf counter keys
a flat representation of a set of prometheus labels, returned as a
std::string. this string can either be used for sorting an ordered
container of perf counters, or for hashing an unordered container

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-23 12:04:19 -05:00
Rishabh Dave
fec9be184d qa/cephfs: add 'rhel' to family of RH OS in xfstest_dev.py
The string returned by get_system_type() for RHEL 8.4 recently wasn't
one of the strings that xfstests_dev.py expects. The string returned
this time was instead "rhel". Add this string to the list of strings
that xfstests_dev.py expects for Red Hat OS family so that this failure
won't occur again in future. Also log the detected distro and version so
that it is easier to debug when this issue occurs next time.

Fixes: https://tracker.ceph.com/issues/58726
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-23 22:31:27 +05:30
Ernesto Puerta
1c7fb66856
mgr/dashboard: fix constraints.txt again
The previous attempt to fix this issue
(https://github.com/ceph/ceph/pull/50207) was not successful, since the
PIP syntax for compatible versions (~=) behaves differently compared to
other package management tools (e.g.: npm).

In this case, package~=x.y.z equasl to package>=x.y.z;package==x.y.*,
instead of the desired package>=x.y.z;package==x.*.

Fixes: tracker.ceph.com/issues/58827

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2023-02-23 17:22:10 +01:00
Soumya Koduri
9f0440e6d5
Merge pull request #50098 from soumyakoduri/wip-skoduri-cloud-trans-azure
rgw/cloud: Add custom headers for objects transitioned to cloud

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-02-23 17:49:31 +05:30
Ilya Dryomov
b21a379c5b librbd: call apply_changes() after setting librados_thread_count
Otherwise the setting doesn't take effect.  While at it, replace
home-grown stringify() with standard to_string().

Fixes: https://tracker.ceph.com/issues/58833
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-23 12:50:45 +01:00
Rishabh Dave
4017bfa2c5
Merge pull request #46574 from rishabh-d-dave/xfstests-dev-improvements
qa/cephfs: improvements for xfstests_dev.py

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-23 17:08:53 +05:30
Radoslaw Zarzynski
6668814a17
Merge pull request #49528 from NitzanMordhai/wip-nitzan-filestore-removal
Reef: filestore removal

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-23 12:20:49 +01:00
Yingxin
7818c6e003
Merge pull request #50176 from xxhdx1985126/wip-seastore-throttle
crimson/os/seastore: throttle concurrent transactions in seastore

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-23 16:49:03 +08:00
Anthony D'Atri
d67427a425
Merge pull request #50226 from anthonyeleven/anthonyeleven/tweak-cephadm-troubleshooting
doc: preen cephadm/troubleshooting.rst and radosgw/placement.rst
2023-02-23 02:39:10 -05:00
Anthony D'Atri
d5bf55bcd6 doc: preen cephadm/troubleshooting.rst and radosgw/placement.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2023-02-23 02:12:34 -05:00
Anthony D'Atri
c04a41f82d
Merge pull request #50221 from zdover23/wip-doc-2023-02-23-glossary-bucket
doc/glossary: add "Bucket"
2023-02-23 01:06:19 -05:00
Anthony D'Atri
3cf687d616
Merge pull request #50220 from tnyeanderson/fix-word-repeat
doc: remove word repeat in docs
2023-02-23 00:59:16 -05:00
Zac Dover
ffbc5ea2b9 doc/glossary: add "Bucket"
Add "Bucket" to the glossary. The definition added here is just a first
attempt, but a man's reach must exceed his grasp, or what's a heaven
for?

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-23 15:53:39 +10:00
thomas
4e9eaf8b5d doc/cephadm/troubleshooting: remove word repeat
The original text was "In case the cluster the last MGR of a cluster"

Signed-off-by: thomas <me@thomasanderson.cloud>
2023-02-23 00:47:58 -05:00
Venky Shankar
da7928adab Merge PR #50196 into main
* refs/pull/50196/head:
	qa: export DIFF_LENGTH env parameter to print the full git diff

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-23 11:16:21 +05:30
Anthony D'Atri
5129a6d1b8
Merge pull request #50179 from rishabh-d-dave/doc-fs-quota
doc/cephfs: improve CephFS Quota document
2023-02-23 00:36:36 -05:00
zdover23
c0df8ce5bf
Merge pull request #50213 from pponnuvel/osd_backfill_doc
doc/rados: update OSD_BACKFILLFULL description

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-02-23 15:16:33 +10:00
Xuehan Xu
7d60549032 crimson/os/seastore/async_cleaner: make backref retrieval trans non-weak
backref retrieval transactions may conflict with others on RootBlock

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2023-02-23 02:03:06 +00:00
Xuehan Xu
7ad7b31fef crimson/os/seastore/cache: return the RootBlock it's waiting on in
`Cache::get_root()`

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2023-02-23 01:30:59 +00:00
Laura Flores
96b45cb61a
Merge pull request #50215 from ljflores/wip-temporarily-disable-cronjobs 2023-02-22 19:29:20 -06:00
Laura Flores
39df3b8827 qa/crontab: temporarily disable teuthology cronjobs
Temporarily disable teuthology cron jobs before
the Reef dev freeze. This commit is intended to
lessen the load on the teuthology queue as we
test final patches for Reef, but should be reverted
after the dev freeze.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-02-22 23:17:19 +00:00
Nizamudeen A
f3bc98d335
Merge pull request #50201 from ceph/fix-github_action-main
.github/workflows: update failing action

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: rkachach <NOT@FOUND>
2023-02-22 22:49:44 +05:30
Ilya Dryomov
c192eee0d8
Merge pull request #48127 from petrutlucian94/cephfs_tests
Windows tests follow-up (librados_asio error category, disable some tests)

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-22 17:43:17 +01:00
Laura Flores
c61fe847c6 test/osd: add read balancer unit tests
This commit adds unit test coverage to the read balancing
feature, including small vs. large osdmap scenarios,
random osdmap scenarios, and scenarios that involve
tweaking primary affinity on an OSD.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-02-22 16:29:38 +00:00
Laura Flores
10b89f96d9 tools, test/cli: add read balancer to osdmaptool
This commit adds the capability to balance reads on a
given osdmap with the osdmaptool. The user has the option
of performing a "dry run" of read balancing OR taking it a
step further and applying the results to a live cluster.

Performing a "dry run" would involve simply running an
osdmaptool command and inspecting the results.

The template for the command is:
`osdmaptool <osd map file> --read <file for command output> --read-pool <pool name>`

An example command a user might run is:
`osdmaptool om --read out.txt --read-pool default.rgw.control`

This commit also adds a `--vstart` flag that allows a user to print ceph
commands in the outfile formatted for a vstart cluster. An example command
a user might run is:
`./bin/osdmaptool om --vstart --read out.txt --read-pool default.rgw.control`
The out.txt file would contain ceph commands prefixed with `./bin/`.
The `--vstart` flag may also be applied to an `--upmap` osdmaptool command.

If the user wants to apply read balancing results from their dry run to a
live cluster, they may either manually apply the ceph commands from the out
file, or run `source <outfile>`.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-02-22 16:29:38 +00:00