Commit Graph

125508 Commits

Author SHA1 Message Date
Deepika Upadhyay
e80a80c7d9 test: update test for when cache tier is not available
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-27 07:06:29 +05:30
Deepika Upadhyay
fafebb1824 mon/PGMap: DIRTY field as N/A in df detail when cache tier not in use
'ceph df detail' reports a column for DIRTY objects under POOLS even
    though cache tiers not being used.  In replicated or EC pool all objects
    in the pool are reported as logically DIRTY as they have never been
    flushed .
    we display N/A for DIRTY objects if the pool is not a cache tier.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-27 07:06:20 +05:30
Sage Weil
f8f7b86571 Merge PR #42292 into master
* refs/pull/42292/head:
	qa/suites/upgrade: log_to_journald=false

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-26 19:23:26 -04:00
Sage Weil
ac63ab6125 Merge PR #42489 into master
* refs/pull/42489/head:
	qa/suites/upgrade/pacific-x/stress-split: do not avoid_pacific_features

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 19:23:03 -04:00
Sage Weil
8d9db910f7 doc/mgr/nfs: document ingress in more detail
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
0448a8d39d doc/mgr/nfs: typo
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
684c6a2dba doc/mgr/nfs: add note about incomplete ingress
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
cd089ee74e qa/suites/orch/cephadm: add rgw nfs export test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
bae406f746 mgr/cephadm: ingress: tolerate no daemons
This doesn't normally happen, but did before the daemon inventory breakage
(see previous patches) was fixed.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
ad5fe4452d mgr/nfs: add --squash option to 'nfs export create rgw ...'
The cephfs variant already has it.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
7d2f7efe3f mgr/nfs: use bucket owner creds for rgw bucket export
The bucket owner can always read/write to the bucket, so use those creds
for the export.  This is less complicated than setting up a dedicated
user anyway.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
deaff0c42c mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image
We were using CEPH_TYPES + GATEWAY_TYPES, but that isn't really accurate.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
45737fe95a qa/tasks/python: simple task to run python code
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
8b96d9c22b doc/mgr/nfs: revisions
- clean up language
- move config hierarchy to the bottom (this is an implementation detail
that is only useful if managing ganesha externally)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
b73f730749 mgr/nfs/export: nicer exceptions on cap update
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:21:24 -04:00
Guillaume Abrioux
4bfa87ee55
Merge pull request #42181 from lyind/51526-fix-ceph-volume-zero-division-error
ceph-volume: lvm batch: fast_allocations(): avoid ZeroDivisionError
2021-07-26 21:58:07 +02:00
Zac Dover
d27bb21cc1 doc/rados: rewrite index.rst
This PR makes minor alterations to the
text at the beginning of the RADOS Guide.

Most notably, the monitor daemon has been
added to the list of types of daemons that
constitute a Ceph cluster.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-07-27 05:16:41 +10:00
Michael Fritch
d9198d8668
cephadm: ceph-volume should raise fsid mismatch
raise an fsid mismatch error when passed differing fsids via `--fsid` and `--config`:

```
self = <tests.test_cephadm.TestCephVolume object at 0x7f1c711961f0>, cephadm_fs = <pyfakefs.fake_filesystem.FakeFilesystem object at 0x7f1c713addc0>

    def test_fsid(self, cephadm_fs):
        cv_cmd = ['--', 'inventory', '--format', 'json']
        fsid = '00000000-0000-0000-0000-0000deadbeef'

        cmd = ['ceph-volume', '--fsid', fsid] + cv_cmd
        with with_cephadm_ctx(cmd) as ctx:
            cd.command_ceph_volume(ctx)
            assert ctx.fsid == fsid

        s = get_ceph_conf(fsid=fsid)
        f = cephadm_fs.create_file('ceph.conf', contents=s)

        cmd = ['ceph-volume', '--fsid', fsid, '--config', f.path] + cv_cmd
        with with_cephadm_ctx(cmd) as ctx:
            cd.command_ceph_volume(ctx)
            assert ctx.fsid == fsid

        cmd = ['ceph-volume', '--fsid', '10000000-0000-0000-0000-0000deadbeef', '--config', f.path] + cv_cmd
        with with_cephadm_ctx(cmd) as ctx:
            err = 'fsid does not match ceph.conf'
            with pytest.raises(cd.Error, match=err):
                cd.command_ceph_volume(ctx)
>               assert ctx.fsid == None
E               AssertionError: assert '10000000-0000-0000-0000-0000deadbeef' == None
E                +  where '10000000-0000-0000-0000-0000deadbeef' = <cephadm.CephadmContext object at 0x7f1c7121c1c0>.fsid
```

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-07-26 10:29:39 -06:00
Michael Fritch
2d2bb9f96d
cephadm: add ceph-volume tests
add basic ceph-volume tests for `--fsid`, `--config`, and `--keyring`

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-07-26 10:29:39 -06:00
Michael Fritch
b589ae68fa
cephadm: remove get_parm mock
fixture does not need to patch the `get_parm` func

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-07-26 10:29:39 -06:00
Kalpesh Pandya
3be2b4bcc5 rgw/notifications: Changing the Multipart Upload notification behavior
Changing the notification behavior in case of Multipart Upload, updating
the related test cases and adding the documentation changes for the same.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-07-26 21:22:33 +05:30
Guillaume Abrioux
d69ba6ce58
Merge pull request #42443 from dsavineau/ceph_volume_tests_functional_ssh_args
ceph-volume/tests: update ansible environment variables in tox
2021-07-26 17:08:10 +02:00
Patrick Donnelly
83d252cc30 qa: fold frag confs into conf/mds.yaml
These overrides are standard for all configurations. The config to
enable fragmentation is also long removed.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-26 07:14:38 -07:00
Kefu Chai
54c3e1b688 *: drop strict_iecstrtoll(const char *str,..)
replace strict_iecstrtoll(const char *str,..) with
strict_iecstrtoll(std::string_view, ..). which is more convenient.
and both of them share the same implementation:

strict_iec_cast<uint64_t>(str, err);

so they are interchangeable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
5d22af9316 common,mon: drop strict_sistrtoll()
strict_sistrtoll() is but an alias of strict_si_cast<uint64_t>(..),
let's just drop the former. there are way too many thin wrappers in
strtol.{h,cc}. they don't offer lots of benefit to us.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
650a0c9363 common/strtol: drop strict_strto*(const char*,..)
they are but proxies to strict_strto*(string_view, ...). and their
callers can just call the latter without any change. so drop these
proxies.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
55ed86bb98 common, mon: use strict_si_cast<>(std::string_view,..)
simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
b6f29f4c1b common/strtol: expose strict_si_cast(std::string_view, ..)
this variant is better than strict_si_cast(const char*), because:

* we can just pass std::string to it, as std::string_view can be
  constructed from a std::string implicitly
* strict_si_cast(std::string_view, ..) is the underlying
  implementation of strict_si_cast(const char*,..), so less
  indirection helps with the readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
58c4ece8f3 rgw,mon,common/strtol: use strict_iec_cast(std::string_view, ..)
this variant is better than strict_iec_cast(const char*), because:

* we can just pass std::string to it, as std::string_view can be
  constructed from a std::string implicitly
* strict_iec_cast(std::string_view, ..) is the underlying
  implementation of strict_iec_cast(const char*,..), so less
  indirection helps with the readability.
* rgw,mon: use strict_iec_cast(std::string_view, ..) instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
f608782797
Merge pull request #42403 from BlaineEXE/ceph-volume-use-safer-check-for-has-bluestore-label
ceph-volume: use safer check for bluestore label

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-26 18:31:13 +08:00
Kefu Chai
0dfab38c01
Merge pull request #42474 from tchaikov/wip-kv-cleanup
kv/RocksDBStore: always initialize local variable

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-26 18:29:01 +08:00
Igor Fedotov
8c49d56b93
Merge pull request #42456 from ifed01/wip-ifed-alloc-histogram2
os/bluestore: implement allocation perf histogram.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 13:21:28 +03:00
Akanksha Chaudhari
2f785b1004 mgr/volumes/fs/operations/group.py: add extra blank line
Fixes:https://tracker.ceph.com/issues/51393
Signed-off-by: Akanksha Chaudhari <astroakanksha24@gmail.com>
2021-07-26 12:23:17 +05:30
Mingxin Liu
c5b38306d7 osd/PGLog: set acceptable rollback_info_trimmed_to for pg of replicated pool
if this PG belongs to a replica pool, since replica pool does not
require rollback, see `pg_pool_t::require_rollback()`, we don't
update the omap entry for noting down the
`on_disk_rollback_info_trimmed_to`. see
`PGLog::_write_log_and_missing()` and `PGLog::_write_log_and_missing()`.
in this case, `read_log_and_missing()` should not be able to find
the value for "rollback_info_trimmed_to" in the omap entries.

because `roll_foward_to()` replays all entries starting at
`rollback_info_trimmed_to`, if it is empty, when the primary osd
is merging the pglog from its peers while peering, it would have
to walk through *all* pglog entries. this would incur unnecessary
overhead, and slows down the peering.

in this change, if this PG belongs to a replica pool, we just fast
forward it to `info.last_update` to avoid this overhead.

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
2021-07-26 11:58:11 +08:00
Kefu Chai
4fc80fe094 tools/ceph_objectstore_tool: pass require_rollback to write_log_and_missing()
before this change, `require_rollback` is always true when it is passed
to write_log_and_missing(), which does not hurt. but this diverges from
how `PG::prepare_write()` is implemented. and probably, more important,
this is not correct, as replica pool does not `require_rollback`.

after this change, `require_rollback` is deduced from pgid. so we don't
always pass `true` to `write_log_and_missing()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 11:58:11 +08:00
Kefu Chai
da35cbe6bc
Merge pull request #42466 from adk3798/mgr-stop
mgr/cephadm: Don't allow stopping full mgr, mon or osd services

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-07-25 21:42:29 +08:00
Kefu Chai
c2ec6489fa vstart.sh: set objectstore to "cyanstore" if --cyanstore is specified
in e6ed65db8b, "cyanstore" is added to the
help message of vstart.sh, but we should also check for this option, and
set the ceph option accordingly.

in this change, the option is checked and "objectstore" is updated
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-25 14:28:02 +08:00
Kefu Chai
c4e1b4d7ef crimson/osd: print logging using logger() in main()
more consistent and easier to parse this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-25 11:07:52 +08:00
Kefu Chai
2e3f4689a1 kv/RocksDBStore: always initialize local variable
silences following warning from GCC:

 ../src/kv/RocksDBStore.cc:386:7: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   386 |       if (r < 0) {
       |       ^~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:31:15 +08:00
Kefu Chai
d6c931dc78 crimson/osd: cancel timer when stopping
so that reactor does not complain when being stopped, like:

ERROR 2021-07-24 20:50:23,711 [shard 0] seastar - Timer callback failed: seastar::gate_closed_exception (gate closed)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
9816cb9940 crimson/osd: disable var-tracking-assignments
to disable following warning:

In file included from ../src/common/config.h:27,
                 from ../src/crimson/common/config_proxy.h:8,
                 from ../src/crimson/osd/main.cc:23:
../src/common/config_values.h: In copy constructor ‘ConfigValues::ConfigValues(const ConfigValues&)’:
../src/common/config_values.h:19:7: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
   19 | class ConfigValues {
      |       ^~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
1c45d3ae02 crimson: use seastar::try_with_gate to guard background task
for less indent, and for more consistency with how tools/store_nbd
runs background task.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
d686621fe7
Merge pull request #42467 from dsavineau/issue_51796
mgr/cephadm/grafana: check if dashboard is enabled

Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:51:25 +08:00
Kefu Chai
7520a391fd
Merge pull request #42468 from dsavineau/refact_mgr_cephadm_iscsi_dashboard_check
mgr/cephadm/iscsi: simplify the dashboard check

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:41:19 +08:00
Kefu Chai
6144acfb32
Merge pull request #41168 from aclamk/wip-bluefs-more-sync-metadata
os/bluestore/bluefs: Force metadata sync after file rename and file deletion

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-07-24 19:39:51 +08:00
Kefu Chai
1af0c6195c cmake: define OpenLDAP::OpenLDAP library
* define OpenLDAP::OpenLDAP, so this library can be consumed in a simpler way.
* use OpenLDAP::OpenLDAP instead of OpenLDAP_LIBRARIES when appropriate
* do not link against unused ${OpenLDAP_LIBRARIES}

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:56 +08:00
Kefu Chai
1c3c42fe52 cmake: s/OPENLDAP/OpenLDAP/
the upstream project of OpenLDAP is named "OpenLDAP", so rename OPENLDAP to
OpenLDAP for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:56 +08:00
Kefu Chai
99ba98cef0 cmake: use generator exp. for passing fmt::fmt's COMPILE_DEFINITIONS
more concise this way, without yet another redirection.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:56 +08:00
Kefu Chai
43d10b9e44 cmake: make rgw_common a static library
it was changed from a shared library to object library back in
3c0bba40b2, because we wanted to link both
rgw_a and rgw_admin_user against it.

but now that rgw_admin_user was dropped in
c89b59428e, let's compile rgw_common
as a static library. it'd be much easier for us to apply various compile
options and linkages to it this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:54 +08:00
Kefu Chai
2d955ebb5f cmake: build rgw_kmip_client_impl as part of radosgw
instead compiling rgw_kmip as an object library, just include
rgw_kmip_client_impl.cc in radosgw. simpler this way, as
rgw_kmip_client_impl.cc includes rgw/rgw_common.h. while rgw_common.cc
is indirectly included by ${rgw_libs} as a part of rgw_a, so
it would be simpler if we just compile rgw_kmip_client_impl.cc
as a part of radosgw, and let radosgw link against ${rgw_libs},
which are able to take care of the include directories of 3rd party
libraries, like spawn/spawn.hpp, which is pulled in by rgw/rgw_common.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:06 +08:00