Commit Graph

125544 Commits

Author SHA1 Message Date
Dimitri Savineau
a6808efca4 mgr/cephadm/iscsi: simplify the dashboard check
We don't need to run an extra command (mgr module ls) to obtain the mgr
modules list since we already have this information in the mgr_map.
This workflow is already done for the monitoring stack or for configuring
the iscsi integration within the dashboard (during creation) via the
config_dashboard method.

The mgr_map is mocked in the tests with the dashboard module enabled so we
don't need _mon_command_mock_mgr_module_ls anymore.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:21:02 -04:00
Dimitri Savineau
16bb5b8076 mgr/cephadm/grafana: check if dashboard is enabled
When deploying the grafana service but the mgr dashboard isn't enabled then
dashboard set-grafana-api-ssl-verify command fails.

Closes: https://tracker.ceph.com/issues/51796

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:03:42 -04:00
Kefu Chai
9054b1a7f7
Merge pull request #42157 from markhpc/wip-osd_client_message_cap
common/options: Set osd_client_message_cap to 256.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-23 20:58:53 +08:00
Varsha Rao
da5aa88844 src/pybind/mgr/test_orchestrator: remove nfs pool and namespace
'orch apply nfs' no longer requires pool and namespace.

Fixes: https://tracker.ceph.com/issues/51794
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-07-23 15:15:26 +05:30
Varsha Rao
ff33b4f23b src/vstart: update nfs-ganesha pool name
Recently, the default pool name for nfs was changed from 'nfs-ganesha' to
'.nfs'. Change the pool name in vstart too.

Fixes: https://tracker.ceph.com/issues/51795
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-07-23 15:13:15 +05:30
Arthur Outhenin-Chalandre
e2b2faef11
osd: log snaptrim message to dout
This log message is not an error and is done on every tick of the
snaptrim process. Replace the derr logging to dout(10) to not log it
by default.

Fixes: https://tracker.ceph.com/issues/51799
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2021-07-23 09:32:21 +02:00
Kefu Chai
ade690b549
Merge pull request #42449 from kevinzs2048/wip-test-sanitize
qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password…

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-23 14:18:11 +08:00
Kefu Chai
042aea0ad2
Merge pull request #41794 from ceph/dependabot/pip/src/ceph-volume/plugin/zfs/pip-19.2
ceph-volume: bump pip in /src/ceph-volume/plugin/zfs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-23 10:55:29 +08:00
Kefu Chai
8706aa15d0 ceph-volume: bump pip in /src/ceph-volume/plugin/zfs
Bumps [pip](https://github.com/pypa/pip) from 9.0.1 to 19.2.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst)
- [Commits](pypa/pip@9.0.1...19.2)

---
updated-dependencies:
- dependency-name: pip
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-23 09:57:35 +08:00
Kefu Chai
af79875460
Merge pull request #42255 from tchaikov/wip-cmake-mgr-cleanup
cmake: install mgr module separately

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-23 09:51:52 +08:00
Kefu Chai
b82b5f963d
Merge pull request #42437 from rzarzynski/wip-crimson-generic-alienstore
crimson/os: generalize AlienStore beyond BlueStore.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2021-07-23 09:47:53 +08:00
Nathan Cutler
1cb84a1160 rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.

Thanks to Franck Bui for providing the original patch
926433f5d4 that this patch is modeled after.

NOTE: the use of FIRST_ARG had already been eliminated by
926433f5d4 but was re-introduced later by
9466d70985

Fixes: 9466d70985
Fixes: https://tracker.ceph.com/issues/51797
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-07-22 22:52:39 +02:00
Casey Bodley
dc6e6aee83
Merge pull request #42455 from dang/wip-dang-zipper-get_obj_state
RGW - Zipper - Make default args match in get_obj_state

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-22 13:23:56 -04:00
Igor Fedotov
0a5ad09370 os/bluestore: implement allocation perf histogram.
This allows to monitor how fragmented resulting allocations are
depending on the requested block size.

Could be coupled with https://github.com/ceph/ceph/pull/41600/ for easy
live monitoring for either an individual OSD or full set of them.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2021-07-22 19:45:14 +03:00
Deepika Upadhyay
c33df0136c cmake: install libopentracing.so.1(dependency of libjaeger)
fixes:
dpkg-shlibdeps: error: cannot find library libopentracing.so.1 needed by debian/libjaeger/usr/lib/libjaegertracing.so.0.6.1 (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '/build/ceph-17.0.0-6279-g97a36368/obj-x86_64-linux-gnu/external')

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:38:06 +05:30
Deepika Upadhyay
c140147af9 cmake: always require thrift v0.13.0 if WITH_JAEGER
* without this the build shall continue and eventually fail
as thrift is a required dependency for jaeger

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:37:33 +05:30
Sebastian Wagner
8c5edecc13
.githubmap: Update myself
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-22 18:07:10 +02:00
Deepika Upadhyay
4df43cff61 cmake: reorder find_package_handle_standard_arg for thrift
* Findthrift: thrift_FOUND fails to set if called before
  find_package_handle_standard_arg, hence reorder adding
  thrift::libthrift target after it

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:35:29 +05:30
Adam C. Emerson
9c00a343e0
Merge pull request #42247 from pritha-srivastava/wip-rgw-sts-session-policy-fixes
rgw/sts: createbucket op should take session_policies into account

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-07-22 11:59:29 -04:00
Deepika Upadhyay
fcb1eb91c4 cmake: remove buildthrift methods
we use Findthrift.cmake method for adding thrift dependencies, the
cleanup 80e82686eb missed removing these
methods intended to buildthrift from source.

this address the missing failure due to ,
```
CMake Error at cmake/modules/BuildJaeger.cmake:61 (include):
  include could not find load file:

    Buildthrift
```
this fixes regression introduced by: 80e82686eb

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:25:56 +05:30
Casey Bodley
fb60094c97
Merge pull request #42249 from kalebskeithley/rgw-51599
rgw: cleanup virtual dtor decls and defns

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-07-22 11:52:02 -04:00
Daniel Gryniewicz
664fe75c2a RGW - Zipper - Make default args match in get_obj_state
The default arg for follow_olh was different between the base class and
the Rados class.  Fix it so they're the same.

Fixes: https://tracker.ceph.com/issues/51677
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2021-07-22 11:48:08 -04:00
Neha Ojha
54b41f71d9
Merge pull request #42365 from PracticalMetal/typo
doc: Fix typos and grammar in ecbackend.rst

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-22 08:46:33 -07:00
Patrick Donnelly
4ee63174e6
Merge PR #42431 into master
* refs/pull/42431/head:
	cmake: add "mypy" back to tox envlist of "qa""
	qa/tasks/vstart_runner: add optional "sudo" param to _run_python()

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-22 07:56:40 -07:00
Patrick Donnelly
e30663a18c
Merge PR #42444 into master
* refs/pull/42444/head:
	doc/cephfs/journaler: document options using confval directive

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-22 07:54:52 -07:00
Casey Bodley
c011af6903
Merge pull request #42380 from adamemerson/wip-51712
rgw: radosgw-admin errors if marker not specified on data/mdlog trim

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-22 10:19:17 -04:00
Dimitri Savineau
842fc2b605 ceph-volume/tests: use ansible_facts
Since inject_facts_as_vars is set to false in the ansible.cfg file then we
have to update the references to use ansible_facts[<thing>] instead of
ansible_<thing>.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-22 10:00:29 -04:00
Dimitri Savineau
95056a24e4 ceph-volume/tests: use pytest rerunfailures
We already install the dependency from ceph-ansible requirements.txt and to
avoid false positive (like after rebooting a node) we can retry failing test.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-22 10:00:29 -04:00
Dimitri Savineau
b758fdd829 ceph-volume/tests: set ANSIBLE_CONFIG env var
Without loading the ansible.cfg file from ceph-ansible project, we don't
have the pipelining enabled which can result in significant performance
improvement.
This removes the ANSIBLE_ACTION_PLUGINS, ANSIBLE_RETRY_FILES_ENABLED and
ANSIBLE_SSH_RETRIES environment variables as it is already included in the
ansible.cfg file.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-22 10:00:29 -04:00
Dimitri Savineau
0ad615bab5 ceph-volume/tests: update ansible ssh_args env var
The ansible ssh_args parameter is usually defined in the ansible.cfg file.
Currently this variable is overrided in tox to manage the vagrant ssh file
but we lost all default values.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-22 10:00:29 -04:00
Milind Changire
6edebf8a1f
Merge pull request #42329 from vshankar/wip-cephfs-mirror-dir-remove-registery
cephfs-mirror: record directory path cancel in DirRegistry

Reviewed-by: Milind Changire <mchangir@redhat.com>
2021-07-22 18:20:59 +05:30
Kefu Chai
3117a6c3dd
Merge pull request #42450 from tchaikov/wip-debian-jaeger
debian/control: depend on libjaeger only if <pkg.ceph.jaeger>

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 20:25:46 +08:00
Kevin Zhao
d04ef800ab qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password.txt file
To allow running multiple instances of the same tests.

Fixes: https://tracker.ceph.com/issues/51792

Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
2021-07-22 13:20:29 +01:00
Sebastian Wagner
1f8560816e
Merge pull request #42433 from sebastian-philipp/fix-gen-keepalive-config
mgr/cephadm: Fix haproxy not being recognized as a proper daemon

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Asbjørn Sannes <asbjorn.sannes@interhost.no>
2021-07-22 12:04:54 +02:00
Kefu Chai
8e1edd746f debian/control: depend on libjaeger only if <pkg.ceph.jaeger>
otherwise we'd have following failure when trying to install ceph
packages if they are not built with pkg.ceph.jaeger profile:

ceph-common : Depends: libjaeger (= 17.0.0-6321-g62349ba4-1focal) but it is not installable

this change should address the regression introduced by
4b69d3a184

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 17:59:51 +08:00
Jonas Zeiger
d83d4f3586 ceph-volume: lvm batch: fast_allocations(): avoid ZeroDivisionError
* Fixes Bug #51526 ( https://tracker.ceph.com/issues/51526 )

Signed-off-by: Jonas Zeiger <jonas.zeiger@talpidae.net>
2021-07-22 11:19:37 +02:00
Sebastian Wagner
a8f1cf2edb
mgr/cephadm: Fix haproxy not being recognized as a proper daemon
Turns out daemon types != service types:

    cephadm [WRN] Found unknown service type haproxy on host smithi019
    cephadm [WRN] Found unknown service type keepalived on host smithi019

leading to `self.mgr.cache.get_daemons_by_service(spec.service_name())`
not returning any daemons.

Fixes: https://tracker.ceph.com/issues/51311

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-22 10:55:54 +02:00
Kefu Chai
71e20279e9 rgw/rgw_lua: s/boost::filesystem/std::filesystem/
for less dependencies on 3rd party libraries, and to reduce the
compilation time.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:49:30 +08:00
Kefu Chai
e3245d12a0 cmake: s/NPM/NPM_EXECUTABLE/
so the naming is more consistent with other executables found by
find_program().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:35:57 +08:00
Kefu Chai
6d17e684ec cmake: use ${NPM} instead of npm
so it is more explicit that ${NPM} is used when NODEENV is not
specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:35:47 +08:00
Kefu Chai
e4f1de64d4 cmake: simplify the regex for matching npm command
npx command was added in 3fe39d4899,
but we don't use npx anymore, so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:35:40 +08:00
Kefu Chai
993e250682 cmake: push dashboard build logic down
it's simpler to build dashboard in dashboard/CMakeLists.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:35:30 +08:00
Kefu Chai
7633a97a15 cmake: push frontend build logic down
it's simpler to build frontend in frontend/CMakeLists.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:35:12 +08:00
Kefu Chai
73e983cfe8 pybind/mgr/dashboard: build frontend/dist in build
to enable us to build out of source, we should build the frontend
artifacts out of src. in this change:

* cmake:
  - install frontend/dist from build
  - drop rules to exclude unused frontend artifacts.
  - pass `--output-path` option to ng
  - copy `frontend/package.json` to build so that the frontend
    can find it
* pybind/mgr/dashboard/module.py: fall back to build directory
  if frontend/dist is not found.
* pybind/mgr/dashboard/__init__.py: use frontend/dist when performing
  unit test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 16:33:33 +08:00
Pulkit Mittal
8f84f121e1 doc: fix typo and grammar
Signed-off-by: Pulkit Mittal <2pulkit2@gmail.com>
2021-07-22 13:48:07 +05:30
Kefu Chai
7048172cad
Merge pull request #38783 from ideepika/wip-osd-tracing
cmake/ninja: support ninja for jaegertracing

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-22 15:33:13 +08:00
Yingxin Cheng
97bcfda549 crimson/os/seastore/segment_manager: respect len parameter of BlockSegmentManager::read()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-22 15:28:40 +08:00
Kefu Chai
a4d654b9c0 cmake: use PATTERN instead of REGEX to exclude installed files
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 13:45:04 +08:00
Kefu Chai
dfd3deb5b2 cmake: install mgr module separately
instead of installing the whole directory, install mgr modules
individually, so it's more clear that what EXCLUDE pattern is
used in which mgr module.

this improves the maintability and readability. in future, we
should only install the artifacts. with this change, we can
specify the include pattern only for dashboard mgr module so
that, for instance, only .js, .html and .css files are installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-22 13:44:34 +08:00
Deepika Upadhyay
5f5de4ccb5 cmake: fix jaeger linking to ceph libraries
updates:
* custom target jaeger_base
* target jaeger-base which encapsulates all jaeger libs
* include external libraries path needed linking libraries and including
  opentracing and jaegertracing headers files

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 11:09:09 +05:30