Commit Graph

115192 Commits

Author SHA1 Message Date
Ramana Raja
a177b470aa vstart_runner: add write_file and sudo_write_file
... methods in LocalRemote class. These methods are called in some of
the recently added cephfs tests. They were implemented in teuthology's
Remote class, but not in vstart_runner's LocalRemote class. Hence some
cephfs tests couldn't be run locally using vstart_runner without this
change.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-09-25 14:34:35 +05:30
Kefu Chai
36d67cf59e
Merge pull request #37187 from kamoltat/patch-1
doc/dev/macos: remove nonexistence flag `--with-toolchain`

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-17 12:04:08 +08:00
Patrick Donnelly
23f9da70a7
Merge PR #37199 into master
* refs/pull/37199/head:
	qa/cephfs: add a note to test_nfs.py

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-16 16:44:38 -07:00
Patrick Donnelly
08c9cb85f9
Merge PR #37159 into master
* refs/pull/37159/head:
	mon/MDSMonitor: return zero when mds is absent for "mds fail"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-09-16 12:20:13 -07:00
Rishabh Dave
aa08a38b61 qa/cephfs: add a note to test_nfs.py
A note that tells that test_nfs.py is not yet compatible with
vstart_runner.py.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-16 22:10:42 +05:30
Neha Ojha
ca43c6e8ee
Merge pull request #37167 from neha-ojha/wip-47239
qa/tasks/ceph_manager.py: remove redundant check in raw_cluster_cmd_result

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-16 09:30:11 -07:00
Kamoltat Sirivadhna
825df66bb8 doc/dev/macos: remove nonexistence flag --with-toolchain
` --with-toolchain` is an old option from llvm 6

Signed-off-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2020-09-16 22:39:57 +07:00
Abhishek L
0f61e6ea28
Merge pull request #37109 from theanalyst/doc/releases/v15.2.5
doc: releases: notes for v15.2.5 octopus

Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>
2020-09-16 16:51:22 +02:00
Kefu Chai
cdca6c3992
Merge pull request #37166 from adamemerson/wip-undetailed-time
common/ceph_time: Don't define public things in time_detail

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-16 20:59:35 +08:00
Rishabh Dave
a246a56776 mon/MDSMonitor: return zero when mds is absent for "mds fail"
... instead of EINVAL.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-16 18:28:36 +05:30
Jan Fajerski
3ca25daa1d
Merge pull request #37093 from guits/guits-bz1877672
ceph-volume: fix simple activate when legacy osd

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2020-09-16 14:25:24 +02:00
Lenz Grimmer
b17ca08037
Merge pull request #35956 from ishanrai05/hover
mgr/dashboard: Display description on mouse hover

Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-16 13:49:54 +02:00
Matt Benjamin
6b31e3747b
Merge pull request #35924 from pritha-srivastava/wip-rgw-sts-error-messages
rgw/sts: adding error log messages to STS code.
2020-09-16 07:20:51 -04:00
Matt Benjamin
a9e40accb8
Merge pull request #36255 from pritha-srivastava/wip-rgw-sts-user-as-owner
rgw/sts: adding code for federated user as owner in case of STS.
2020-09-16 07:09:14 -04:00
Guillaume Abrioux
a3e9e215bd ceph-volume: fix simple activate when legacy osd
`ceph-volume simple activate --all` relies on the presence of json files
in `/etc/ceph/osd` that was created with `ceph-volume simple scan`
command.

In a cluster lifecycle, it is very likely an OSD which was deployed with
ceph-disk at some point gets removed or replaced. It means the corresponding
json file in `/etc/ceph/osd` becomes unrelevant. It makes `ceph-volume
simple activate --all` fails because it tries to mount non existing
partitions.
The idea here is to simply warn the user that the osd described in the
json file doesn't exist anymore and exit properly instead of throwing an
error.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1877672
Closes: https://tracker.ceph.com/issues/47493

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-16 11:13:11 +02:00
Joshua Schmid
25e5b8726d
Merge pull request #36898 from sebastian-philipp/doc-fix-orch-apply-mgr
doc/mgr: fix outdated `orch apply` docs.
2020-09-16 10:04:28 +02:00
Ali Maredia
6607cd8538
Merge pull request #35986 from TRYTOBE8TME/wip-rgw-sts-new
STS Testing Files
2020-09-16 01:38:50 -04:00
Kefu Chai
8016ba885c
Merge pull request #37172 from tchaikov/wip-crimson-object-context-lock
crimson/common/tri_mutex: update the class comment

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-16 12:42:39 +08:00
Kefu Chai
0a5fb96b10 crimson/osd/object_context: s/rwlock/lock/
so it's less confusing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-16 10:28:58 +08:00
Kefu Chai
441360dbb0 crimson/common/tri_mutex: update the class comment
to explain the reason why we have tri_mutex, and how it is related to
pipelined read / write. and the mutual exclusion between read, write
and rmw operations.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-16 09:10:14 +08:00
Kefu Chai
03d5b5d9dc
Merge pull request #37131 from tchaikov/wip-47311
crimson/object_context: make obc interruptible

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-16 09:01:32 +08:00
Adam C. Emerson
01f706ca0f common/ceph_time: Don't define public things in time_detail
Defining things in a detail section and then using them outside turned
out to not be the best idea.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-09-15 13:58:30 -04:00
Neha Ojha
e7eddec5a0 qa/tasks/ceph_manager.py: remove redundant check in raw_cluster_cmd_result
Fixes 530982129e. The check for cephadm is no
longer needed since it was moved to run_cluster_cmd.

Fixes: https://tracker.ceph.com/issues/47239
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-09-15 17:56:33 +00:00
Pritha Srivastava
57afbeed2f rgw/sts: adding documentation for a shadow user created for
a user that authenticates with oidc provider.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2020-09-15 21:39:00 +05:30
Pritha Srivastava
94f1b974c5 rgw/sts: adding code for federated user as owner in case of STS.
A new user under the namespace 'oidc' is created for every federated
user in case of AssumeRoleWithWebIdentity.

In case of AssumeRole, the user that needs cross account access becomes
the owner.

In both cases buckets are created in the tenant that the role belongs to.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2020-09-15 21:39:00 +05:30
Pritha Srivastava
d6af250bff rgw/sts: adding error log messages to STS code.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2020-09-15 21:37:01 +05:30
Adam C. Emerson
d94f2268a3 common/ceph_time: Don't indent under namespaces
This is a violation of our guidelines and since I'm making a larger
change anyway.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-09-15 11:16:53 -04:00
Kefu Chai
ce658a6aa7 crimson/osd: replace "ceph_abort_msg()" with assert()
these are programming errors, and are easy to detect. also assert() does
not return, so compiler won't complain at seeing a branch does not
return value in a function that returns value.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-15 16:48:27 +08:00
Xuehan Xu
aeb20dd2f5 crimson/object_context: make obc interruptible
* add tri_mutex::abort() to pass given exception to all waiters
* add ObjectContext::interrupt() to abort all pending consumers
  of current object context

Fixes: https://tracker.ceph.com/issues/47311
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-15 16:48:27 +08:00
Kefu Chai
ef72fc736d crimson: use tri_mutex for guarding ObjectContext
before this change, a seastar::shared_mutex, a RWState and a
shared_promise are used for tracking the consumers of ObjectContext.
and all of the consumers are put into writers if the predicate function
evaluates to "false", and is awaken if the predicate function evaluates
to "true" afterwards in a polling loop waiting on the shared_promise,
which is in turn fulfilled once the last consumer of the given category
relinquishes the lock.

this approach has couple issues:

* it is heavy weighted. seastar::shared_mutex already tracks each of
  the waiters' continuation using separate promise<>, and it does try
  to reschedule them once a given consumer releases the last lock.
  so it's like a design of a customized shared_mutex over a
  shared_mutex.
* it is complicated. 3 variables for tracking the different
  consumers of ObjectContext.

in this change,

* `tri_mutex` is introduced as a variant of the original
  `seastar::shared_mutex` to track two different shared users in
  addition to an exclusive user.
* replace `shared_mutex` with `tri_mutex` in `ObjectContext`, to
  simplify the design.
* move recovery_read_marker into `ObjectContext`. assuming all
  pending actions will be added as a waiter for the related
  object context before they acquire the lock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-15 16:48:27 +08:00
Kefu Chai
111a145371 crimson/osd: add ObjectContext::wait_recovery_read()
instead of reusing ObjectContext::get_recovery_read() for both
sync call and async call. just add a new method for the async call
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-15 16:45:48 +08:00
Mykola Golub
ebdb8e56e5
Merge pull request #37113 from dillaman/wip-47390
librbd: ensure local site is included in mirror image global status

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-09-15 10:29:47 +03:00
Kalpesh Pandya
60904c907e qa : STS Testing Files (AssumeRole and GetSessionToken)
Added required files for testing of AssumeRole and GetSessionToken API's and modified s3tests.py to handle the same.

(cherry picked from commit c2c90eaf52)

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2020-09-15 11:58:59 +05:30
Kefu Chai
6f1dc2994c
Merge pull request #37126 from tchaikov/wip-crimson-test
test/crimson: cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-15 09:21:57 +08:00
Casey Bodley
fde62037dc
Merge pull request #35167 from BryceCao/wip-inspect-zonegroup-when-create-existing-bucket
rgw : add check for zonegroup when create existing bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-09-14 17:11:08 -04:00
Casey Bodley
7b587a05e9
Merge pull request #36978 from cbodley/wip-47028
rgw: add check for ACL when create existing bucket

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2020-09-14 17:09:59 -04:00
Patrick Donnelly
9fcc49fae7
Merge PR #36125 into master
* refs/pull/36125/head:
	doc: cephfs/fs-nfs-exports: more cleanups
	doc: cephfs/fs-nfs-exports: command requires enabled orch module
	doc: cephfs/fs-nfs-exports.rst: write "RADOS" in ALL-CAPS
	doc: cephfs/fs-nfs-exports: use "NFS Ganesha" everywhere
	doc: cephfs: clarify that pseudo root path must be absolute
	doc: cephfs: fix misleading "ceph fs volume create" command
	doc: cephfs: fix misleading "ceph nfs create" command

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-09-14 13:49:50 -07:00
Jason Dillaman
9929eff533
Merge pull request #37086 from trociny/wip-qa-rbd-nbd-unmap
qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-14 14:32:30 -04:00
Jason Dillaman
c5325306ed
Merge pull request #37038 from trociny/wip-rbd-nbd-quiesce-error
rbd-nbd: return quiesce hook exit code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-14 14:31:58 -04:00
Mykola Golub
0d844728e9
Merge pull request #37107 from dillaman/wip-47405
qa/workunits/mon: fixed excessively large pool PG count

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-09-14 18:13:28 +03:00
Sebastian Wagner
bf9fef96c1
Merge pull request #37115 from sebastian-philipp/cephadm-refactor-cephamservice-create
mgr/cephadm: CephadmService.create -> CephadmService.prepare_create

Reviewed-by: Stephan Müller <smueller@suse.com>
2020-09-14 16:43:45 +02:00
Sebastian Wagner
209afa4869
Merge pull request #35945 from jschmid1/osdspec_limit_fixes
mgr/cephadm: allow to map existing daemons to osdspecs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-09-14 16:41:57 +02:00
Casey Bodley
b15dadac70
Merge pull request #36997 from linuxbox2/wip-rgwlc-mtime
rgwlc:  fix (post-parallel) non-current expiration

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2020-09-14 10:19:25 -04:00
Casey Bodley
461f4a5f78
Merge pull request #37125 from tchaikov/wip-rgw-const-char
rgw: convert string constant to string_view

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-09-14 10:15:31 -04:00
Jason Dillaman
2fa9442dcc qa/workunits/mon: fixed excessively large pool PG count
Fixes: https://tracker.ceph.com/issues/47405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-09-14 08:58:52 -04:00
Sebastian Wagner
245cf50132 mgr/cephadm: CephadmService.create -> CephadmService.prepare_create
Refactored this to simplify the control flow. It became obvious that

* `CephadmService.make_daemon_spec`
* `CephadmService.prepare_create`
* `CephadmService.generate_config`

are basically the same thing. They're all called directly
before the deployment of a daemon. All of them should be
unified. This PR makes this refactorization simpler

By renaming `create` to `prepare_create`, we make `create`
no longer being the entrypoint to call
`create_daemon`. Thus all the functions above
return some data structures.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-09-14 13:38:15 +02:00
Lenz Grimmer
4432903f00
Merge pull request #37044 from votdev/update_on_demand
mgr/dashboard: Update datatable only when necessary

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-14 11:50:19 +02:00
Lenz Grimmer
856d9de394
Merge pull request #36845 from bk201/wip-44803
mgr/dashboard: allow getting fresh inventory data from the orchestrator

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-14 11:48:22 +02:00
Lenz Grimmer
42a32baaaa
Merge pull request #36365 from bk201/wip-44637
mgr/dashboard: add e2e tests for Orchestrator components

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-14 10:56:41 +02:00
Kefu Chai
a2cc67f563
Merge pull request #37127 from tchaikov/wip-crimson-errorator
crimson/common: add specialization for futurize::invoke(Func, monostate)

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-14 13:19:47 +08:00