1
0
mirror of https://github.com/ceph/ceph synced 2025-03-11 02:39:05 +00:00
Commit Graph

100599 Commits

Author SHA1 Message Date
Patrick Donnelly
0f31e1e1fb
Merge PR into master
* refs/pull/28239/head:
	cephfs-shell: Add tox for testing with flake8

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-30 11:35:13 -07:00
Andrew Schoen
eb48959d2e
Merge pull request from alfredodeza/wip-rm40987
ceph-volume  simple: better detection when 'type' file is not present

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-07-30 13:33:40 -05:00
Varsha Rao
e90d58208d
cephfs-shell: Add tox for testing with flake8
Fixes: https://tracker.ceph.com/issues/39947
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-07-30 10:34:24 -07:00
Patrick Donnelly
6015ff4cf6
Merge PR into master
* refs/pull/29331/head:
	common: add comment about pod memory requests/limits

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2019-07-30 10:14:57 -07:00
Kefu Chai
32595268a3
Merge pull request from valentinbajrami/improve-grep-regex
src/script/check_commands.sh: fix grep regex class range

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-30 22:09:32 +08:00
Willem Jan Withagen
fdc0e2baeb cmake: update FindBoost.cmake
FreeBSD is moving to Boost 1.70
we should allow user to use boost 1.70.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-07-30 16:00:04 +02:00
Jan Fajerski
f42287c341 ceph-volume: refactor ceph-disk_member unittests
This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-07-30 15:27:20 +02:00
Mark Kogan
b0f6946a85 rgw: add option to specify shard-id for "bi list" admin command
related to: https://tracker.ceph.com/issues/40700
Signed-off-by: Mark Kogan <mkogan@redhat.com>
2019-07-30 08:55:07 -04:00
Valentin Bajrami
02ebe3c344
src/script/check_commands.sh: fix grep regex class range
Correct the faulty regex class.

Signed-off-by: Valentin Bajrami <valentin.bajrami@gmail.com>
2019-07-30 11:14:20 +02:00
Shilpa Jagannath
67e98eb413 On new uid check, return EEXIST.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:46 +05:30
Shilpa Jagannath
14f06ecfb8 Fixed line wrap
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:46 +05:30
Shilpa Jagannath
20c5e7417b Updated help.t with user rename command
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:46 +05:30
Shilpa Jagannath
80d5e962b7 Added user rename test
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Shilpa Jagannath
4eeb9d9526 Minor fixes.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Shilpa Jagannath
fa91577b43 Modified rgw_store_user_info() checks to suit user rename.
Added a helper function to modify bucket acl.
Rebased onto master.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Shilpa Jagannath
203d1f0ef3 Create new user before deleting existing user. Migrate buckets one at a time.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Shilpa Jagannath
be2b7e6b01 This commit deals with renaming user. Under the hood, the attributes of an
existing user are moved under the new user namespace specified by "--new-uid".
It calls bucket link and bucket chown to link the buckets and objects to the new
user namespace. Access and secret keys of the user(and the subusers) are preserved.

Usage: "radosgw-admin user rename --uid=<> --new-uid=<>"

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Shilpa Jagannath
6dc48620a1 Documentation for 'bucket chown'
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Shilpa Jagannath
d81d96a014 Set object ACLS to reflect bucket owner supporting bucket unlink/link.
Provides command line tool to update the acl on object
of a bucket after bucket unlink/link.
"radosgw-admin bucket chown --bucket <bucket>"

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
2f25d4dc74 rgw: bucket link: master rebase adaptions.
The previous few commits for rgw_bucket.cc could originally be applied
one at a time resulting in a complete buildable copy of ceph at each step.
Recent independent changes to rgw_bucket.cc have affected the same logic,
so the original commits no longer build.  This commit resyncs things
with master such that the result will build.

I'm preserving the intermediate commits for now, since that's what was
originally backported to jewel, If this causes problems, these commits
should be squashed together; except for backport purposes the intermediate
versions have no other value.

One other change here: omit logic that copied "VERSION_ATTR" to the
new container object.  This should already be provided for elsewhere.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
ea24c7fb66 rgw: bucket link: "bucket move" documentation changes
Improve and add to documentation for "bucket move" functionality;
including use moving to multi-tenancy and further deprecating bucket-id
which is no longer necessary.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
b908e45c02 rgw: bucket link: "bucket move"; handle bucket names too.
This is the remainder of bucket move function.  Implement bucket "rename"
- use string passed in via '--bucket-new-name' to optinally override
the resultant bucket name when doing bucket link.  This is basically
a slight generalization of changing the tenant of a bucket; the same
operations are required for either.

Further refinements included here are minor improvements to associated
error messages.

This does not change any "tail placement" information in any objects
contained in the bucket.  The bucket name is encoded there, along with
the bucket id, but neither appears to be used, and the existing reshard
logic which changes bucket ids also does not appear to alter that.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
100865d7a6 rgw: bucket link: base "bucket move" (tenant id only)
This is the base or primitive "bucket move" function.  It handles rewriting
the endpoint and info rados objects for the rgw bucket, plus deleting
the original rados objects that are no longer correct.  It doesn't
handle changing the bucket name; that's in a future commit.  Some of
the changes here will get overwritten by that commit.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
422cc79ae4 rearrange / simplify RGWBucket::link logic - start bucket move support
The existing RGWBucket::link logic changed things incrementally in
order to relink a bucket.  When doing a "bucket move", this is no longer
a good idea - bucket objects must be written to new names which don't
exist, so it is better to create then wholly out of in-memory data.

Also, add != for rgw_bucket - inverse of existing of ==, provides
another option to arrange code to make it more readable.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
190f293550 rgw: bucket link: use data from bucket_info to rewrite bucket_endpoint.
The bucket link command was doing a fetch of the entrypoint late
in the link process.  This makes it harder to do "bucket move"
functionality, because then it would need to know the old
bucket late in the process.  The bucketinfo structure has
all the data elements necessary to recreate the endpoint,
so the changes here arrange to just use that data.

In order to write the object it's also necessary to propagate
xattrs.  The only xattr that seems to be present on the endpoint
is "ceph.objclass.version", so that's what this copies out.
It appears that attribute may be set set separately by cls,
so I'm not sure this is actually necessary.  However, the
old code would have written it, so this code preserves
that behavior.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
31f01da633 rgw: bucket link: simplify use of get bucket info.
The bucket link command was doing an extra bucket
get info call because it needed attributes.  Revised
::init so that attributes could optionally be requested,
and eliminate now unnecessary call.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
edc18a56d2 rgw: bucket link: Add ability to name bucket w/ different tenant.
This is not a complete fix; but it makes it possible for the
bucket link command to correctly find and attempt to link a bucket
to a user with a different tenant.  The reason this is not a complete
fix is that with just this change, the resulting bucket is "broken";
a duplicate endpoint but 0 length contents is created, and
the info entry is not correctly moved.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
7f4cc1d21b Add "--bucket-new-name" option to radosgw-admin.
This just adds the command line option and related bits.

Underlying functionality will be a later commit.

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
f7236ba1f7 Add several types to ceph-dencoder.
Add types: RGWBucketEntryPoint obj_version rgw_user
These are structures that are visible as data at rest
inside of rados when a bucket is made via radosgw.

RGWBucketEntryPoint is the contents of a rados
object with names that may be either "<bucket-name>"
or "<tenant>/<bucket-name>"

rgw_user is a structure contained inside of RGWBucketEntryPoint
and other structures.

obj_version is visible as the xattr "ceph.objclass.version"
on rados objects in ".rgw.meta" that contain ".bucket.meta."

Fixes: http://tracker.ceph.com/issues/35885

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 14:00:45 +05:30
Marcus Watts
3ba7be8d1a rgw: making implicit_tenants backwards compatible.
In jewel, "rgw keystone implicit tenants" only applied to swift. As of
luminous), this option applies to s3 also.
Sites that used this feature with jewel now have outstanding data that
depends on the old behavior.

The fix here is to expand "rgw keystone implicit tenants" so that it
can be set to any of "none", "all", "s3" or "swift" (also 0=false=none,
1=true=all).  When set to "s3" or "swift", the actual id lookup
is also partitioned.

Formerly "rgw keystone implicit tenants" was a legacy opt.
This change converts it to the new style of option,
including support for dynamically changing it.

Fixes: http://tracker.ceph.com/issues/24348

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2019-07-30 13:49:20 +05:30
Sebastian Wagner
475159ec73
Merge pull request from sebastian-philipp/python-common-package-2
python-common:  Python common package

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2019-07-30 09:31:34 +02:00
Sebastian Wagner
3999238081
Merge pull request from SUSE/wip-orchestrator-deepsea-iscsi-ganesha
mgr/deepsea: return ganesha and iscsi endpoint URLs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-30 09:13:25 +02:00
Kai Wagner
3660e17e4b doc: pg_num should always be a power of two
Signed-off-by: Kai Wagner <kwagner@suse.com>
2019-07-30 08:35:16 +02:00
Jan Fajerski
60fe6eac74
Merge pull request from ceph/wip-ceph-volume-tests-no-dashboard
ceph-volume: when testing disable the dashboard
2019-07-30 08:23:59 +02:00
Kefu Chai
3169da7a1a
Merge pull request from smithfarm/wip-40974
rpm: improve ceph-mgr plugin package summaries

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-30 12:46:37 +08:00
Kefu Chai
bcd4ed6724
Merge pull request from smithfarm/wip-40973
rpm: fdupes in SUSE builds to conform with packaging guidelines

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-30 12:44:32 +08:00
Kefu Chai
227d1e9486
Merge pull request from tchaikov/wip-doc-crush-rule
doc/rados/operations: crush_rule is a name

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-07-30 09:28:08 +08:00
Kefu Chai
d59090afe0
Merge pull request from tchaikov/wip-40618
qa/tasks/mgr/test_progress.py: s/ev/new_event/

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-07-30 09:27:29 +08:00
Sage Weil
f1eac8ba4b mgr/telemetry: specify license when opting in
Choosing not to include this in the docs so that the user is more likely
to see this interactively.  (That is...probably good?)

Choose sharing-1-0.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-29 19:37:25 -05:00
Sage Weil
90f04775fc qa/suites/multimds/basic/tasks/ceph_test_snapshots: disable RECENT_CRASH
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-29 18:17:15 -05:00
Patrick Donnelly
3f3189b4d4
Merge PR into master
* refs/pull/28804/head:
	mds: delay exporting directory whose pin value exceeds max rank id

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-29 14:53:28 -07:00
Patrick Donnelly
ecf49c0820
Merge PR into master
* refs/pull/29339/head:
	client: don't report any vxattrs to listxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
2019-07-29 14:49:29 -07:00
Patrick Donnelly
7cec28542e
Merge PR into master
* refs/pull/29304/head:
	cephfs: avoid map been inserted by mistake

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-29 14:47:03 -07:00
Patrick Donnelly
7b59c02c87
Merge PR into master
* refs/pull/29321/head:
	Client: unlink dentry for inode with llref=0

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-07-29 14:45:44 -07:00
Patrick Donnelly
602e5f6eb4
qa: sleep briefly after resetting kclient
Continuation of 6b83f43ba0.

Fixes: https://tracker.ceph.com/issues/40999
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-29 14:08:17 -07:00
Andrew Schoen
7b7ff0d530 ceph-volume: when testing disable the dashboard
ceph-ansible has started enabling the dashboard by
default as of https://github.com/ceph/ceph-ansible/pull/4268

Disabling the dashboard gets us around needing to run
the grafana_server role which is required for the dashboard
and not needed for ceph-volume functional tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2019-07-29 15:25:19 -05:00
Patrick Donnelly
618c3576c0
Merge PR into master
* refs/pull/29338/head:
	qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-07-29 13:10:02 -07:00
Patrick Donnelly
2a82081f77
Merge PR into master
* refs/pull/28825/head:
	qa: wait for kernel client death
	qa: use hard_reset to reboot kclient

Reviewed-by: David Galloway <dgallowa@redhat.com>
2019-07-29 13:08:29 -07:00
Alfredo Deza
e48ad3ab26 ceph-volume tests: ensure that better heuristics exist for objectstore detection
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-29 15:50:14 -04:00
Alfredo Deza
b6a449e2e3 ceph-volume simple.activate better detect bluestore/filestore when type is not found
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-29 15:46:48 -04:00