Commit Graph

98355 Commits

Author SHA1 Message Date
Radoslaw Zarzynski
c400cf336f crimson: object state can be manually evicted from OSD's cache.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 18:05:34 +08:00
Radoslaw Zarzynski
d86ffe51aa crimson: retrieve object state once per MOSDOp.
Justification behind the change is behaviour of classical OSD.
It calls PrimaryLogPG::find_object_context() far before going
through OSDOps in ::do_osd_ops().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 18:04:53 +08:00
Radoslaw Zarzynski
de5e5160ca crimson: add support for basic write path, part 1.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 17:49:01 +08:00
Radoslaw Zarzynski
c8cd0dd54b crimson/os: add inclusion guard in Transaction.h
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 17:48:35 +08:00
Radoslaw Zarzynski
1dfa5b8b88 crimson: PG caches ObjectState instead of object_info_t.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 17:48:35 +08:00
Radoslaw Zarzynski
0bfcde95e8 crimson: PG::do_osd_op() takes OSDOp by reference, not pointer.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 17:48:10 +08:00
Radoslaw Zarzynski
3c3320d996 crimson: PG::do_osd_op throws about unsupported op.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 17:48:10 +08:00
Radoslaw Zarzynski
c3473cdbc0 crimson: handle hot CaynStore exceptions without throwing.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-05-07 17:47:45 +08:00
Yuval Lifshitz
2d410b5a2e
Merge pull request #27931 from yuvalif/pubsub_info_update_fix
rgw/pubsub: fix doc on updates. fix multi-notifications
2019-05-07 12:18:53 +03:00
Yuval Lifshitz
4def929395 rgw: add tenant as parameter to User in multisite tests
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-05-07 12:05:33 +03:00
Ricardo Marques
f243f7a531 mgr/dashboard: Fix iSCSI form when using IPv6
Fixes: https://tracker.ceph.com/issues/39578

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-05-07 09:38:41 +01:00
Kefu Chai
5d31861aa8 crimson/osd: shutdown services in the right order
we should stop config service *after* osd is stopped, as osd depends on
a working and alive config subsystem when stopping itself. for instance,
the destructor of AuthRegistry unregisters itself from the ObserverMgr,
which is in turn a member variable of ConfigProxy, so if ConfigProxy is
destroyed before we destroy mon::Client, we will have a segfault with
following backtrace

ObserverMgr<ceph::md_config_obs_impl<ceph::common::ConfigProxy>
>::remove_observer(ceph::md_config_obs_impl<ceph::common::ConfigProxy>*)
at /var/ssd/ceph/build/../src/common/config_obs_mgr.h:78
AuthRegistry::~AuthRegistry() at
/var/ssd/ceph/build/../src/crimson/common/config_proxy.h:101
 (inlined by) AuthRegistry::~AuthRegistry() at
/var/ssd/ceph/build/../src/auth/AuthRegistry.cc:28
ceph::mon::Client::~Client() at
/var/ssd/ceph/build/../src/crimson/mon/MonClient.h:44
ceph::mon::Client::~Client() at
/var/ssd/ceph/build/../src/crimson/mon/MonClient.h:44
OSD::~OSD() at /usr/include/c++/9/bits/unique_ptr.h:81

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-07 15:06:43 +08:00
Brad Hubbard
4825072720 osd: Improve dump_pgstate_history json output
Fixes: http://tracker.ceph.com/issues/38846

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-05-07 14:36:06 +10:00
Kefu Chai
4a8d366672
Merge pull request #27975 from tchaikov/wip-pidfile_write
global/pidfile: pass string_view instead of ConfigProxy to pidfile_wr…

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-05-07 11:19:13 +08:00
Sage Weil
55d5ed4f77 Merge PR #27932 into master
* refs/pull/27932/head:
	osd: clean up osdmap sharing
	osd: minor cleanup in dequeue_op

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-05-06 21:40:11 -05:00
Richael Zhuang
40c97b91b3 vstart.sh: enable creating multiple OSDs backed by spdk backend
Currently vstart.sh only support deploying one OSD based on NVMe SSD.
The following two cases will cause errors:
1.There are 2 more NVMe SSDs from the same vendor on the machine
2.Trying to deploy 2 more OSDs if we only get 1 pci_id available

Add the support for allowing deploying multiple OSDs on a machine with
multiple NVME SSDs.

Change-Id: I6016435c1438bb4d16aff31f4575e03ccd3c9b3d

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
2019-05-07 10:11:19 +08:00
Casey Bodley
7179536314 test/rgw: add url to endpoints in test-rgw-multisite.sh
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-05-06 16:33:04 -04:00
Casey Bodley
3cb21a4d00 test/rgw: add bash variable for url
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-05-06 16:32:40 -04:00
Gregory Farnum
6dbe49605e
Merge pull request #26152 from noonedeadpunk/mgr/zabbix_discovery
mgr/zabbix Added pools discovery and per-pool statistics

Reviewed-by: Wido den Hollander <wido@42on.com>
2019-05-06 13:15:54 -07:00
Yuval Lifshitz
10b8b57d72 rgw/pubsub: revert the RGWSysObjectCtx change
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-05-06 23:08:48 +03:00
Jason Dillaman
87278548a5 qa/workunits/rbd: wait for rbd-nbd unmap to complete
The "unmap" request is asynchronous, so wait for a short amount
of time for the "rbd-nbd" daemon process to exit.

Fixes: http://tracker.ceph.com/issues/39598
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-06 15:16:03 -04:00
Casey Bodley
0316a81875 rgw: delete_obj_index() takes mtime for bilog
writing an empty timestamp to the bilog prevents other zones from
applying the delete. this means that the --bypass-gc flag for
'radosgw-admin bucket rm' doesn't work in multisite

Fixes: http://tracker.ceph.com/issues/24991

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-05-06 15:05:57 -04:00
Sebastian Wagner
5affd3b943 Apply suggestions from code review
Signed-off-by: Dmitriy Rabotjagov <noonedeadpunk@ya.ru>
2019-05-06 20:33:22 +03:00
Casey Bodley
f5c515e72a
Merge pull request #27967 from liewegas/wip-fix-rgw-bias
rgw/rgw_tools: fix 'osd pool set' json syntax

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-05-06 13:05:54 -04:00
Lenz Grimmer
22d6a82ad9
mgr/dashboard: Allow the decrease of pg's of an existing pool (#27785)
mgr/dashboard: Allow the decrease of pg's of an existing pool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-05-06 17:38:34 +02:00
Casey Bodley
1da57c3aa7
Merge pull request #27977 from cbodley/wip-39597
doc/rgw: document CreateBucketConfiguration for s3 PUT Bucket api

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2019-05-06 10:50:07 -04:00
Kefu Chai
4276ee233a
Merge pull request #27968 from iotcg/master
vstart.sh: correct ceph-run path

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-05-06 21:39:16 +08:00
Casey Bodley
e97668bb7a doc/rgw: document CreateBucketConfiguration for s3 PUT Bucket api
Fixes: http://tracker.ceph.com/issues/39597

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-05-06 09:33:07 -04:00
letterwuyu
ecb88dce63 osd: change pg_missing_item encode and decode
since luminous, recovery_deletes is always true, and octopus won't
maintain compatibility with jewel OSDs

Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 21:30:06 +08:00
Kefu Chai
89d62afcdf global/pidfile: pass string_view instead of ConfigProxy to pidfile_write()
there is no need to pass ConfigProxy to this function. and passing a
string_view also make it easier to reuse this function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-06 21:04:27 +08:00
Casey Bodley
c2b0fa3c79
Merge pull request #27970 from Liuchang0812/duplicate-tags-bl
rgw: eliminates duplicated tags_bl var

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-05-06 08:43:46 -04:00
letterwuyu
20eb8a2e95 osd: fix add_next_event about clean_regions
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:44 +08:00
letterwuyu
6ab1e39941 osd: fix ReplicatedBackend::submit_push_data log output
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:44 +08:00
letterwuyu
e12f8cf93d osd: add a comment to pg_missing_set::add
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
7b581ed9ad osd: fix indentation
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
2e935df15c osd: make ObjectCleanRegions::dump consistent
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
6ec07190da osd: using SERVER_OCTOPUS everywhere
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
7148dda076 osd: supplement doc
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
e9f328ef08 osd: fix ObjectRecoveryInfo decode
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
lishuhao
b7a0002888 fix the type of _size in class interval_set, int64_t -> uint64_t
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
lishuhao
cb927925af osd: mark ObjectCleanRegions dirty for related operations
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:42 +08:00
lishuhao
2455e08347 osd: change build_push_op and submit_push_data base recovery strategy
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:32 +08:00
Nathan Cutler
d04b448160 pybind/tox: pass additional command line arguments through to tox
Fixes: 9426f1f204
Fixes: https://tracker.ceph.com/issues/39579
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-05-06 12:36:19 +02:00
Abhishek Lekshmanan
d4c462a94c rgw: admin: introduce objects expire-stale list and rm
These commands help cleanup stale expired objects after a reshard has happened
Contains the following changesets:

* rgw_bucket: add a AdminOp function to cleanup stale expired objects
There is also a dry run option, which will allow listing to just view the
objects before we actually delete them
* rgw_bucket: introduce rgw_object_get_attr helper function
Since this could be re-used across other functions that try to get a single
xattr, both get acl and the new object expire stale commands now use this
function to read the value of a single xattr

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-05-06 11:52:13 +02:00
Abhishek Lekshmanan
5e56ea6f07 rgw_common: introduce a decode_bl function to decode a bl to type
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-05-06 11:51:57 +02:00
Abhishek Lekshmanan
1d1ea1fa39 rgw: rgw_bucket: set BucketInfo as const in rgw_remove_object
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-05-06 11:51:57 +02:00
Abhishek Lekshmanan
7f958c64df rgw: object expirer: handle resharded buckets
Previously we fetched the bucket instance info which changes during a reshard
causing the bucket info to fail, since the subsequent checks will assume that
would mean a deleted bucket, the expiry hint is purged as well, leaving a non
deleted object as well as a deleted hint. This should fix newer runs of object
expiry processes. Finding out stale expired objects will require more complex
rgw-admin tooling support

Fixes: https://tracker.ceph.com/issues/39495
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-05-06 11:51:57 +02:00
Lenz Grimmer
1991495a22
doc: fix small typo in dashboard documentation (#27850)
doc: fix small typo in dashboard documentation

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2019-05-06 10:22:39 +02:00
lishuhao
28af7820f2 osd: add ObjectCleanRegions for pg log
we use merge ObjectCleanRegions to calculate the data that needs recovery when merge log

Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00
lishuhao
19e71a2f89 test: add unittest case for ObjectCleanRegion
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00