Commit Graph

80443 Commits

Author SHA1 Message Date
Casey Bodley
fa8a0713de qa/rgw: add kwargs for debug output
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:03 -05:00
Casey Bodley
d9dc58cf2c test/rgw: add test_bucket_index_log_trim()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
916f5995c9 rgw: add TrimComplete to watch/notify api
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
1c50d727b5 rgw: RGWDataSyncSingleEntryCR calls BucketChangeObserver
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
74aedaef07 rgw: start BucketTrimManager in RGWRados
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
35cf54e105 rgw: add BucketTrimInstanceCR
fetches bucket sync status from each peer, calculates the min markers
for each shard, and trims the bilog shards. calls the TrimObserver on
success

Fixes: http://tracker.ceph.com/issues/18229

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
04400bc60b rgw: add HTTPManager to BucketTrimManager
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
689badbe7f rgw: RGWGetBucketInstanceInfoCR takes rgw_bucket or metadata key
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
0dea44dad3 rgw: add RGWRadosBILogTrimCR
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
f0caa1ca8a rgw: add RGWBucketInfo overload for BucketShard::init
for use by callers that have already read the bucket instance info

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:02 -05:00
Casey Bodley
fab595ab9b rgw: add /admin/log rest api for bucket sync status
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
28c6ac62b8 rgw: add json decoders for bucket sync status
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
403d139c67 rgw: add rgw_bucket_sync_status() to bypass manager
RGWBucketSyncStatusManager::init() is doing a lot of extra work that's
not needed to serve the rest api (spawning an http manager thread,
fetching the bucket instance info, etc)

uses RGWShardCollectCR to limit the number of concurrent reads to 16

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
4f736f1ec7 rgw: add 'radosgw-admin bilog autotrim'
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
d29f96ae3e rgw: add configure_bucket_trim()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
7be4eab8a3 rgw: BucketTrimManager implements BucketTrimObserver
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
06a22a134f rgw: collect cold buckets for trim
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
2c07d7dd0e rgw: add BucketTrimStatus
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
c9d50860b7 rgw: add MetadataListCR to loop over bucket instances
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:01 -05:00
Casey Bodley
82c0595308 rgw: add BucketTrimCR to spawn trim for active buckets
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:00 -05:00
Casey Bodley
129fc99d52 rgw: add BucketTrimPollCR for interval and lease logic
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:00 -05:00
Casey Bodley
5bcf109eac rgw: add TrimCounters api to BucketTrimWatcher
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:00 -05:00
Casey Bodley
f96d9a8e22 rgw: add BucketTrimWatcher to serve watch/notify apis
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:00 -05:00
Casey Bodley
b4249cc432 rgw: BucketTrimManager implements BucketChangeObserver
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:23:00 -05:00
Casey Bodley
e9a5ec9f64 common: introduce BoundedKeyCounter and unit test
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 13:22:58 -05:00
Casey Bodley
4309adb36b rgw: add skeleton for BucketTrimManager
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 12:16:55 -05:00
Casey Bodley
964d966969 rgw: introduce RGWRadosNotifyCR for aio_notify
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 12:16:55 -05:00
Casey Bodley
b3a5c5a056 rgw: MetadataManager interface takes const string refs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 12:16:55 -05:00
Yuri Weinstein
54b657ec0d
Merge pull request #18821 from prallabh/wip-22084
rgw: Fix swift object expiry not deleting objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 08:04:31 -08:00
Yuri Weinstein
c6d5612063
Merge pull request #18846 from yaozongyou/fix-lifecycle-bug
rgw: check going_down() when lifecycle processing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-10 08:04:11 -08:00
Haomai Wang
39da34bfba
Merge pull request #18877 from Jeegn-Chen/wip-issue-22103
rados: make ceph_perf_msgr_client work for multiple jobs

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-11-10 08:43:13 -06:00
Jeegn Chen
5c1a860d61 rados: make ceph_perf_msgr_client work for multiple jobs
Move the invocation of msgr->start() before a client initiates a connection.
So the nonce is probably initialized and the server side is able to differenciate
the connections from different Messengers from the same client.

Fixes: http://tracker.ceph.com/issues/22103
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>
2017-11-10 20:59:01 +08:00
Abhishek L
f9d8317cbc
Merge pull request #18876 from smithfarm/wip-rn-typos
doc: release notes: fix grammar/style nits

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2017-11-10 13:34:20 +01:00
Nathan Cutler
d12ce99fd4 doc: release notes: fix grammar/style nits
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-11-10 11:49:14 +01:00
Haomai Wang
8ccaa64bf1
Merge pull request #18862 from Jeegn-Chen/wip-issue-22100
rados: Copy payload in ceph_perf_msgr_client

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-11-09 21:47:28 -06:00
Jeegn Chen
f1625e8905 rados: Copy payload in ceph_perf_msgr_client
Copy payload in ceph_perf_msgr_client so that payload can keep unchanged

Fixes: http://tracker.ceph.com/issues/22100
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>
2017-11-10 10:14:50 +08:00
Sage Weil
9ef5f20bf4
Merge pull request #18707 from liewegas/wip-filestore-rocksdb-compression
os/filestore: disable rocksdb compression

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2017-11-09 20:08:04 -06:00
Andrew Schoen
cbf06de900
Merge pull request #18823 from ceph/wip-bz1498200
ceph-volume: support GPT and other deployed OSDs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2017-11-09 12:01:12 -06:00
Yuri Weinstein
89c26b9838
Merge pull request #18817 from ZVampirEM77/wip-em-zone-cleanup
rgw cleanup: some unnecessary function called and repeated assignment

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-09 09:36:26 -08:00
Sage Weil
3855d2cbfd
Merge pull request #18683 from tchaikov/wip-erasure-cleanup
erasure-code: refactor the interfaces to hide internals from public

Reviewed-by: Sage Weil <sage@redhat.com>
2017-11-09 11:10:27 -06:00
Alfredo Deza
e2f281c115 doc/ceph-volume include the top level systemd.rst in the toctree
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:16 -05:00
Alfredo Deza
ff60592897 ceph-volume simple.activate add --file in help docstring as an example
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:16 -05:00
Alfredo Deza
29913951b7 doc/ceph-volume activate has a --file option for json files
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:16 -05:00
Alfredo Deza
366fb8c43a ceph-volume simple.activate enable the osd when activating
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:16 -05:00
Alfredo Deza
cb470622fd ceph-volume systemd.systemctl always force symlink when masking ceph-disk
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:16 -05:00
Alfredo Deza
c954c91339 ceph-volume simple.activate use the contents of the JSON file, fallback to CLI args
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:15 -05:00
Alfredo Deza
826c2709dd ceph-volume simple.activate only check for missing id and uuid if file is not passed in
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:15 -05:00
Alfredo Deza
bd3498f3ea ceph-volume simple.activate allow to pass a file to activate
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:15 -05:00
Alfredo Deza
d7fc26ab4e ceph-volume tests.util add tests for OSDPath validator
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:15 -05:00
Alfredo Deza
cda47c247b ceph-volume systemd.systemctl masking ceph-disk needs to be done by directly symlinking to dev/null
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-09 12:02:15 -05:00