Commit Graph

59707 Commits

Author SHA1 Message Date
Casey Bodley
885606c76c rgw: add accounted_size argument to Write::write_meta
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
b8fb18ade3 rgw: add accounted_size argument to UpdateIndex::complete
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
2d2d370762 rgw: add accounted_size to RGWObjEnt
and use accounted_size for BucketList operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
9a105a6966 rgw: add accounted_size to RGWObjState
RGWRados::get_obj_state() now initializes accounted_size based on
the compression attribute, if present

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
20c723b6d4 cls/rgw: use rgw_bucket_dir_entry_meta::size to accumulate actual_size
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:05 +01:00
Casey Bodley
63db0ebc7f cls/rgw: add actual_size to rgw_bucket_caregory_stats
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:10:25 +01:00
Casey Bodley
abecf85177 cls/rgw: fix function names of log output
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:10:25 +01:00
Adam Kupczyk
a5f8ebc9ee Added missing decompression when reading SLO.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:10:25 +01:00
Kefu Chai
39cbb557f3 Merge pull request #11738 from ceph/wip-cmake
cmake: do not link erasure tests again libosd

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-11-02 18:44:26 +08:00
Mykola Golub
8e1cc88e06 rbd-mirror: snap protect of non-layered image results in split-brain
Fixes: http://tracker.ceph.com/issues/16962
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-11-02 12:43:45 +02:00
Kefu Chai
9613de3002 Merge pull request #11384 from wjwithagen/patch-14
test_subman.sh: Don't use --tmpdir

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-02 18:39:28 +08:00
Adam Kupczyk
6fd1910516 Added compression handling when syncing.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 11:35:23 +01:00
Adam Kupczyk
1fe4aa9fc7 Moved handling extra data from RGWPutObjProcessor to RGWRadosPutObj
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 11:35:22 +01:00
Adam Kupczyk
681f65b4b9 Cleanup of compression-related code in RGWGetObj::execute.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 11:35:22 +01:00
Adam Kupczyk
128132d390 RGWRados::Object::Read::prepare no longer calculates offsets.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 11:35:22 +01:00
Adam Kupczyk
c609215e5a Fixed problem with using uninitialized 'compressed' field.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 11:35:22 +01:00
Casey Bodley
6db48432b9 test/rgw: add unit test for rgw compression
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 11:35:22 +01:00
Ved-vampir
cd0cc75a86 rgw: move compression code in filters system
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:35:22 +01:00
Adam Kupczyk
b1a6f77eb6 Renamed and moved RGWPutObjEncryptFilter -> RGWPutObj_Filter.
Signed-off-by: Adam Kupczyk (akupczyk@mirantis.com)
2016-11-02 11:34:51 +01:00
Adam Kupczyk
69740441fe Introduced filter concepts to RGWGetObj. Plugged in decryption.
Allows to plug in decompression in future.

Signed-off-by: Adam Kupczyk (a.kupczyk@mirantis.com)

Conflicts:
	src/rgw/rgw_op.cc
2016-11-02 11:34:50 +01:00
Adam Kupczyk
8c8ce72a8e Change contract of rgw put operation. Now last, 0 - sized chunk is also processed.
Allows to implement flush-like operations on data.

Signed-off-by: Adam Kupczyk (akupczyk@mirantis.com)

Conflicts:
	src/rgw/rgw_op.cc
2016-11-02 11:34:50 +01:00
Adam Kupczyk
70ff7147ac Added RGWPutObjEncryptFilter. Will be used for encryption and compression filters.
Signed-off-by: Adam Kupczyk (akupczyk@mirantis.com)
2016-11-02 11:34:50 +01:00
Adam Kupczyk
9f4233d843 Changed hash calculation logic.
Changed RGWPutObjProcessor interface with regard to hash calculation.

Signed-off-by: Adam Kupczyk (akupczyk@mirantis.com)

Conflicts:
	src/rgw/rgw_op.cc
	src/rgw/rgw_rados.cc
2016-11-02 11:34:50 +01:00
Ved-vampir
e66ea7d3af rgw: fix output for stat functions to return decompressed data size
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:34:50 +01:00
Ved-vampir
89174520f6 rgw: compression: add multipart and copy support
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:34:49 +01:00
Ved-vampir
8a1ce0fb8e rgw: add compression support for ranged queries
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:31:10 +01:00
Ved-vampir
44d397d4c8 rgw: replace multiple compression attributes by struct
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:31:09 +01:00
Ved-vampir
988462a801 rgw: add decompression & cmpressor attrs save
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:31:09 +01:00
Ved-vampir
11f8d97949 rgw: add compression to RGWPutObj
Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>
2016-11-02 11:31:09 +01:00
Haomai Wang
31d5e4b9ae pybind/rgw_file: fix unnecessary return value
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-02 18:19:13 +08:00
Haomai Wang
2a120bedc8 rgw/file: don't crash if st is NULL
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-02 18:18:38 +08:00
Haomai Wang
f4abfe5b45 test/pybind: add tests for rgwfile
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-02 18:18:38 +08:00
Haomai Wang
4e5f66a801 pybind/rgw_file: add rgwfile pybinding
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-02 18:18:38 +08:00
Haomai Wang
c06b8f53f4 rgw/rgw_file: add rgwfile_version
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-02 18:18:38 +08:00
Willem Jan Withagen
0894865ed8 test_subman.sh: Don't use --tmpdir
test_subman.sh: Don't use --tmpdir since on most Linux-is it is consideren default
when creating a directory:
`If TEMPLATE is not specified, use tmp.XXXXXXXXXX, and --tmpdir is implied.`

And on FreeBSD --tmpdir is not available anyways.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-11-02 11:03:34 +01:00
xie xingguo
ec071376e7 os/bluestore: use string for most cases in _open_db()
For a better readability.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 17:41:49 +08:00
Kefu Chai
9bdc5d1328 cmake: do not link erasure tests again libosd
they are not referencing any symbols from it. and also the tests testing
plugins are not linked against the ec plugins, they *depend* on them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-11-02 14:56:05 +08:00
haodong
bb340c8d7a kv/rocksdb: enable rocksdb write path breakdown
bluestore has unstable write performance partly because of
rocksdb internal overhead. So we add more breakdown in
rocksdb writh path.

Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2016-11-02 12:57:49 +08:00
Li Wang
7a7f2fc81b FileStore: Only verify split when it has been really done and done correctly
Signed-off-by: Li Wang <li.wang@kylin-cloud.com>
2016-11-02 03:37:58 +00:00
Kefu Chai
3d83dc8454 Merge pull request #11730 from ceph/wip-17654
librados-dev: install inline_memory.h

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-02 10:53:24 +08:00
xie xingguo
857263fe3e os/bluestore: drop unused parameter "deep" from _fsck_check_extents()
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 09:56:37 +08:00
xie xingguo
69088dc893 tools/ceph_objectstore_tool: handle fsck and fsck-deep op in unified path
181e28ba99
introduces too much code redundancy, this patch simplifies it a little.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 09:19:15 +08:00
Michal Jarzabek
3e82fb6a84 mds/MDSRank: add const to member functions
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-11-01 22:31:37 +00:00
Josh Durgin
ac1c1f6f0e librados-dev: install inline_memory.h
This is needed to compile against the c++ api since
c38869232c

Fixes: http://tracker.ceph.com/issues/17654
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-11-01 15:15:38 -07:00
Michal Jarzabek
fdf66e1a6a mds/MDSRank.h: add override to virtual function
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-11-01 21:57:55 +00:00
Yehuda Sadeh
f890a71501 Merge pull request #11697 from weiqiaomiao/wqm-wip-rgw-bilist
rgw: fix osd crashes when execute "radosgw-admin bi list --max-entries=1" command

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-01 13:57:07 -07:00
Sage Weil
352078deaa Merge pull request #11695 from tanghaodong25/cache-logger-wip
os/bluestore: update cache logger after 'trim_cache' operation

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-01 13:25:06 -05:00
Yehuda Sadeh
d36cc6d0fd Merge pull request #11506 from cbodley/wip-17569
rgw: clear data_sync_cr if RGWDataSyncControlCR fails

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-01 11:16:09 -07:00
Matt Benjamin
2aecf6b73c Merge pull request #11707 from yehudasa/wip-17755
rgw: json encode/decode index_type, allow modification
2016-11-01 13:51:54 -04:00
Sage Weil
27cee5ab9e os/bluestore: rename variable in set_lextent
Defensive naming

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 13:46:06 -04:00