Commit Graph

70620 Commits

Author SHA1 Message Date
Radoslaw Zarzynski
6f1871bc85 rgw: switch to std::array in RGWBulkUploadOp due to C++11 and FreeBSD.
Before this patch RGWBulkUploadOp::terminal_errors was declared as
std::initializer_list<int>. Unfortunately, a constexpr constructor
for it is available since C++14, not C++11. This was causing build
failures on FreeBSD.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-04-04 18:24:20 +02:00
Sage Weil
f8f9094964 os/bluestore: do not extend blobs with unused bitmap
If we resize the blob we need to adjust the resolution of the
unused bitmap, and that is only possible for some bit patterns.
For now just ignore blobs with unused blocks.

Add an assert in add_tail() so that we don't forget that
add_tail is (probably) where we'd would do that adjustment.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-04 12:12:26 -04:00
Sage Weil
83e7b81737 os/bluestore: some whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-04 12:12:25 -04:00
Sage Weil
045acc3fb0 os/bluestore: mark_used for wctx->writes
The refactored blob reuse code needs to mark blobs used when
processing wctx->writes.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-04 12:12:25 -04:00
Anjaneya Chagam
614175b1c0 Bug fix for http://tracker.ceph.com/issues/18576
Refer to http://lttng.org/docs/v2.9/ on dlclose side effects. Workaround is to use
RTLD_NODELETE flag when calling dlopen.

Signed-off-by: Anjaneya Chagam <anjaneya.chagam@intel.com>
2017-04-04 08:56:26 -07:00
Sage Weil
31c550f7bd os/bluestore: prefix fsck errors with 'fsck error:'
Greppable!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-04 11:28:29 -04:00
Amir Vadai
af2f1ecd27 msg/async/rdma: Make port number an attribute of the Connection not of the Device
Since multiple connections on different ports could exist, shouldn't use
device->active_port, instead use conn->ibport.
Or in other words, now Device object doesn't have an active_port,
instead every port specific action (create_qp, get_lid, get_gid etc.)
need to specify the port number. The information about the port number
is known to the connection (RDMAConnectedSocket*) who is the caller of
those actions.

Issue: 995322
Change-Id: I482cb87c04ba99845dc44f6dd0547835fe814ebf
Signed-off-by: Amir Vadai <amir@vadai.me>
2017-04-04 17:52:10 +03:00
Kefu Chai
c0e044c1bd Merge pull request #14310 from tchaikov/wip-yield-for-ppc
common/simple_spin: use __ppc_yield() on all powerpc archs

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Tested-by: Andrew Solomon <asolomon@us.ibm.com>
2017-04-04 21:58:30 +08:00
Kefu Chai
ffea11550e Merge pull request #14289 from tchaikov/wip-conditionalize-crc32c
cmake,crc32c: conditionalize crc32c on different archs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Tested-by: Andrew Solomon <asolomon@us.ibm.com>
2017-04-04 21:57:22 +08:00
Loic Dachary
9f64925658 Merge pull request #14275 from ceph/evelu-cephdisk-race
ceph-disk: Adding retry loop in get_partition_dev()

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-04-04 14:17:18 +02:00
Kefu Chai
06cf99cbcd Merge pull request #14185 from DHB-liuhong/objects
mon/PGMap: make si units more readable in PGMap summary

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-04 19:10:16 +08:00
Kefu Chai
d7d32be097 Merge pull request #14313 from wjin/clean_mgr
mgr: rm nonused main function

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-04 19:01:50 +08:00
Erwan Velu
93e7b95ed8 ceph-disk: Adding retry loop in get_partition_dev()
There is very rare cases where get_partition_dev() is called before the actual partition is available in /sys/block/<device>.

It appear that waiting a very short is usually enough to get the partition beein populated.

Analysis:
update_partition() is supposed to be enough to avoid any racing between events sent by parted/sgdisk/partprobe and
the actual creation on the /sys/block/<device>/* entrypoint.
On our CI that race occurs pretty often but trying to reproduce it locally never been possible.

This patch is almost a workaround rather than a fix to the real problem.
It offer retrying after a very short to be make a chance the device to appear.
This approach have been succesful on the CI.

Note his patch is not changing the timing when the device is perfectly created on time and just differ by a 1/5th up to 2 seconds when the bug occurs.

A typical output from the build running on a CI with that code.
	command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
	get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
	get_partition_dev: Try 1/10 : partition 2 for /dev/sda does not in /sys/block/sda
	get_partition_dev: Found partition 2 for /dev/sda after 1 tries
        get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
	get_dm_uuid: get_dm_uuid /dev/sda2 uuid path is /sys/dev/block/8:2/dm/uuid

fixes: #19428

Signed-off-by: Erwan Velu <erwan@redhat.com>
2017-04-04 12:46:46 +02:00
Wei Jin
3f857b9ae4 mgr: rm nonused main function
Signed-off-by: Wei Jin <wjin.cn@gmail.com>
2017-04-04 17:47:56 +08:00
lvshuhua
7d448b7d8d rgw: add --num-zonegroups option for multi test
Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>
2017-04-04 17:20:15 +08:00
Nathan Cutler
26c1b4c2d7 Merge pull request #14174 from shinobu-x/wp-sk-typo-osdmonitor-20170328
mon/OSDMonitor: Clean up: delete extra S signature for plural

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-04 08:37:09 +02:00
Brad Hubbard
7943ab2e01 ceph-disk: Populate mount options when running "list"
Also tidy up by moving duplicated code into a function

Fixes: http://tracker.ceph.com/issues/17331
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-04 15:25:48 +10:00
Kefu Chai
7f2ac2334a crc32c,compressor: only compile functions on supported arch
so we can still compile if certain header file (<sys/auxv.h>) used
for detecting cpu capabilities on ppc is missing on x86 host machine.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-04 11:38:29 +08:00
Kefu Chai
58e173c54d common/simple_spin: use __ppc_yield() on all powerpc archs
__ppc_yield() is declared in sys/platform/ppc.h by glibc, for better
portability we just use the inlined assembly here. the shared resource
hints are supported by PowerPC ISA 2.06 but on older PowerPC cores, they
are no-ops. so it's fine to do this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-04 11:23:48 +08:00
Ning Yao
5b486d9628 test: add test for rbd import,import-diff,copy with sparse-size option
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
2017-04-04 03:18:52 +00:00
yaoning
a7dd9f1816 librbd: add sparse copy support
Fixes: fix issue #15648
http://tracker.ceph.com/issues/15648

Signed-off-by: yaoning <yaoning@unitedstack.com>
2017-04-04 03:18:52 +00:00
yaoning
f60b2d0cba rbd: import/import-diff real thin-provision image
currently, import/import-diff check whether object is zero based on the whole object
therefore, holes inside object will be filled with zero

Signed-off-by: yaoning <yaoning@unitedstack.com>
2017-04-04 03:18:52 +00:00
yaoning
8a83a2d611 rbd: add sparse size support in rbd
add --sparse_size in rbd command options
add calc_sparse_extent in rbd utils

Signed-off-by: yaoning <yaoning@unitedstack.com>
2017-04-04 03:18:52 +00:00
Brad Hubbard
b05bc817be client/SyntheticClient.cc: Fix warning in random_walk
Fixes: http://tracker.ceph.com/issues/19445
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-04 11:12:12 +10:00
Orit Wasserman
14867ad398 Merge pull request #12243 from rzarzynski/wip-rgw-bulkupload
rgw: add support for the BulkUpload of Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-04-03 21:54:32 +03:00
Casey Bodley
8e31c002e3 doc/rgw: update section on global quotas
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:48:58 -04:00
Casey Bodley
4ede51f2d4 radosgw-admin: new 'global quota' commands update period config
Fixes: http://tracker.ceph.com/issues/19409

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:48:50 -04:00
Casey Bodley
6c6f78bcc0 rgw: load period config separately when there's no current period
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:42:51 -04:00
Casey Bodley
b49b065343 rgw: synchronize period config with period updates
read period config from local storage on RGWPeriod::update(), and write
the new config to storage on RGWPeriod::reflect()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:42:51 -04:00
Casey Bodley
70a2f03810 rgw: allow RGWPeriodConfig to be stored separately
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:42:51 -04:00
Casey Bodley
275b3182bb rgw: constify some rgw_pool args
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:42:51 -04:00
Casey Bodley
a6034436e3 rgw: remove unused RGWPeriod::update(RGWZoneGroupMap)
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 14:42:51 -04:00
Yehuda Sadeh
e18b2f26e9 Merge pull request #14120 from wangzhengyong/uninitialized
rgw: fix uninitialized fields

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-04-03 11:33:23 -07:00
Andrew Solomon
e11756b494 build: src/arch/ppc.c FreeBSD build break
The #include file <sys/auxv.h> is present on linux but
not FreeBSD.  Move it under the #if __linux__ directive.

Signed-off-by: Andrew Solomon <asolomon@us.ibm.com>
2017-04-03 17:04:21 +00:00
Sage Weil
bd4ba40b9e mgr: move 'osd perf' and 'osd blocked-by' to mgr
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-03 12:46:32 -04:00
Sage Weil
ba730ecd70 mon: move 'osd perf' and 'osd blocked-by' to PGMap
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-03 12:46:21 -04:00
Sage Weil
7b7402f695 Merge pull request #14217 from Liuchang0812/wip-18584-pg-info-dump
osd: print pg_info_t::purged_snaps as array, not string

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-03 10:46:36 -05:00
John Spray
2bf485d366 Merge pull request #14036 from batrick/mds-debug-reply
mds: bump client_reply debug to match client_req

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-03 16:20:19 +01:00
Loic Dachary
21f5737d4a Merge pull request #14296 from theanalyst/fix/script/credits
script: drop the -x arg for credits script

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-04-03 17:03:12 +02:00
Sage Weil
c3d1333794 Merge pull request #13401 from liewegas/wip-kill-subop
osd: kill all remaining MOSDSubOp users

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-04-03 10:00:53 -05:00
John Spray
531118fbe8 Merge pull request #14221 from jcsp/wip-19253-b
qa: whitelist another fullness log message

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-03 15:31:29 +01:00
Sage Weil
aec22a7d57 Merge pull request #14279 from jimmyway/wip-fix-pg-comment
osd: fix comments about pg refs and lock

Reviewed-by: Pan Liu <liupan1111@gmail.com>
2017-04-03 09:27:13 -05:00
Sage Weil
f5803eb7b3 Merge pull request #14277 from atheism/fix-test-objectstore-chain_xattr
test: objectstore: chain_xattr: fix wrong memset usage to fill buf

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-03 09:26:30 -05:00
Sage Weil
2c7ee9dfec os/bluestore: flush osr on collection split
We need to ensure that any preceding txcs have finished their deferred
writes before splitting, or else later txcs on the child sequencer will
not order/wait for deferred writes correctly before deallocated extents.

Fixes: http://tracker.ceph.com/issues/19379
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-03 09:57:18 -04:00
Sage Weil
78a5bdeef1 os/bluestore: make deferred_aggressive atomic
This is written and read outside of any mutex.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-03 09:57:18 -04:00
Sage Weil
3ea9790ba3 osd: use target object hash (not pgid.ps) for temp objects
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-03 09:57:18 -04:00
Sage Weil
4d888d491e Merge pull request #13800 from xiexingguo/wip-clean-pgcreate-throttle
common/config_opts.h: kill mon_pg_create_interval

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-03 08:38:07 -05:00
Radoslaw Zarzynski
51dd349d65 Merge pull request #13621 from Jing-Scott/add-remov-x-delete-feature
rgw: add the remove-x-delete feature to cancel swift object expiration

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-04-03 15:19:12 +02:00
John Spray
13e8315d1a Merge pull request #13862 from jcsp/wip-16523
qa, mds: add checks for fragmentation, and enable it by default
2017-04-03 11:56:37 +01:00
Shinobu Kinjo
daae49f959 mon/OSDMonitor: Clean up: delete extra S signature for plural
Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>
2017-04-03 19:28:21 +09:00