Commit Graph

43994 Commits

Author SHA1 Message Date
Sage Weil
ea8609b25a mon/OSDMonitor: debug why pool creation fails
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-21 17:45:27 -04:00
Radoslaw Zarzynski
7c08c54fec rgw: fix dangerous removal from STL map in filter_out_temp_url().
Fixes: #12750
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2015-08-21 22:29:47 +02:00
Sage Weil
88bfd79669 test/erasure-code: drop directory from profile
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-21 16:03:30 -04:00
Sage Weil
5df12716b1 do not include directory in ec profiles
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-21 16:03:30 -04:00
Sage Weil
660ae5bcbb osd: always load erasure plugins from the configured directory
Ignore the profile 'directory' field.

This ensures that we can always find plugins even when teh cluster
is installed across a mix of distros.

Rename the option to have no osd_ (or mon_) prefix since anybody
may use the ec factory/plugin code.

We still hard-code .libs in the unit tests... sigh.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-21 16:03:30 -04:00
Jenkins
7295612d29 9.0.3 2015-08-21 12:46:32 -07:00
Loic Dachary
271513f8f0 erasure-code: shec plugin feature
There is one new plugin (shec). When upgrading a cluster, there
must be a protection against the following scenario:

   * the mon are upgraded but not the osd
   * a new pool is created using plugin shec
   * the osd fail to load the shec plugin because they have not been
     upgraded

A feature bit is added : PLUGINS_V3. The monitor will only agree to
create an erasure code profile for the shec plugin if all OSDs
supports PLUGINS_V3. Once such an erasure code profile is stored in the
OSDMap, an OSD can only boot if it supports the PLUGINS_V3 feature,
which means it is able to load the shec plugin.

The monitors will only activate the PLUGINS_V3 feature if all monitors
in the quorum support it. It protects against the following scenario:

   * the leader is upgraded the peons are not upgraded
   * the leader creates a pool with plugin=shec because all OSD have
     the PLUGINS_V3 feature
   * the leader goes down and a non upgraded peon becomes the leader
   * an old OSD tries to join the cluster
   * the new leader will let the OSD boot because it does not contain
     the logic that would excluded it
   * the old OSD will fail when required to load the plugin shec

This is going to be needed each time new plugins are added, which is
impractical. A more generic plugin upgrade support should be added
instead, as described in http://tracker.ceph.com/issues/7291.

See also 9687150cea for the PLUGINS_V2
implementation.

http://tracker.ceph.com/issues/10887 Fixes: #10887

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-08-21 21:27:04 +02:00
Kefu Chai
5e99a57805 mon: add a cache layer over MonitorDBStore
the cache of of leveldb does not perform well under some condition,
so we need a cache in our own stack.

* add an option "mon_osd_cache_size" to control the size of cache size
  of MonitorDBStore.

Fixes: #12638
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-08-22 00:26:49 +08:00
Gregory Farnum
929cf29fb3 Merge pull request #5634 from ceph/wip-fix-FTBFS-boost
rbd: fix the FTBFS on old boost introduced by 2050d08
2015-08-21 14:49:53 +01:00
Kefu Chai
2d13a47203 rbd: fix the FTBFS on old boost introduced by 2050d08
in boost 1.49, BOOST_SCOPE_EXIT() does not accept capture_tuple,
only `(capture) (capture) ...` is supported.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-08-21 21:41:01 +08:00
Gregory Farnum
bfc259efe5 Merge pull request #5583 from ceph/wip-12676
mds: check mds up before calling MDSMap::get_mds_info

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-08-21 13:11:46 +01:00
Kefu Chai
6425111861 Merge pull request #5587 from caibo2014/master
rbd: remove the output file if rbd export-diff fails

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-08-21 19:33:59 +08:00
Gregory Farnum
2daccff5de Merge pull request #5632 from ceph/wip-fix-ftbfs-4998
osd: fix the FTBFS introduced by be28319

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-08-21 12:22:34 +01:00
Kefu Chai
d1c4086808 osd: fix the FTBFS introduced by be28319
the signature of Scrubber::write_blocked_by_scrub() was changed in #5470.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-08-21 02:24:52 -07:00
Kefu Chai
e06207a38b Merge pull request #4998 from wonzhq/hitset-remove-all
osd: implement hit_set_remove_all

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-08-21 16:51:46 +08:00
Zhiqiang Wang
5fedc8414b osd: consolidate encoding snaps into pg log code in finish_ctx
'MODIFY', 'PROMOTE' and 'CLEAN' are sharing the same encoding snaps
code.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-21 16:08:46 +08:00
Zhiqiang Wang
fd38902dd4 osd: only remove all the hit set objects on primary osd
Only the primary PG is allowed to remove all the hit set objects. And
the PG should be in the active or peered states.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-21 15:48:20 +08:00
Zhiqiang Wang
5812adb6c8 osd: no need to check current hitset object is degraded when persisting
hit set

The current hit set object doesn't exist yet. We are about to create it.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-21 15:48:20 +08:00
Zhiqiang Wang
be28319bf3 osd: implement hit_set_remove_all
When hit set is not configured on startup or on a change, remove all
previous hitsets.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-21 15:45:04 +08:00
Yunchuan Wen
9c325d2b11 skip promote if flag set when doing proxy read
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>
2015-08-21 14:38:21 +08:00
Haomai Wang
faa964d3e5 Fix compile warning unused-result
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-08-21 14:16:17 +08:00
Kefu Chai
b33209b469 Merge pull request #5315 from tchaikov/wip-10930
mon: remove osd_epoch to avoid out-dated osdmap_cache

Reviewed-by: Sage Weil <sage@redhat.com>
2015-08-21 11:18:32 +08:00
Zhiqiang Wang
1546d57d71 osd: do either flush or evict but not both in agent_work
An cache pool object is either dirty or not. It's unlikely the agent
will do both flush and evict at the same time for an object.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-21 10:56:51 +08:00
Zhiqiang Wang
5eb80b81c8 osd: do evict before flush in agent_work
This is to avoid the extreme case that the agent continuously does
flush, but not evict. This may lead to the cache pool to be full.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-21 10:51:05 +08:00
Yehuda Sadeh
8314384a3c Merge pull request #5579 from jbweber/s3_encoding_type
rgw: implement s3 encoding-type for get bucket

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-08-20 15:45:53 -07:00
Jeff Weber
180ca7b56b rgw: implement s3 encoding-type for get bucket
This change introduces handling for the encoding-type request
parameter on the get bucket operation. An object key may contain
characters which are not supported in XML. Passing the value "url" for
the encoding-type parameter will cause the key to be urlencoded in the
response.

Fixes: #12735
Signed-off-by: Jeff Weber <jweber@cofront.net>
2015-08-20 22:33:35 +00:00
Sage Weil
566907779d buffer: move inline memory ops to inline_memory.h; gcc + x86_64 only
Keep the architecture-sensitive code in a separate header.
Avoid duplicating the unrolled memcpy in each buffer.cc
method.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-20 16:40:53 -04:00
Sage Weil
cdaf9974de unittest_bufferlist: benchmark buffer::ptr::append, copy_in, copy_out
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-20 16:30:44 -04:00
Sage Weil
5686d9badb Merge branch 'dont-cache-recovery' of git://github.com/yuyuyu101/ceph
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

# Conflicts:
#	src/osd/ReplicatedBackend.cc
#	src/osd/ReplicatedBackend.h
#	src/osd/ReplicatedPG.cc
2015-08-20 15:20:44 -04:00
Sage Weil
779466e34d Merge pull request #5259 from branch-predictor/bp-tuned-bufferlists
Bufferlist tuning

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2015-08-20 15:09:17 -04:00
Orit Wasserman
8bb76f2927 Merge pull request #5575 from ceph/wip-12370
rgw: don't preserve acls when copying object
2015-08-20 20:59:49 +02:00
Sage Weil
181dc28845 Merge pull request #5518 from ceph/wip-12652
osd/PGLog: dirty_to is inclusive

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-08-20 14:53:30 -04:00
Sage Weil
6468ffb67b Merge pull request #5539 from wonzhq/flush-mode-stat
mon: show # of PGs at high flush mode in the command output

Reviewed-by: Sage Weil <sage@redhat.com>
2015-08-20 14:47:24 -04:00
Yehuda Sadeh
aeb80d03e1 Merge pull request #5556 from zaitcev/cleanup-dead-code
rgw: cleanup dead init_bucket

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-08-20 09:15:01 -07:00
Yehuda Sadeh
f04322a848 Merge pull request #5598 from jasuarez/remove-useless-code
rgw: Remove useless code in calc_hmac_sha1()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-08-20 09:13:55 -07:00
Loic Dachary
bf108a3471 Merge pull request #5620 from dachary/wip-12734-uuid
build/ops: add uuid-runtime to debian/control

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-08-20 17:49:08 +02:00
Loic Dachary
d2d540c165 build/ops: add uuid-runtime to debian/control
uuid-runtime was incorrectly removed by
62bfc7a1ab, restore it.

http://tracker.ceph.com/issues/12734 Fixes: #12734

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-08-20 16:19:00 +02:00
Loic Dachary
4e88268d93 Merge pull request #5621 from ceph/revert-5613-wip-12729
Revert "Fix compile warning unused-result"

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-08-20 16:18:01 +02:00
Loic Dachary
951a3d6dcd Revert "Fix compile warning unused-result" 2015-08-20 16:17:13 +02:00
Sage Weil
910cc17b63 Merge pull request #5613 from ceph/wip-12729
Fix compile warning unused-result

Reviewed-by: Sage Weil <sage@redhat.com>
2015-08-20 09:08:14 -04:00
Yan, Zheng
bb70ac8e23 Merge pull request #5523 from ceph/wip-scan-use-layout
Striped layouts in cephfs-data-scan
2015-08-20 18:13:02 +08:00
Kefu Chai
aef8491319 Merge pull request #5272 from chenji-kael/work-branch
rados.py: wrap omap operation for librados

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-08-20 18:10:03 +08:00
Yan, Zheng
ea103798f3 Merge pull request #5614 from liewegas/wip-logtomonitors
mds/MDSRank: less noisy about log_to_monitors
2015-08-20 17:47:38 +08:00
caibo
2050d08cfd rbd:remove the local file when rbd export-diff fail
Signed-off-by: Bo Cai <cai.bo@h3c.com>
2015-08-20 17:40:41 +08:00
Radoslaw Zarzynski
69cf089596 rgw: url_decode values from X-Object-Manifest during GET on Swift DLO.
Fixes: #12728
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2015-08-20 11:25:16 +02:00
chenji
d353dbc53a rados.py: This module now supports omap operations
you can operate omap in python
test_rados.py: add unit-test cases

Signed-off-by: Ji Chen <insomnia@139.com>
2015-08-20 17:19:45 +08:00
Kefu Chai
050e1c5c74 Merge pull request #5601 from wonzhq/doc-peering
doc: fix the format of peering.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-08-20 11:01:29 +08:00
Sage Weil
14d3a2b82b os/FileStore: fix version check
We need to do this when we first read the version, before we
proceed with the mount.  By the time we get to upgrade() it is too
late (the DBObjectMap may have already tried a conversion, journal
may have replayed, etc.).

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-19 22:30:24 -04:00
Sage Weil
e7e4aefb95 osd: some debug output during store version upgrades
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-19 22:30:24 -04:00
Sage Weil
ffa224a25b mon: disallow post-hammer OSDs if there are up pre-hammer OSDs
Force *all* OSDs to upgrade to hammer before allowing post-hammer
OSDs to join.  This prevents any pre-hammer OSDs from running at
the same time as a post-hammer OSD.

This commit, as well as the definition of the sentinal post-hammer
feature, should get backported to hammer stable series.

Backport: hammer
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-19 22:30:24 -04:00