Commit Graph

59795 Commits

Author SHA1 Message Date
Sage Weil
b0b8c5a9bf Merge pull request #11752 from stiopaa1/mds_mdsrank_addConstToMemFunc
mds/MDSRank: add const to member functions
2016-11-03 08:48:37 -05:00
Sage Weil
13dbff2de9 Merge pull request #11761 from xiexingguo/xxg-wip-fix-envmirror-cleanup
os/bluestore: fixes and cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-03 08:43:15 -05:00
Matt Benjamin
47a2749f2f Merge pull request #11762 from atheism/doc-rgw-s3-javaapi
doc: rgw: fix a typo in S3 java api example
2016-11-03 09:31:56 -04:00
Mykola Golub
5577ada030 librbd: proper check for get_data_pool compatibility
Fixes: http://tracker.ceph.com/issues/17791
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-11-03 15:27:14 +02:00
Mykola Golub
2012b4dfc6 rbd-nbd: disallow mapping images >2TB in size
Fixes: http://tracker.ceph.com/issues/17219
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-11-03 14:50:47 +02:00
Weibing Zhang
654112000e doc: rgw: fix a typo in S3 java api example
ObjectSummary.getSize() should be objectSummary.getSize()

Signed-off-by: Weibing Zhang <zhangweibing@unitedstack.com>
2016-11-03 18:23:02 +08:00
Orit Wasserman
f216268083 Merge pull request #11753 from yehudasa/wip-17790
rgw: support for x-robots-tag header
Reviewed-by: Orit wasserman <owasserm@redhat.com>
2016-11-03 11:07:24 +01:00
Uday Mullangi
7c7395c1cc doc: Fixing the broken hyperlinks by pointing to correct documentation.
Signed-off-by: umullangi@walmartlabs.com
2016-11-03 14:55:12 +05:30
xie xingguo
a47e20aeae os/bluestore: fix improper call of shutdown() to mempool_thread
Because below here we haven't called mempool_thread.init() yet,
and call to mempool_thread.shutdown() will therefore raise an assert instead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-03 16:57:33 +08:00
xie xingguo
d129a2868a os/bluestore: drop safe check against g_conf
We are using g_conf unconditionly everywhere...

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-03 14:44:42 +08:00
xie xingguo
69b8e4d55a os/bluestore: drop redundant boundary check
As we have done the sanity check ahead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-03 14:18:24 +08:00
xie xingguo
88ca5b113d os/bluestore: fix "rm -rf" cmd
The original cmd won't do the tidy-up work completedly.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-03 14:14:49 +08:00
Venky Shankar
05653b7c51 librbd: batch ObjectMap updations upon trim
Shrinking a clone which has snapshots and does not share
majority of objects with its parent (i.e., there are less
objects to be copied up) involves huge number of object
map updates -- two (pre, post) per object. This results
in lots of requests to be send to OSDs especially when
trimming a gigantus image. This situation can be optimized
by sending batch ObjectMap updates for an object range
thereby significantly cutting down OSD traffic resulting
in faster trim times.

Fixes: http://tracker.ceph.com/issues/17356
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-03 08:53:55 +05:30
Samuel Just
3ba2e2e21a ReplicatedPG::do_update_log_missing: take the pg lock in the callback
Fixes: http://tracker.ceph.com/issues/17789
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-02 17:40:43 -07:00
Joao Eduardo Luis
f0e29abe4c tools: monmaptool: support monmap features
Adds three new options:

  --feature-list     will list monmap features, as well as available
                     features.
  --feature-set      sets a feature on the monmap
  --feature-unset    unsets a feature on the monmap

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
acf4b8604c test/mon: few more cases for mon_feature_t
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
dcd70789f6 mon: Monitor: mon debug cli via admin socket
Purpose: allow a dev to test certain portions of the monitors on-the-fly
that would otherwise be nearly impossible without requiring shutting
them off.

Description:

  Currently we are adding support to debug monitors on-the-fly via the
  ceph tool cli, via the admin socket.

  An initial attempt was made to make this interface available via
  over-the-network ceph cli, but that would require a quorum to exist,
  or significant changes to be made to the handling of commands to allow
  this interface to be an exception to the rule.

  Therefore, this interface will live on the admin sockets for the time
  being; potentially forever, because the jury is still out on whether
  there's added value making this available via any other medium that's
  not the admin socket.

  The work means to test the newly added monmap features. This new
  interface will allow one to set, unset and list supported features.

  This will make it easier to implement functional tests for the
  features themselves, for quorum features that are expected to be set
  on-quorum, as well as to test upgrade paths from version A to version
  B.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
5435aca15b mon: MonMap: move functions from header to source file
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
cd6ff52e75 mon: MonMap: add a new test instance
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
af3aaaebb1 mon: MonMap: mon addr and all now held by mon_info_t
Instead of keeping several maps mapping back and forth from string to
entity_addr_t for each monitor, have the monitor's address and name in
purpose-specific class, mon_info_t, and everything references it
instead.

Although the benefits may not be obvious right now, it will allow us to
add other ip addresses for the monitors a bit more easily, instead of
having to kludge around the existing structures.

We also keep compatibility with older versions by maintaining the
traditional 'mon_addr' map. This is a logical change inside the monitor,
so we can easily accomodate older monitors without having to rely on
quorum features; just needs a bit of care and foo.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
2ee1b9a408 krbd.cc: don't rely on MonMap internal members
Use the public interface instead. That's stable and not as prone to
change.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Yehuda Sadeh
ca1be285f9 rgw: support for x-robots-tag header
Fixes: http://tracker.ceph.com/issues/17790

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-02 15:56:06 -07:00
Sage Weil
fac6335a1e Merge pull request #11331 from liewegas/wip-mempool
mempool: mempool infrastructure, bluestore changes to use it

Reviewed-by: Allen Samuels <allen.samuels@sandisk.com>
2016-11-02 12:50:23 -05:00
Sage Weil
9cecff8d18 mempool: use function static for pool table
The compiler/linker guarantee this is initialized before any invocation
of this function... even if it is by a ctor in another compilation unit
that is initialized by the mempool.cc compilation unit.

Suggested by Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>
2016-11-02 13:48:49 -04:00
Sage Weil
ebf052762e mempool: do not dump unused pools
Notably, unittest_[12]. :)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
a54d21749d mempool: put pool namespaces within mempool::
e.g., mempool::bluestore_meta_other::list<...> ...

This avoids colliding with other names and types when the pool
name is something common (like "osd").

Signed-off-by: Sage Weil <sage@redhat.com>

# Conflicts:
#	src/include/mempool.h
2016-11-02 13:48:49 -04:00
Sage Weil
b15a377a7b mempool: simplify debug settings
There's no need for a per-pool debug flag; just use the global
debug_mode bool!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
62fa72c0e1 mempool: force per-type factories to register with the pool
We want these to get their debug flag adjusted when it is turned
on and off.

(In contrast, we accept that STL containers will only be debugged
if mempool debug was on when they are constructed.)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
82251cc51d include/encoding: parameterize stl containers encoders with allocator
This will let us encode/decode containers with custom allocators
(i.e., in mempools).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
bfeaed6412 mempool: simplify initialization
Assume that the array will be zeroed when the process starts, and
avoid the possibility that it will be zeroed *again* when the
ctors run for this link module (potentially clobbering values
that have been filled in by other ctors who ran earlier than
ours).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
79ee6eec6c include/types: parameterize operator<< helpers on comparator, allocator
This allows these templates to apply to containers with
non-default comparators or allocators.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
ca93284120 include/mempool: fix comment
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
049797641b mempool: dynamic index by type in pool_t
This avoids having to use statics for the pool_allocators and
guessing what intenral types the containers are going to need.
It'll be a bit slower in debug on on construction and destruction,
but who cares!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
55d42d0830 include/mempool: define additional pair factory for unordered_map
This fixes build on debian jessie, libstdc++-4.9-dev:amd64 4.9.2-10.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
a6eec64c95 mempool: pick shard on every allocation; move types to pool_t
If we have a static choice of shard for every *type*
(pool_allocator_base_t) then we will hammer that shard from all
threads and play cacheline ping-pong.  Instead, move the types
list to pool_t (there aren't that many anyway) and pick a shard
on every allocate/deallocate call.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
b174694493 mempool: make num_shards a power of 2
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
735be3662f include/mempool: rename containers -> types
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
1b51c05068 include/mempool: note about runtime complexity
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
34f434be90 os/bluestore/bluestore_types: disable ref_map internal _check
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
bcf20a1ca1 os/bluestore: restructure cache trimming in terms of mempool
Trim cache based on overall memory utilization by cache objects,
as tracked by the bluestore_meta_* mempools.  This lets you
configure the bluestore cache size in terms of bytes of memory.

Note that we do not account for other memory utilization by the
OSD.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
030bc063e4 os/bluestore: move most cache types into mempools
Keep onodes separate so we can use onodes as a sentinal for
overall memory usage (this is what we trim in terms of anyway).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
81295c61c4 global: introduce mempool_debug config option, asok command
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
9cbacc81cc include/mempool: introduce mempool infrastructure
This is heavily based on Allen Samuels' prototype.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Adam Kupczyk
144b031242 Merge pull request #2 from cbodley/wip-rgw-compression-doc
doc/rgw: document rgw_compression_type
2016-11-02 17:34:56 +01:00
Alfredo Deza
bea802bd13 ceph-create-keys: wait 10 minutes to get or create the bootstrap key, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2016-11-02 12:30:30 -04:00
Alfredo Deza
8f84681a4d ceph-create-keys: wait 10 minutes to get or create a key, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2016-11-02 12:25:32 -04:00
Alfredo Deza
32cedd2c18 ceph-create-keys: wait for quorum for ten minutes, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2016-11-02 12:19:10 -04:00
Joao Eduardo Luis
20dcb597e3 mon: MonmapMonitor: drop unnecessary 'goto' statements
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 15:38:36 +00:00
Joao Eduardo Luis
c9d46cfbf2 mon: MonmapMonitor: return success when monitor will be removed
Fixes: http://tracker.ceph.com/issues/17725

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 15:33:52 +00:00
Casey Bodley
2fdd35dc2b rgw: remove unused rgw_compression_enabled
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 11:18:41 -04:00