Commit Graph

59611 Commits

Author SHA1 Message Date
Joao Eduardo Luis
5435aca15b mon: MonMap: move functions from header to source file
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
cd6ff52e75 mon: MonMap: add a new test instance
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
af3aaaebb1 mon: MonMap: mon addr and all now held by mon_info_t
Instead of keeping several maps mapping back and forth from string to
entity_addr_t for each monitor, have the monitor's address and name in
purpose-specific class, mon_info_t, and everything references it
instead.

Although the benefits may not be obvious right now, it will allow us to
add other ip addresses for the monitors a bit more easily, instead of
having to kludge around the existing structures.

We also keep compatibility with older versions by maintaining the
traditional 'mon_addr' map. This is a logical change inside the monitor,
so we can easily accomodate older monitors without having to rely on
quorum features; just needs a bit of care and foo.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Joao Eduardo Luis
2ee1b9a408 krbd.cc: don't rely on MonMap internal members
Use the public interface instead. That's stable and not as prone to
change.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 23:50:12 +00:00
Yehuda Sadeh
ca1be285f9 rgw: support for x-robots-tag header
Fixes: http://tracker.ceph.com/issues/17790

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-02 15:56:06 -07:00
Sage Weil
fac6335a1e Merge pull request #11331 from liewegas/wip-mempool
mempool: mempool infrastructure, bluestore changes to use it

Reviewed-by: Allen Samuels <allen.samuels@sandisk.com>
2016-11-02 12:50:23 -05:00
Sage Weil
9cecff8d18 mempool: use function static for pool table
The compiler/linker guarantee this is initialized before any invocation
of this function... even if it is by a ctor in another compilation unit
that is initialized by the mempool.cc compilation unit.

Suggested by Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>
2016-11-02 13:48:49 -04:00
Sage Weil
ebf052762e mempool: do not dump unused pools
Notably, unittest_[12]. :)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
a54d21749d mempool: put pool namespaces within mempool::
e.g., mempool::bluestore_meta_other::list<...> ...

This avoids colliding with other names and types when the pool
name is something common (like "osd").

Signed-off-by: Sage Weil <sage@redhat.com>

# Conflicts:
#	src/include/mempool.h
2016-11-02 13:48:49 -04:00
Sage Weil
b15a377a7b mempool: simplify debug settings
There's no need for a per-pool debug flag; just use the global
debug_mode bool!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
62fa72c0e1 mempool: force per-type factories to register with the pool
We want these to get their debug flag adjusted when it is turned
on and off.

(In contrast, we accept that STL containers will only be debugged
if mempool debug was on when they are constructed.)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
82251cc51d include/encoding: parameterize stl containers encoders with allocator
This will let us encode/decode containers with custom allocators
(i.e., in mempools).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
bfeaed6412 mempool: simplify initialization
Assume that the array will be zeroed when the process starts, and
avoid the possibility that it will be zeroed *again* when the
ctors run for this link module (potentially clobbering values
that have been filled in by other ctors who ran earlier than
ours).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
79ee6eec6c include/types: parameterize operator<< helpers on comparator, allocator
This allows these templates to apply to containers with
non-default comparators or allocators.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
ca93284120 include/mempool: fix comment
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
049797641b mempool: dynamic index by type in pool_t
This avoids having to use statics for the pool_allocators and
guessing what intenral types the containers are going to need.
It'll be a bit slower in debug on on construction and destruction,
but who cares!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
55d42d0830 include/mempool: define additional pair factory for unordered_map
This fixes build on debian jessie, libstdc++-4.9-dev:amd64 4.9.2-10.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
a6eec64c95 mempool: pick shard on every allocation; move types to pool_t
If we have a static choice of shard for every *type*
(pool_allocator_base_t) then we will hammer that shard from all
threads and play cacheline ping-pong.  Instead, move the types
list to pool_t (there aren't that many anyway) and pick a shard
on every allocate/deallocate call.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
b174694493 mempool: make num_shards a power of 2
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
735be3662f include/mempool: rename containers -> types
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
1b51c05068 include/mempool: note about runtime complexity
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
34f434be90 os/bluestore/bluestore_types: disable ref_map internal _check
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
bcf20a1ca1 os/bluestore: restructure cache trimming in terms of mempool
Trim cache based on overall memory utilization by cache objects,
as tracked by the bluestore_meta_* mempools.  This lets you
configure the bluestore cache size in terms of bytes of memory.

Note that we do not account for other memory utilization by the
OSD.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
030bc063e4 os/bluestore: move most cache types into mempools
Keep onodes separate so we can use onodes as a sentinal for
overall memory usage (this is what we trim in terms of anyway).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
81295c61c4 global: introduce mempool_debug config option, asok command
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
9cbacc81cc include/mempool: introduce mempool infrastructure
This is heavily based on Allen Samuels' prototype.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Adam Kupczyk
144b031242 Merge pull request #2 from cbodley/wip-rgw-compression-doc
doc/rgw: document rgw_compression_type
2016-11-02 17:34:56 +01:00
Alfredo Deza
bea802bd13 ceph-create-keys: wait 10 minutes to get or create the bootstrap key, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2016-11-02 12:30:30 -04:00
Alfredo Deza
8f84681a4d ceph-create-keys: wait 10 minutes to get or create a key, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2016-11-02 12:25:32 -04:00
Alfredo Deza
32cedd2c18 ceph-create-keys: wait for quorum for ten minutes, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2016-11-02 12:19:10 -04:00
Joao Eduardo Luis
20dcb597e3 mon: MonmapMonitor: drop unnecessary 'goto' statements
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 15:38:36 +00:00
Joao Eduardo Luis
c9d46cfbf2 mon: MonmapMonitor: return success when monitor will be removed
Fixes: http://tracker.ceph.com/issues/17725

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-11-02 15:33:52 +00:00
Casey Bodley
2fdd35dc2b rgw: remove unused rgw_compression_enabled
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 11:18:41 -04:00
Casey Bodley
42a18edf26 doc/rgw: document rgw_compression_type
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 11:18:19 -04:00
Sage Weil
d033f15b2d Merge pull request #11726 from liewegas/wip-bluestore-fsck-omap
os/bluestore: fsck: fix omap_head check

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 08:49:11 -05:00
Sage Weil
886d201eea os/bluestore: fsck: fix omap_head check
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 09:48:37 -04:00
Sage Weil
bcbcc5a4ce Merge pull request #11740 from xiexingguo/xxg-wip-bluestore-1102
os/bluestore: less code redundancy

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-02 08:47:53 -05:00
Sage Weil
a126f8d59e Merge pull request #11725 from liewegas/wip-bluestore-shard-span
os/bluestore: fix extent shard span check

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2016-11-02 08:34:50 -05:00
Adam Kupczyk
867917c7b7 Fixed warnings.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:14:10 +01:00
Casey Bodley
8548241dfd rgw: maintain uncompressed size in copy_obj_data
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:14:10 +01:00
Adam Kupczyk
fc8f9a15b6 Fixed bad rebase on read_op.prepare()
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:14:10 +01:00
Adam Kupczyk
1fa1a2e061 Fixed problem with isal-regular gzip compatibility.
Added tests.

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:14:10 +01:00
Adam Kupczyk
d8ed3e8b8c Fixed missing param to processor->complete().
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
e67e85c954 radosgw-admin: add 'size_utilized' to bucket stats
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
b6e6fc107e rgw: use RGWStorageStats::dump() to format 'bucket stats'
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
df64c45803 rgw: Bucket::List::list_objects no longer reads compression attribute for size
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
bbf5fddbfc rgw: remove compression attribute from bucket instance
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00
Adam Kupczyk
99fd03bb12 Fixed possible problems when compression expands data.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:13:06 +01:00
Adam Kupczyk
262f6683c0 Fixed problem with snappy compressor when data size <=3.
Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>
2016-11-02 12:13:06 +01:00
Casey Bodley
7902ce2069 rgw: pass accounted_size directly into RGWPutObjProcessor::complete
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-02 12:13:06 +01:00