Commit Graph

58186 Commits

Author SHA1 Message Date
Mykola Golub
2882f8c01a qa/workunits/rbd: check status also in pool dir after asok commands
wait_for_image_replay_stopped returns not when the state is stopped,
but when the state is not replaying. So a race was possible when an
asok command was running when the previos stop command was still in
progress, leading to unexpected results.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-10-03 10:10:33 +03:00
Dan Mick
62d8e60cb6 Merge pull request #11285 from liewegas/wip-mgr-rpm
ceph.spec: fix ceph-mgr version requirement

Reviewed-by: Dan Mick <dmick@redhat.com>
2016-09-30 15:40:35 -07:00
Sage Weil
3447601ac3 ceph.spec: fix ceph-mgr version requirement
Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 18:13:50 -04:00
Sage Weil
ad6aeb0f01 Merge pull request #11282 from liewegas/wip-bluestore-gc-fix
os/bluestore: fix gc when blob extends past eof
2016-09-30 16:17:57 -05:00
Sage Weil
570615b63a os/bluestore: fix gc when blob extends past eof
This can happen if we aren't referencing the bob.  Don't try to
read what isn't logically referenced.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 17:06:47 -04:00
Sage Weil
2dc5775449 Merge pull request #11279 from liewegas/wip-bluestore-gc
os/bluestore: compaction fixes
2016-09-30 13:50:08 -05:00
Sage Weil
ea0aee6073 os/bluestore: fix compaction left-side length
Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 14:49:35 -04:00
Sage Weil
1164a30235 os/bluestore: more concise SharedBlob print
Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 14:32:31 -04:00
Sage Weil
506e4aedd4 os/bluestore: cosmetic gc cleanups
Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 14:23:32 -04:00
Sage Weil
75bf4d18a2 Merge pull request #11271 from stiopaa1/osd_repicatedpg_addOverride
osd/ReplicatedPG.h: add override to virtual funct

Reviewed-by: Sage Weil <sage@redhat.com>
2016-09-30 12:57:02 -05:00
Sage Weil
ce9781ae8b Merge pull request #11239 from liewegas/wip-bluestore-max-blob
os/bluestore: add max blob size; fix compressed min blob size logic

Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>
2016-09-30 12:47:58 -05:00
Sage Weil
beed433254 os/bluestore: set min compressed blob size floor at 2x min_alloc_size
Otherwise there's not point in compressing--we won't be able to
allocation a smaller space to store it.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 13:47:23 -04:00
Sage Weil
87f6d23b2f os/bluestore: introduce bluestore_max_blob_size
This is enforced for compressed and non-compressed blobs alike.
We keep the separate compressoin min and max tunables since these
may be adjusted on a per-pool basis as well.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 13:47:22 -04:00
Sage Weil
f32b682f13 os/bluestore: adjust compression blob sizes down
These were way too big.  64KB is enough to get good compression,
and 4MB is way too big.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 13:47:22 -04:00
Sage Weil
a389a29e16 os/bluestore: fix blob depth encoding
1) Fix the flag constnat
2) Redefine the meaning to be more intuitive.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-30 13:45:36 -04:00
Sage Weil
c72bcd667e Merge pull request #11232 from Roushan81/wip-bluestore-blob-gc
os/bluestore: garbage collect partially overlapped blobs

Reviewed-by: Sage Weil <sage@redhat.com>
2016-09-30 09:43:33 -05:00
Sage Weil
599b6e459f Merge pull request #11263 from liewegas/wip-bluestore-reshard-slop
os/bluestore: sloppy reshard boundaries to avoid spanning blobs

Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>
2016-09-30 09:41:46 -05:00
Sage Weil
6bf86fefdc Merge pull request #11275 from xiexingguo/xxg-wip-make-audit-buffer-option
os/bluestore: drop redundant call of get_blob

Reviewed-by: Sage Weil <sage@redhat.com>
2016-09-30 09:21:37 -05:00
xie xingguo
a2aaf2b4a4 os/bluestore: drop redundant call of get_blob
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-09-30 22:12:14 +08:00
Sage Weil
094944230a Merge pull request #11268 from liewegas/wip-bluefs-aio-leak
os/bluestore/BlueFS: release completed aios
2016-09-30 08:51:39 -05:00
Jason Dillaman
42d51117ad Merge pull request #11005 from jazeltq/add-sparse-read
rbd-mirror: Add sparse read for sync image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-09-30 09:39:58 -04:00
Jason Dillaman
f3d3941177 Merge pull request #11168 from trociny/wip-16212
rbd-mirror: replicate image metadata settings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-09-30 09:37:50 -04:00
Sage Weil
979c6e8754 Merge pull request #11274 from xiexingguo/xxg-wip-bluestore-0930
os/bluestore: still more cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
2016-09-30 08:31:35 -05:00
John Spray
3707b30b98 Merge pull request #11136 from jcsp/wip-16016
mds: populate DamageTable from scrub and log more quietly

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2016-09-30 12:41:37 +01:00
Orit Wasserman
f555ddfafc Merge pull request #11270 from ceph/wip-17443
rgw: set correct instance on the object
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-09-30 10:41:25 +02:00
Kefu Chai
fce16433c6 Merge pull request #10328 from jcsp/wip-ceph-mgr
Create ceph-mgr

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-09-30 15:34:04 +08:00
xie xingguo
adb274c631 os/bluestore: avoid comparing object key and name twice
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-09-30 14:57:53 +08:00
xie xingguo
bea0e347c0 os/bluestore: use id of shard_id_t for encoding
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-09-30 14:35:20 +08:00
xie xingguo
6cece53d5f os/bluestore: kill struct DeferredCsum
Which is dead, and kill some other dead lines too.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-09-30 13:58:15 +08:00
Michal Jarzabek
b387747894 osd/ReplicatedPG.h: add override to virtual funct
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-09-30 06:40:47 +01:00
Roushan Ali
9cec7b6d15 os/bluestore: add garbage collection
Signed-off-by: Roushan Ali <roushan.ali@sandisk.com>
2016-09-29 22:11:09 -07:00
Sage Weil
76e120c705 Merge pull request #11267 from liewegas/wip-bluestore-ref-map
os/bluestore/bluestore_types: uint64_t for ref_map
2016-09-29 21:22:04 -05:00
Yehuda Sadeh
d2ada35c8c rgw: set correct instance on the object
Fixes: http://tracker.ceph.com/issues/17443

This was broken by commit bc840afafd

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-09-29 18:35:59 -07:00
Sage Weil
812f8d3e21 os/bluestore/BlueFS: be less complainy about fallback allocations
Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-29 18:05:20 -04:00
Sage Weil
5389133ff9 ceph_test_objecstore: use 1TB block device
Big enough to get into 32-bit LBAs.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-29 17:52:59 -04:00
Sage Weil
5fb4ea1a02 os/bluestore/bluestore_types: uint64_t for ref_map
We use this to track raw extents on disk for SharedBlob.  We
also use it for the in-memory Blob with the blob namespace, but
using uint64_t intead of uint32_t doesn't hurt us there.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-29 17:52:59 -04:00
Sage Weil
276c02b416 os/bluestore/BlueFS: release completed aios
Signed-off-by: Sage Weil <sage@redhat.com>
2016-09-29 17:05:53 -04:00
Sage Weil
b020ea78da Merge pull request #11266 from stiopaa1/osd_passSharedPtrByConstRef
osd: pass shared_ptr by const reference

Reviewed-by: Sage Weil <sage@redhat.com>
2016-09-29 15:17:09 -05:00
John Spray
6537e63457 pybind/mgr/rest: remove unused DebugJob
This dates back from the Calamari/Salt days.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:18 +01:00
John Spray
9a33766bd8 pybind/mgr/rest: fix "sync_object" API endpoint
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:18 +01:00
John Spray
592bb9c2a7 vstart: include cython mods in PYTHONPATH
This is useful for ceph-mgr because it means that
any in-mgr modules that want to use e.g. rbd
can find it when running in a vstart cluster.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:17 +01:00
John Spray
f8514ef581 mgr: remove hardcoded site-packages
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:17 +01:00
John Spray
042b6b4454 packaging: add debian packaging for ceph-mgr
...at least, try to!  This is a copy-paste of the ceph-mds
packaging with a search and replace mds to mgr.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:17 +01:00
Tim Serong
082199f69d systemd: autogenerate ceph-mgr key during daemon startup
This is a hack to inject a key for the mgr daemon, using whatever
key already exists on the mon on this node to gain sufficient
permissions to create the mgr key.  Failure is ignored at every
step (the '-' prefix) in case someone has already used some other
trick to set everything up manually.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
61d779345e systemd: encourage ceph-mgr to start in sync with ceph-mon
This change introduces the following behaviour:

- When ceph-mon starts, it will try to start ceph-mgr with the same
  instance id (Wants=), but will *not* fail to start if ceph-mgr
  doesn't start (i.e. the mon still works as it always did).
- ceph-mgr will start After= ceph-mon, and will stop and start when
  ceph-mon stops and starts, because it's PartOf= ceph-mon.

If you don't want ceph-mgr to run on the mons, you need to mask the
service, i.e. `systemctl mask ceph-mgr@INSTANCE`.  Hostnames are
typically instance names, so `systemctl mask ceph-mgr@$(hostname)`
should suffice if you wish to disable ceph-mgr on the mons.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
2e9e21eb42 ceph.spec.in: add ceph-mgr package
This adds ceph-mgr following the same pattern used for ceph-mon,
ceph-mds, ceph-osd.  I've removed the boost-python BuildRequire
for SUSE distros, because we don't ship that (the relevant bits
are already included in boost-devel).

NOTE: This gives an installable ceph-mgr, but not necessarily a
runnable ceph-mgr.  This is because ceph-mgr needs the following
python packages:

- CherryPy
- Django
- gevent
- python-dateutil
- djangorestframework

Not all of these are available as distro packages.  For example,
on openSUSE Tumbleweed, we don't have djangorestframework available
in the base distro, and our Django version is 1.9, which seems to
be incompatible with ceph-mgr.  I'm not sure of the situation on
other distros.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
d8ded57a87 systemd: add ceph-mgr service and target files
Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
c6972b983b cmake: install ceph-mgr and its python modules
Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:07 +01:00
John Spray
da76c59efc pybind/mgr: clean up unused code
...and change the "if False" django-rest-framework
conditionals to actual version checks, with a view
to genuinely supporting 2.x in future if needed.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:07 +01:00
John Spray
1bb9edbaf2 mgr: fix OSDMap notifications
This broken when MgrStandby was separated out,
because it now eats the original OSDMap message
without calling maybe_request_map.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:07 +01:00