Commit Graph

50148 Commits

Author SHA1 Message Date
Samuel Just
cc1b2c6f34 ReplicatedPG: move client reply handling out of eval_repop
execute_ctx is the only path which actually has a client op,
let's clarify eval_repop by moving that logic inline there.
This also eliminates most of the repop->ctx users from
eval_repop.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
5110214337 osd/: replace simple_repop_.* with simple_opc_.*
This way, we don't expose the RepGather structure
up into the users (which pretty much exclusively
use the repop->ctx member anyway).  This will pave
the way to removing RepGather::ctx.

Part of this involves generalizing the repop callback
members (queue_snap_trimmer and on_applied) to
on_applied, on_committed, on_success, and on_finish
on both OpContext and RepGather.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
6db7fe7de3 introduce CEPH_FEATURE_SERVER_JEWEL feature bit
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Jason Dillaman
0c44f20a30 Merge pull request #7793 from tchaikov/wip-fix-dencoder-MirrorImage
cls_rbd: fix the test for ceph-dencoder

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-02-25 10:33:19 -05:00
Kefu Chai
a704270287 cls_rbd: fix the test for ceph-dencoder
initialize MirrorImage::state to a certain value, otherwise the
generated test instances would be undetermined, and hence fail
the dencoder tests.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 23:07:49 +08:00
Loic Dachary
7131c977f0 Merge pull request #7792 from tchaikov/wip-13505
fix FTBFS introduced by d0af316

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-25 22:04:26 +07:00
Loic Dachary
61ab9e8e70 Merge pull request #7731 from SUSE/wip-shebang
ceph_detect_init/__init__.py: remove shebang

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-25 21:03:21 +07:00
Kefu Chai
557c3bfe0d osd: always cleanup the scrub results
the destructor of Scrub::Store asserts that `this->results` should
always be empty when it is free'd. before this change, if we are
repairing, the results is not flushed or cleaned up, this crashes
osd daemon when a new PG internval comes. also we should not keep
the results around even we don't want to populate them to the omap
if we are repairing, it's just a waste.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 21:47:30 +08:00
Kefu Chai
b0b402193d pybind: fix the FTBFS introduced by d0af316
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 21:47:30 +08:00
Kefu Chai
67aa66ee9c Merge pull request #7734 from dachary/wip-14834-pidfile-test
tests: test_pidfile.sh lingering processes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-25 16:19:08 +08:00
Kefu Chai
ac7d11ad73 librados: add missing tracpoint for scrub APIs
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-24 22:39:57 -08:00
Loic Dachary
c1e1c8d18e tests: test_pidfile.sh lingering processes
For the mon, using ps aux to find the process will only show part of the
command line and the $RUNID may be missing. For the OSD, the $RUNID is
passed using an environment variable and will not show at all.

There is no need to test both ceph-osd and ceph-mon since they both end
up in the same code path, pidfile_write. Add a log line that only shows
when the --pid-file is empty and grep for it in the simplest possible
way.

The TEST_pidfile test failed to kill a process because there was
no pidfile with its pid. This is fixed by backuping the pidfile before
it is modified.

http://tracker.ceph.com/issues/14834 Fixes: #14834

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-25 12:24:00 +07:00
Kefu Chai
ce1169d245 Merge pull request #6898 from tchaikov/wip-13505
add scrub persist/query API

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-02-25 12:48:14 +08:00
Kefu Chai
e2374c43c9 rados: add "list-inconsistent-snapset" cmd
to list inconsistent snapsets of a given PG, this command exposes
get_inconsistent_snapsets() rados API to user.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
8018eab344 rados: add "list-inconsistent-obj" cmd
to list inconsistent objects of a given PG, this command exposes
get_inconsistent_objects() rados API to user.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
c9b593d2d7 librados: add get_inconsistent_snapsets() API
Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
dfc2f4823b librados: add get_inconsistent_objects() API
Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
3dea4f1f5c osd: add CEPH_OSD_OP_SCRUBLS pg op
it is a new pg op which returns the encoded objects stored when
scrubbing.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
2009ed274d osd: persist inconsistent snapsets using omap
the inconsistent snapsets are identified in ReplicatedPG::_scrub()
after we compared the authorized objects with their replica/shards.
these inconsistent information are stored in the omap of objects
with prefix "SCRUB_SS_".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
c828c398b1 librados: add inconsistent_snapset_t type
for presenting the inconsistent snapsets found in scrub

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Samuel Just
8ed62772bd osd/: clear scrub store safetly
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
fb956b742c osd: persist inconsistent objs using omap
persist inconsistent objects found when comparing the ScrubMaps
collected from replica/shards. the discrepancies between the auth
copy and the replica are identified as inconsistencies. and hence
encoded into the omap of an object of the temp coll of the PG in
question.
scrub_types.{h,cpp} are introduced to hide the details of how we
persist the scrub types from the librados client.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
f236b7d736 osd: more constness to spg_t
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:40:17 +08:00
Kefu Chai
b43d480938 librados: add inconsistent_obj_t types
which present the inconsistent objects found in scrub

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:40:17 +08:00
Kefu Chai
4c3270692e rados: add "list-inconsistent-pg" command
to list inconsistent PGs of a given pool. this command exposes
the underlying get_inconsistent_pgs() API to user.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:40:17 +08:00
Kefu Chai
d0af316cf2 pybind: add Rados.get_inconsistent_pgs method
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:40:17 +08:00
Kefu Chai
cb4efbd72f librados: add get_inconsistent_pgs() to librados
to list the inconsistent PGs of given pool, it's a wrapper
around the "ceph pg ls" command.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:40:17 +08:00
Kefu Chai
50bbf7f92e tools/rados: support more --format options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:40:17 +08:00
Yehuda Sadeh
1c4ccfe3c4 Merge pull request #7597 from rzarzynski/wip-rgw-keystone-proper-fail
rgw: improve error handling in S3/Keystone integration

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-24 14:40:02 -08:00
Sage Weil
f053e3e7f0 Merge pull request #7051 from wonzhq/scrub-cmp-map
osd: avoid FORCE updating digest been overwritten by MAYBE when comparing scrub map

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-24 11:35:38 -08:00
Sage Weil
48d007f659 Merge pull request #7617 from liewegas/wip-14745
osd: fix forced prmootion for CALL ops

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-24 10:45:59 -08:00
Sage Weil
e67b70e7fa Merge pull request #7286 from ktdreyer/wip-10587-init-ceph-disk
init-ceph.in: skip ceph-disk if it is not present

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-24 10:42:37 -08:00
Sage Weil
ccfbca425a Merge pull request #7599 from ifed01/wip-14707
log: fix stack overflow when flushing large log lines

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-24 10:42:04 -08:00
Sage Weil
c10f62874a Merge pull request #7643 from jazeltq/add_count_github
osd: filejournal: report journal entry count

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-24 10:41:17 -08:00
Sage Weil
fa8ba08b96 Merge pull request #7658 from majianpeng/bluestore
osd: bluestore: misc fixes

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-24 10:40:26 -08:00
Sage Weil
6aa9a4735a Merge pull request #7677 from xiexingguo/xxg-wip-14786
osd: fix fusestore hanging during stop/quit

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-24 10:39:27 -08:00
Sage Weil
4229cd436d Merge pull request #7680 from ceph/wip-da-SCA-20160203
common: various fixes from SCA runs

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <jdillama@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-02-24 10:38:06 -08:00
Sage Weil
b3141ce1d0 Merge pull request #7681 from jjhuo/kstore-updates
osd: kstore: sync up kstore with recent bluestore updates

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-24 10:37:13 -08:00
Jason Dillaman
8ff1a8df1c Merge pull request #7759 from trociny/wip-rbd-mirror-image-replayer-improvements
rbd-mirror: ImageReplayer improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-02-24 11:40:43 -05:00
Jason Dillaman
53d87e1775 Merge pull request #7736 from trociny/fix-librbd-asok-empty-name
librbd: retrieve image name when opening by id

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-02-24 11:07:22 -05:00
Jason Dillaman
f0e428ad45 Merge pull request #7620 from jdurgin/wip-14419
cls_rbd: mirroring directory

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-02-24 10:35:28 -05:00
Josh Durgin
28e2d1bc55 cls_rbd: add methods for keeping track of mirrored images
These will track whether local images should be mirrored, and map them
to a unique global id. There's a state field for safely disabling
mirroring while operating on multiple objects.

Fixes: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-02-24 10:34:28 -05:00
Josh Durgin
da9b36a70d librbd: rename rbd_pool_settings object to rbd_mirroring
We'll use this object only for mirroring-related purposes, not generic
settings on a pool.

Refs: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-02-24 10:34:28 -05:00
Loic Dachary
894b3b7cf4 Merge pull request #7762 from ErwanAliasr1/evelu-check
Improving 'make check' for ceph-disk

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-24 21:42:02 +07:00
Mykola Golub
5360d8612d librbd: init asok_hook on open so name is always known
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-02-24 16:28:05 +02:00
Mykola Golub
80656b1131 librbd: get image name on open if it is opened by id
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-02-24 16:28:05 +02:00
Kefu Chai
1ee8c13f9c Merge pull request #7743 from JiYou/open-test-for-bug-2339
test: enable test for bug #2339 which has been resolved.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-24 21:57:05 +08:00
Haomai Wang
281b1e37bb Merge pull request #7773 from roidayan/xio_remove_redundant_magic_methods
xio: remove redundant magic methods

Reviewed-by: Haomai Wang<haomai@xsky.com>
2016-02-24 21:49:21 +08:00
Jason Dillaman
5c5ca11b74 Merge pull request #7771 from xiexingguo/xxg-wip-fixawatch
librados: do not clear handle for aio_watch()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-02-24 08:35:06 -05:00
Roi Dayan
af12582b35 xio: remove redundant magic methods
The same methods are inherited from Messenger.

Signed-off-by: Roi Dayan <roid@mellanox.com>
2016-02-24 14:00:59 +02:00