Commit Graph

50439 Commits

Author SHA1 Message Date
Casey Bodley
0dd061c3ed Merge pull request #7748 from ceph/wip-cmake-2-22-16
cmake: fix paths to various EC source files
2016-02-25 14:06:16 -05:00
Ali Maredia
83b4aab3a8 cmake: installing _tp libraries into lib
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-02-25 14:02:11 -05:00
Samuel Just
430027f70b PGLog: factor out claim_log_entries_update_missing
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
7eadf88b73 PGLog::merge_log: update missing, after removing divergent
This should not change behavior at all.  It'll be useful in
the next few commits.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
50e4e2c7ab PG,PGLog: convert to DoutPrefixProviders
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
b33dd14ec9 dout.h: add a DoutPrefixProvider and macro to allow passing of prefixes
This way, we can get proper logging within static functions.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
d3eac8fe6e ECBackend: use CEPH_MSG_PRIO_HIGH for write replies
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
b1151c4b8b ECBackend: send subop writes and reads at normal priority
We want these to be processed ahead of new client ops since
there are resources being held.

Fixes: 14313
Backport: hammer
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
257225d301 ReplicatedPG: remove OpContext from Repop
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
43f3d82946 ReplicatedPG: s/release_op_ctx_locks/release_object_locks, don't use opcontext
This gets us one step closer to removing OpContext from Repop.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
e68b5507ec ReplicatedPG: refactor OpContext to use ObcLockManager
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
82896f2c6e ReplicatedPG::close_op_ctx: the err argument is unused
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
7ae52fcfa2 osd_types: add ObcLockManager
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
0a5b4c1d93 src/osd: use unique_ptr for backend trasaction, move into submit_transaction
submit_transaction takes ownership of the transaction implicitely.  Make
this implicit using an rvalue ref to force usage of std::move and move
constructor.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
612fd3341a ReplicatedPG::RepGather: remove on_finish, no more users
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:41 -08:00
Samuel Just
1ef04ddfaa ReplicatedPG: use register_on_finish rather than on_finish for flush/evict
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
8e82deaf51 ReplicatedPG::scrub: use register_on_commit rather than on_finish
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
90a5359b9c ReplicatedPG: move do_osd_op_effects out of eval_repop
This is better anyway, do_osd_op_effects on a non-client op was somewhat
silly without an op (even if you filled in the other fields like
watch_connects, it would just silently skip it).  It also eliminates
another dependence of RepGather on OpContext.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
e4c5356006 ReplicatedPG: break out complete_disconnect_watches helper
We'll need this shortly when we remove do_osd_op_effects from
eval_repop.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
cc1b2c6f34 ReplicatedPG: move client reply handling out of eval_repop
execute_ctx is the only path which actually has a client op,
let's clarify eval_repop by moving that logic inline there.
This also eliminates most of the repop->ctx users from
eval_repop.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
5110214337 osd/: replace simple_repop_.* with simple_opc_.*
This way, we don't expose the RepGather structure
up into the users (which pretty much exclusively
use the repop->ctx member anyway).  This will pave
the way to removing RepGather::ctx.

Part of this involves generalizing the repop callback
members (queue_snap_trimmer and on_applied) to
on_applied, on_committed, on_success, and on_finish
on both OpContext and RepGather.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Samuel Just
6db7fe7de3 introduce CEPH_FEATURE_SERVER_JEWEL feature bit
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-25 10:56:40 -08:00
Ali Maredia
2e4ad9ea7d cmake: Removed _tp libraries from linking
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-02-25 13:48:26 -05:00
Ken Dreyer
dc54a85664 debian: fix logrotate conf handling
Fixes a failure to build from source; fallout from the server split
change (0cbe3dea69)

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2016-02-25 10:44:18 -07:00
Ken Dreyer
9a66bf00b2 debian: put libexec files in /usr/lib
SUSE does not allow the use of /usr/libexec, so commit
69291f872e changed the RPM
packaging to put the libexec files into /usr/lib instead. Do the same
thing on Debian.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2016-02-25 10:44:17 -07:00
Radoslaw Zarzynski
18da92f32b rgw: add support for caching of Keystone admin token.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-25 18:34:16 +01:00
Erwan Velu
c2e6102955 test/ceph_objectstore_tool: Deleting btrfs subvolumes before exiting
When running the ceph_objectstore_tool test suite, we shall delete the btrfs
subvolumes that have been created before trying deleting the content unless that will generate a permission denied
issue.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2016-02-25 18:14:32 +01:00
Erwan Velu
9ae89269f8 ceph-helpers: Delete remaining subvolumes on destroy_osd()
Before deleting $dir/$id, we have to delete the associated subvolumes unless the
rm will fail.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2016-02-25 16:46:49 +01:00
Jason Dillaman
0c44f20a30 Merge pull request #7793 from tchaikov/wip-fix-dencoder-MirrorImage
cls_rbd: fix the test for ceph-dencoder

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-02-25 10:33:19 -05:00
Jianjian Huo
9dbac6c3b3 os/kstore: fix a race condition in _txc_finish()
{} was missed, which limits the scope of the lock_guard.
osr->qlock has to be unlocked before calling _osr_reap_done(), since _osr_reap_done
needs acquire osr->qlock in the first beginning.

Signed-off-by: Jianjian Huo <samuel.huo@gmail.com>
2016-02-25 07:24:16 -08:00
Orit Wasserman
e23fdcbbce rgw: reset return code in when iterating over the bucket the objects
Fixes: #14826

Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2016-02-25 16:11:15 +01:00
Kefu Chai
a704270287 cls_rbd: fix the test for ceph-dencoder
initialize MirrorImage::state to a certain value, otherwise the
generated test instances would be undetermined, and hence fail
the dencoder tests.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 23:07:49 +08:00
Loic Dachary
7131c977f0 Merge pull request #7792 from tchaikov/wip-13505
fix FTBFS introduced by d0af316

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-25 22:04:26 +07:00
Erwan Velu
cd0389ddb1 ceph-helpers.sh: Deleting forgoten btrfs subvolumes
While running a make check on a btrfs system, many subvolumes are let at the end
of the build. It's pretty common to have several hundreds of those.

btrfs is pretty sensible to the path when requesting a subvolume removal.
The current code was misleading the path and didn't deleted the remaining
volumes.

This patch list the current subvolumes, filter thoses created by the
test process and ajust the path because brtfs reports
  erwan/chroot/ceph/src/testdir/test-7202/dev/osd1/snap_439
while regarding the current working directory we want to delete :
  testdir/test-7202/dev/osd1/snap_439

Signed-off-by: Erwan Velu <erwan@redhat.com>
2016-02-25 15:33:10 +01:00
Piotr Dałek
ba70bdb303 config: increase default async op threads
Default value of two async threads is enough for light workloads,
but is insufficient for heavy ones, reducing maximum throughput on
slower CPUs that have plenty of logical cores. Three async threads
give noticeable perf jump without any noticeable downsides.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-02-25 15:12:52 +01:00
Loic Dachary
61ab9e8e70 Merge pull request #7731 from SUSE/wip-shebang
ceph_detect_init/__init__.py: remove shebang

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-25 21:03:21 +07:00
Kefu Chai
557c3bfe0d osd: always cleanup the scrub results
the destructor of Scrub::Store asserts that `this->results` should
always be empty when it is free'd. before this change, if we are
repairing, the results is not flushed or cleaned up, this crashes
osd daemon when a new PG internval comes. also we should not keep
the results around even we don't want to populate them to the omap
if we are repairing, it's just a waste.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 21:47:30 +08:00
Kefu Chai
b0b402193d pybind: fix the FTBFS introduced by d0af316
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 21:47:30 +08:00
Kefu Chai
67aa66ee9c Merge pull request #7734 from dachary/wip-14834-pidfile-test
tests: test_pidfile.sh lingering processes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-25 16:19:08 +08:00
Kefu Chai
ac7d11ad73 librados: add missing tracpoint for scrub APIs
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-24 22:39:57 -08:00
Loic Dachary
fe5aff3bfd Merge pull request #7637 from SUSE/wip-14756
Refrain from versioning and packaging EC testing plugins

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2016-02-25 12:41:19 +07:00
Loic Dachary
c1e1c8d18e tests: test_pidfile.sh lingering processes
For the mon, using ps aux to find the process will only show part of the
command line and the $RUNID may be missing. For the OSD, the $RUNID is
passed using an environment variable and will not show at all.

There is no need to test both ceph-osd and ceph-mon since they both end
up in the same code path, pidfile_write. Add a log line that only shows
when the --pid-file is empty and grep for it in the simplest possible
way.

The TEST_pidfile test failed to kill a process because there was
no pidfile with its pid. This is fixed by backuping the pidfile before
it is modified.

http://tracker.ceph.com/issues/14834 Fixes: #14834

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-25 12:24:00 +07:00
Kefu Chai
ce1169d245 Merge pull request #6898 from tchaikov/wip-13505
add scrub persist/query API

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-02-25 12:48:14 +08:00
Kefu Chai
e2374c43c9 rados: add "list-inconsistent-snapset" cmd
to list inconsistent snapsets of a given PG, this command exposes
get_inconsistent_snapsets() rados API to user.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
8018eab344 rados: add "list-inconsistent-obj" cmd
to list inconsistent objects of a given PG, this command exposes
get_inconsistent_objects() rados API to user.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
c9b593d2d7 librados: add get_inconsistent_snapsets() API
Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
dfc2f4823b librados: add get_inconsistent_objects() API
Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
3dea4f1f5c osd: add CEPH_OSD_OP_SCRUBLS pg op
it is a new pg op which returns the encoded objects stored when
scrubbing.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
2009ed274d osd: persist inconsistent snapsets using omap
the inconsistent snapsets are identified in ReplicatedPG::_scrub()
after we compared the authorized objects with their replica/shards.
these inconsistent information are stored in the omap of objects
with prefix "SCRUB_SS_".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00
Kefu Chai
c828c398b1 librados: add inconsistent_snapset_t type
for presenting the inconsistent snapsets found in scrub

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-25 12:41:55 +08:00