Commit Graph

29130 Commits

Author SHA1 Message Date
Sage Weil
d1547d2223 Merge pull request #732 from ceph/wip-kvstore-tool
tools: ceph-kvstore-tool: Access & manipulate leveldb store using the KeyValueDB interface

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-16 13:35:07 -07:00
Joao Eduardo Luis
fd6e2b8618 ceph-kvstore-tool: copy one leveldb store to some other place
Iterates over the provided source store's keys and copies them to the
provided destination store.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-16 21:25:06 +01:00
Joao Eduardo Luis
85914b27e6 ceph-kvstore-tool: calc store crc
Reuse 'list()' function to traverse the store and calc not only version's
crcs, but also calc the store's crc.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-16 21:25:06 +01:00
Joao Eduardo Luis
da69fa09c8 tools: move 'test_store_tool' to 'tools/ceph-kvstore-tool'
ceph-kvstore-tool allows for lower-level access to leveldb stores.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-16 20:49:49 +01:00
Josh Durgin
64fa5cbe87 Merge pull request #729 from ceph/wip-5716
rgw: gracefully handle bad root pool names

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-10-15 17:09:10 -07:00
Sage Weil
1d4f501a01 test/filestore/run_seed_to.sh: avoid obsolete --filestore-xattr-use-omap
This option no longer exists.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-15 16:11:54 -07:00
Greg Farnum
982511e968 MonCommands: note that pg dump options don't work in plaintext
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-15 16:10:28 -07:00
Yehuda Sadeh
c7acc2aee2 rgw: gracefully handle bad root pool names
Fixes: #5716
When invalid root pools specified (don't start with a period) we return
with an error instead of asserting.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-15 16:05:14 -07:00
Sage Weil
488678f31a ceph_test_rados: fix snap remove vs rollback fix
In commit 55d279b985 we tried to fix a race
between rollback and snap removal, but got the logic wrong: we need to
prevent *snap removal* on in-use snaps, not prevent multiple rollbacks on
the same snap.

Fixes: #6254 (again)
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-15 15:50:16 -07:00
Sage Weil
2701231244 os/LevelDBStore: handle deletion race when checking store size
This fixes the fix in 64774e5792 which mixed
up the return value and errno.

Fixes: #6550
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-15 15:39:04 -07:00
Sage Weil
3b97b4166a Merge pull request #718 from ceph/wip-6553
rgw: fix authenticated users acl group check

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-15 15:04:41 -07:00
Yehuda Sadeh
8d7dbf8547 rgw: change default log level
Fixes: #6554
Backport: cuttlefish, dumpling
Default log level was just too high, bring it down a bit.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-15 12:32:58 -07:00
Sage Weil
87445ae9ef Merge pull request #720 from liewegas/wip-6521
mon/PGMonitor: set floor below which we do not warn about objects/pg

Reviewed-by: Greg Farnum <greg@inktank.com>
2013-10-15 12:16:26 -07:00
Sage Weil
70cc681326 mon/PGMonitor: set floor below which we do not warn about objects/pg
If a cluster has very few objects, do not generate warnings when the
objects/pg for a pool diverges from the cluster average.  This avoids
spurious errors when you have a relatively empty cluster and a lone pool
with a modest number of objects is too far off the (mostly meaningless)
cluster-wide average.

Also include a per-pool min so we ignore mostly-empty pools.

Fixes: #6521
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-15 11:22:16 -07:00
Yehuda Sadeh
bebbd6cb7b rgw: fix authenticated users acl group check
Fixes: #6553
Backport: bobtail, cuttlefish, dumpling
Authenticated users group acl bit was not working correctly. Check to
test whether user is anonymous was wrong.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-15 10:23:39 -07:00
Sage Weil
a0ffba6de2 Merge pull request #596 from ceph/wip-6147
mon: keep track of per-pool stats

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-14 17:06:48 -07:00
Joao Eduardo Luis
5abe5c273a mon: OSDMonitor: add 'osd pool stats' command
Fixes: #6147

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-15 00:51:35 +01:00
Joao Eduardo Luis
2cd5320019 mon: PGMap: rework client IO rate calc and output
Create a function so we can use the same format when outputting per-pool
stats

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-15 00:51:35 +01:00
Joao Eduardo Luis
e3ba8e82ac mon: PGMap: reuse existing summary functions to output pool stats
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-15 00:51:35 +01:00
Joao Eduardo Luis
82e3317d79 mon: PGMap: keep track of per-pool stats deltas
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-15 00:51:30 +01:00
Yehuda Sadeh
f2645e1c6d rgw: swift update obj metadata also add generic attrs
Fixes: #6462
We were missing the generic attributes when we updated the object
metadata (operation that only exists in the swift api).

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-14 11:26:39 -07:00
Yehuda Sadeh
fb96740651 Merge branch 'wip-6444' into next
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-10-11 15:44:00 -07:00
Yehuda Sadeh
a1825356ad librados: add some clarifying comments
about async pool operation handle

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-11 14:53:45 -07:00
Yehuda Sadeh
7ef5eb06ec librados: drop reference to completion in container destructor
Move the PoolAsyncCompletionImpl reference drop from
C_PoolAsync_Safe::finish() to ~C_PoolAsyncSafe(), as finish() is only
called when the async request is actually sent.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-10-11 14:53:25 -07:00
Sandon Van Ness
6ff9570726 Go back to $PWD in fsstress.sh if compiling from source.
Although fsstress was being called with a static path the directory
it was writing to was in the current directory so doing a cd to the
source directory that is made in /tmp and then removing it later
caused it to be unable to write the files in a non-existent dir.

This change gets the current path first and cd's back into it after
it is done compiling fsstress.

Issue #6479.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>
2013-10-08 12:05:11 -07:00
Greg Farnum
4b911cf817 ReplicatedPG: copy: use aggregate return code instead of individual Op return
It appears that the OSD is not filling in the individual return codes, and they
should be equivalent for all purposes we care about here (the only Op we are
doing is the copy-get, and if it fails we are getting its failure code).

Reported-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 13:50:30 -07:00
Sage Weil
6da4b91c07 os/FileStore: fix ENOENT error code for getattrs()
In commit dc0dfb9e01 the omap xattrs code
moved up a block and r was no longer local to the block.  Translate
ENOENT -> 0 to compensate.

Fix the same error in _rmattrs().

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 13:50:08 -07:00
João Eduardo Luís
02da1a1d79 Merge pull request #695 from ceph/wip-mon-auth
fix mon double-free when dropping unhandled messages, and allow "get monmap" messages to go through without authenticating for MonCliente::get_monmap_privately()

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-07 04:31:36 -07:00
Sage Weil
71ee6d7c8d mon: allow MMonGetMap without authentication
This is used by the MonClient::get_monmap_privately() helper.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 15:17:50 -07:00
Sage Weil
f279641894 mon: do not put() unhandle message
If we return false because we aren't handling a message, we should not put
the ref.  This fixes a double-free.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 15:17:50 -07:00
Sage Weil
92a60a0584 mon/PGMap: make generated test instances obey new invariant
As of 091809b814 we keep an osd_map epoch
for any osd_stat update, and assert as much.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:06:04 -07:00
Gary Lowell
7efbebe20f Merge branch 'next' 2013-10-05 02:11:36 +00:00
Sage Weil
e927941fca doc/release-notes: v0.67.4
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 16:49:41 -07:00
Gregory Farnum
c95d0a164f Merge pull request #676 from ceph/wip-start-copy
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-04 15:58:50 -07:00
David Zafman
5258c9c4e5 Merge pull request #667 from ceph/wip-6143
Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-04 14:59:11 -07:00
David Zafman
bb9b9c8953 common, os: Perform xattr handling based on detected fs type
In FileStore::_detect_fs() store discovered filesystem type in m_fs_type
Add per-filesystem filestore_max_inline_xattr_size_* variants
Add per-filesystem filestore_max_inline_xattrs_* variants
New function set_xattr_limits_via_conf()
  Set m_filestore_max_inline_xattr_size based on override or fs type
  Set m_filestore_max_inline_xattrs based on override or fs type
Handle conf change of any relevant value by calling set_xattr_limits_via_conf()
Change filestore_max_inline_xattr_size to override if non-zero
Change filestore_max_inline_xattrs to override if non-zero

Fixes: #6143

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-10-04 14:10:01 -07:00
athanatos
ce2d9ae617 Merge pull request #692 from ceph/wip-5992-2
Wip 5992 2

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-04 14:04:41 -07:00
Samuel Just
b87bc2311a ReplicatedPG: lock snapdir obc during write
Otherwise, we won't block properly in prep_push_backfill_object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
0c2769d332 PGLog: on split, leave log head alone
This way last_update doesn't go backwards.

Fixes: 6447
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
391a885f70 FileStore: make _setattrs not return -ENOENT most of the time
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
0c1e251351 ReplicatedPG: add debugging in recover_replicas for objects added for backfill
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
2ae9ece1a8 ReplicatedPG,PG: use PGBackend methods for getattr
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
8a919fb41d ReplicatedPG,PG: adapt collection_list* users to PGBackend interface
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
ff17e45fe2 PG,ReplicatedPG: expose PGBackend to PG
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
9a10a801f0 PG.cc: remove leading empty space
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
4df481c2da PGBackend,ReplicatedBackend: add interfaces for scanning the pg
This will be important since the erasure coded pg will have a different
on-disk format than the replicated backend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:06 -07:00
Samuel Just
664b589b05 ReplicatedPG: don't rescan the local collection if we can avoid it
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
e73ec48371 common/hobject: add is_degenerate method
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
c8a4411db1 PGMap: calc_min_last_epoch_clean() will now also use osd_epochs
We don't want to trim past the current osd map for any up osd.
osd_epochs provides a lower bound for that epoch for each osd.

Fixes: 5869
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
091809b814 PGMap,PGMonitor: maintain mapping of osd to recent stat epoch
Also, osd_stat will be empty for out osd.

When an osd is marked out, rather than remove it from osd_stat,
we instead 0 out the structure.

This patch also makes osd_stat_updates and osd_stat_rm private.
This should make it simpler to enforce invariants on these
mappings.

Each up osd will have a mapping since out osds are now included as
empty stats.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:11 -07:00