Commit Graph

29100 Commits

Author SHA1 Message Date
Sage Weil
aaabc65739 common/bloom_filter: methods for density, approx unique element counts
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:35 -07:00
Sage Weil
04091f87dd common/bloom_filter: remember original target size
This isn't strictly needed for core functionality, but it is convenient to
know.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:35 -07:00
Sage Weil
d07f5f3588 common/bloom_filter: drop raw_table_size_ member
We were storing table_size_ and raw_table_size_, where one is the size in
bits and the other is the size in bytes.  This is silly.  Store only the
size in bytes.

Also, bytes are always 8 bits, so use bit shifts and drop some of that
silliness too.

Move the member declarations to the top of the class so you read them
before the methods.

Fix some annoying whitespace.

Avoid allocating a 0-length array.

Mark the encoding incompatible with v1.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:18 -07:00
Sage Weil
ea2378e539 common/bloom_filter: make compressible_bloom_filter encodable
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:56:18 -07:00
Gregory Farnum
c95d0a164f Merge pull request #676 from ceph/wip-start-copy
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-04 15:58:50 -07:00
David Zafman
5258c9c4e5 Merge pull request #667 from ceph/wip-6143
Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-04 14:59:11 -07:00
David Zafman
bb9b9c8953 common, os: Perform xattr handling based on detected fs type
In FileStore::_detect_fs() store discovered filesystem type in m_fs_type
Add per-filesystem filestore_max_inline_xattr_size_* variants
Add per-filesystem filestore_max_inline_xattrs_* variants
New function set_xattr_limits_via_conf()
  Set m_filestore_max_inline_xattr_size based on override or fs type
  Set m_filestore_max_inline_xattrs based on override or fs type
Handle conf change of any relevant value by calling set_xattr_limits_via_conf()
Change filestore_max_inline_xattr_size to override if non-zero
Change filestore_max_inline_xattrs to override if non-zero

Fixes: #6143

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-10-04 14:10:01 -07:00
athanatos
ce2d9ae617 Merge pull request #692 from ceph/wip-5992-2
Wip 5992 2

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-04 14:04:41 -07:00
Samuel Just
b87bc2311a ReplicatedPG: lock snapdir obc during write
Otherwise, we won't block properly in prep_push_backfill_object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
0c2769d332 PGLog: on split, leave log head alone
This way last_update doesn't go backwards.

Fixes: 6447
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
391a885f70 FileStore: make _setattrs not return -ENOENT most of the time
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
0c1e251351 ReplicatedPG: add debugging in recover_replicas for objects added for backfill
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
2ae9ece1a8 ReplicatedPG,PG: use PGBackend methods for getattr
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
8a919fb41d ReplicatedPG,PG: adapt collection_list* users to PGBackend interface
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
ff17e45fe2 PG,ReplicatedPG: expose PGBackend to PG
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
9a10a801f0 PG.cc: remove leading empty space
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
4df481c2da PGBackend,ReplicatedBackend: add interfaces for scanning the pg
This will be important since the erasure coded pg will have a different
on-disk format than the replicated backend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:06 -07:00
Samuel Just
664b589b05 ReplicatedPG: don't rescan the local collection if we can avoid it
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
e73ec48371 common/hobject: add is_degenerate method
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
c8a4411db1 PGMap: calc_min_last_epoch_clean() will now also use osd_epochs
We don't want to trim past the current osd map for any up osd.
osd_epochs provides a lower bound for that epoch for each osd.

Fixes: 5869
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
091809b814 PGMap,PGMonitor: maintain mapping of osd to recent stat epoch
Also, osd_stat will be empty for out osd.

When an osd is marked out, rather than remove it from osd_stat,
we instead 0 out the structure.

This patch also makes osd_stat_updates and osd_stat_rm private.
This should make it simpler to enforce invariants on these
mappings.

Each up osd will have a mapping since out osds are now included as
empty stats.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:11 -07:00
Sage Weil
20043eba12 Merge pull request #690 from ceph/wip-fuse
a few ceph-fuse fixes

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-04 13:05:46 -07:00
Greg Farnum
806725a8b0 ReplicatedPG: copy: add op progression output
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:59:06 -07:00
Greg Farnum
639ff9f776 ReplicatedPG: copy: don't leak a ctx on failed copy ops
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:59:05 -07:00
Greg Farnum
469d471a8b ReplicatedPG: assert that we have succeeded in do_osd_ops on copyfrom repeats
Our callback is handling errors on its own at this point.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:59:05 -07:00
Greg Farnum
f3733a2052 ReplicatedPG: copy: switch CopyCallback to use a GenContext
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:58:58 -07:00
David Zafman
dc0dfb9e01 common,os: Remove filestore_xattr_use_omap option
Now we operate just like when this was set to true

Fixes: #6143

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-10-04 12:40:49 -07:00
athanatos
39b11455c9 Merge pull request #689 from ceph/wip-6254-model
ceph_test_rados: notice read error; do not race rollback with snap_delete

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-04 09:43:20 -07:00
athanatos
e95ca0ecab Merge pull request #688 from ceph/wip-6474
osd/ReplicatedPG: fix null deref on rollback_to whiteout check

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-04 09:43:00 -07:00
Sage Weil
55d279b985 ceph_test_rados: do not let rollback race with snap delete
Note that the OSD behaves in a weird way when you rollback to a non-
existent snap, so the test probably isn't the only party at fault here.

Fixes (test half of): #6254
Backport: dumpling, cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-03 21:47:26 -07:00
Sage Weil
b09a1ef946 ceph_test_rados: stop on read error!
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-03 21:46:36 -07:00
Sage Weil
03ba7408db osd/ReplicatedPG: fix null deref on rollback_to whiteout check
Bring this whole if/else chain up one level so that we can capture both
ENOENT and whiteout in the same case.  (And don't dereference the
pointer when we know it is NULL.)

Fixes: #6474
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-03 21:27:36 -07:00
Sage Weil
6afb16e52b Merge remote-tracking branch 'gh/next'
Conflicts:
	src/Makefile.am
2013-10-03 21:23:05 -07:00
Josh Durgin
aacd67e071 PendingReleaseNotes: fix typo
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-10-03 20:07:13 -07:00
Sage Weil
e9e64545d5 Merge pull request #557 from ceph/wip-5896
mon: MonmapMonitor: make 'ceph mon add' idempotent

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-03 18:27:58 -07:00
Sage Weil
fea1e0e56c PendingReleaseNotes: make a note about K vs k
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-03 17:05:41 -07:00
Sage Weil
8fb04d2680 Merge pull request #686 from ceph/wip-rbd-cli-tests
qa: fix rbd cli tests checking size

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-03 17:03:13 -07:00
Josh Durgin
10335883a9 qa: fix rbd cli tests checking size
b43bc1a0b0 changed the kilo prefix
from K to k in a few places.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-10-03 15:16:59 -07:00
Sage Weil
86e96578be Merge pull request #684 from git-harry/ceph_disk_lowercase_fsid
Make fsid comparison case-insensitive

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-03 14:50:25 -07:00
John Wilkins
c19935cd09 doc: Fixed typo.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-10-03 13:30:18 -07:00
Dan Mick
9ab05312a6 Merge pull request #685 from ceph/wip-rados-ints
librados: drop #include of int_types.h from installed headers

Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-10-03 12:19:50 -07:00
Sage Weil
afb4d83d0c librados: drop #include of int_types.h from installed headers
These are unnecessary, and breaks compilation for outside users.

Prefer inttypes.h over stdint.h.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-03 12:16:55 -07:00
Loic Dachary
739696ef0a Merge pull request #638 from ceph/wip-bloom
bloom filter cleanups, encodability, and unit tests
2013-10-03 12:13:41 -07:00
Sage Weil
de788bd687 Merge pull request #653 from ceph/wip-mon-auth
mon: Monitor: dissociate msg handling from session and connection logic

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-03 11:32:50 -07:00
Joao Eduardo Luis
f1e23937a6 mon: Monitor: reuse 'src_is_mon' bool on dispatch
Instead of making the same comparison twice with no gain at all.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-03 19:29:45 +01:00
Joao Eduardo Luis
29cf2ff02a mon: Monitor: only handle paxos msgs from a mon && if session has exec caps
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-03 19:29:45 +01:00
Joao Eduardo Luis
b8a148804d mon: Monitor: dissociate msg handling from session & connection logic
Makes for simpler logic for patches to come.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-03 19:29:45 +01:00
Joao Eduardo Luis
d0d61b488a mon: Monitor: drop client msg if no session exists and msg is not MAuth
If we are not a monitor and we don't have a session yet, we must first
authenticate with the cluster.  Therefore, the first message to the
monitor must be an MAuth.  If not, we assume it's a stray message and
just drop it.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-03 19:29:45 +01:00
Joao Eduardo Luis
ed1a54ecd3 mon: Monitor: assert on absense of connection during dispatch
We expect to always have a connection associated with a message.
If that happens to not be so, assert.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-03 19:27:19 +01:00
Joao Eduardo Luis
dce3d26d84 mon: MonmapMonitor: make 'ceph mon add' idempotent
MonMap changes lead to bootstraps.  Callbacks waiting for a proposal to
finish can have several fates, depending on what happens: finished, rerun
or aborted.

In the case of a bootstrap right after a monmap change, callbacks are
rerun.  Considering we queued the message that lead to the monmap change
on this queue, if we instead of finishing it end up reruning it, we will
end up trying to perform the same modification twice -- the last one will
try to modify an already existing state and we will return just that:
whatever you're attempting to do has already been done.

This patch makes 'ceph mon add' completely idempotent.  If one tries to
add an already existing monitor (i.e., same name, same ip:port), one
simply gets a 'monitor foo added', with return 0, no matter how many
times one runs the command.

Fixes: #5896

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-03 19:07:12 +01:00