Commit Graph

30473 Commits

Author SHA1 Message Date
Sage Weil
923bff1dba osd/ReplicatedPG: drop RepGather::ondone callback
We kick the blocked contexts in the completion path of process_copy_chunk(),
after we have take the RWWRITE obc lock.  There is no need to delay the
unblocking until the RepGather finishes.

This also fixes a leak: the ondone wasn't getting cleaned up if a peering
interval change happens and the repgather is applied early in on_change().

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:01 -08:00
Sage Weil
2a9c6fcda9 vstart.sh: go faster
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:01 -08:00
Sage Weil
bc893f55d8 osd/ReplicatedPG: fix undirty on clean object
Return success, but do not screw up the stats.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:01 -08:00
Sage Weil
c561d5ea22 osd/ReplicatedPG: track dirty, whiteout stat counts
These counts will be useful (even necessary!) for the cache agent, and are
generally interesting to the admin as well.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:01 -08:00
Sage Weil
028bb0d1e2 osd/osd_types: include num_objects_dirty, num_whiteouts in object_stat_sum_t
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:01 -08:00
Sage Weil
1403798327 osd/ReplicatedPG: EBUSY on cache-evict when watchers are present
Linger operations will follow the object to the cache pool when the pool
overlay process is set.  If we evict the object, the object_info_t will
go away along with the watch state and confusing things will happen.
Prevent that from happening by returning EBUSY when you try to evict a
watched object.

Note that you *can* flush a watched object, and the dirty flag will be
cleared.  But you still can't evict it.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:01 -08:00
Sage Weil
9ed6679a94 ceph_test_rados: test cache_flush, cache_try_flush, cache_evict
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:00 -08:00
Sage Weil
1dcbb663d8 ceph_test_rados_api_tier: fix HitSet* test names
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:00 -08:00
Sage Weil
99cee55c8a osd/osd_types: debug: include size in object_info_t operator<<
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:00 -08:00
Sage Weil
1af6723cdc osd/ReplicatedPG: debug: clean up oi printout
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:00 -08:00
Sage Weil
0d2d6a5faa osd/ReplicatedPG: debug: add an assert for copy-get
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:00 -08:00
Sage Weil
2fda4c0161 osd/ReplicatedPG: fix locking for promote
After we get the copy-from data and unblock the obc, we still need to take
the RWWRITE lock on the object for the duration of the repop while we
actually apply the change locally.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:40:00 -08:00
Sage Weil
927b0e60c2 osd/ReplicatedPG: fix user_version preservation for copy_from
In the process of fixing this for flush, we break promote, so we need to
adjust them both here.  Basic strategy: do not set user_modify, but handle
the user_version explicitly in the callbacks.

For copy_from, we don't have a clean way to pass the result through to
finish_copyfrom in do_osd_ops; do so by putting it in user_at_version. (If
we were to call finish_copyfrom directly from the callback this might
be simpler, but let's not go there right now.)

For promote, it is a trivial fix.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:59 -08:00
Sage Weil
bc05104149 osd/ReplicatedPG: handle ECANCELED in C_CopyFrom, C_Flush
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:59 -08:00
Sage Weil
2d5a7e2cad osd/ReplicatedPG: uninline CopyFromCallback, PromoteCallback
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:59 -08:00
Sage Weil
54f0c60c8b osd/osd_types: make object_info_t::dump() dump user_version
Backport: emperor
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:59 -08:00
Sage Weil
ba2f9e2996 osd/osd_types: include user_version in operator<< object_info_t
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:59 -08:00
Sage Weil
ffdaa5f415 vstart.sh: --cache <pool> to set up pool cache(s) on startup
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:59 -08:00
Sage Weil
57e91455be qa/workunits/rados/test_cache_pool.sh: fixes
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:58 -08:00
Sage Weil
1bde88f87c qa/workunits/rados: rename cache pool tests
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:58 -08:00
Sage Weil
ea519b48c0 qa/workunits/rados: test cache-{flush,evict,flush-evict-all}
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:58 -08:00
Sage Weil
71cd4a2278 rados: add cache-flush, cache-evict, cache-flush-evict-all commands
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:58 -08:00
Sage Weil
ad3b46665f osd/ReplicatedPG: implement cache-flush, cache-try-flush
Implement a rados operation that will flush a dirty object in the cache
tier by writing it back to the base tier.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:58 -08:00
David Zafman
edaec9a836 osd: Fix assert which doesn't apply when compat_mode on
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-19 14:37:28 -08:00
Sage Weil
ac16a9dfe0 osd: remove remaining instances of raid4 pool types (never implemented)
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 13:28:09 -08:00
Sage Weil
30876e69f4 Merge pull request #941 from ceph/wip-6028
#6028 : ensure that erasure coded pools don't work until the osds can handle it

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-19 10:17:15 -08:00
Sage Weil
0d2d937e2d Merge pull request #973 from ceph/wip-mds-coverity2
mds: fixes for coverity scan

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-19 08:14:20 -08:00
Alexandre Oliva
40a48de697 mds: fix Resetter locking
ceph-mds --reset-journal didn't work; it would deadlock waiting for
the osdmap.  Comparing the init code in the Dumper (that worked) with
that in the Resetter (that didn't), I noticed the lock had to be
released before waiting for the osdmap.

Now the resetter works.  However, both the resetter and the dumper
fail an assertion after they've performed their task; I didn't look
into it:

../../src/msg/SimpleMessenger.cc: In function 'void SimpleMessenger::reaper()' t
hread 7fdc188d27c0 time 2013-12-19 04:48:16.930895
../../src/msg/SimpleMessenger.cc: 230: FAILED assert(!cleared)
 ceph version 0.72.1-6-g6bca44e (6bca44ec129d11f1c4f38357db8ae435616f2c7c)
 1: (SimpleMessenger::reaper()+0x706) [0x880da6]
 2: (SimpleMessenger::wait()+0x36f) [0x88180f]
 3: (Resetter::reset()+0x714) [0x56e664]
 4: (main()+0x1359) [0x562769]
 5: (__libc_start_main()+0xf5) [0x3632e21b45]
 6: /l/tmp/build/ceph/build/src/ceph-mds() [0x564e49]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to int
erpret this.
2013-12-19 04:48:16.934093 7fdc188d27c0 -1 ../../src/msg/SimpleMessenger.cc: In
function 'void SimpleMessenger::reaper()' thread 7fdc188d27c0 time 2013-12-19 04
:48:16.930895
../../src/msg/SimpleMessenger.cc: 230: FAILED assert(!cleared)

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
2013-12-19 08:09:46 -08:00
Sage Weil
2cf9fa94ca Merge pull request #969 from yuyuyu101/wip-7040
Fix segmentation fault when handler is NULL pointer

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-19 08:05:32 -08:00
Loic Dachary
eb12c81f4e Merge pull request #972 from dachary/wip-build-depends
packaging: revert adding argparse and uuidgen
2013-12-19 06:39:55 -08:00
Loic Dachary
087fe57f0e packaging: revert adding argparse and uuidgen
Because the gitbuilder build environment does not read the build depends
from the packages and needs to be updated independantly.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-19 15:33:07 +01:00
Loic Dachary
4f0918afa7 Merge pull request #971 from dachary/wip-build-depends
packaging: make check needs argparse and uuidgen

Reviewed-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-12-19 06:27:09 -08:00
Loic Dachary
812e59b278 Merge pull request #970 from dachary/wip-autogen
autogen: test compare strings with != not -ne

Reviewed-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-12-19 05:57:52 -08:00
Loic Dachary
827253851e packaging: make check needs argparse and uuidgen
make check runs vstart.sh to setup a cluster from source and it misses
the python-argparse library as well as uuidgen otherwise it will fail
with:

    ./vstart.sh: 460: uuidgen: not found
    ...
    Error: Import No module named argparse

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-19 13:57:04 +01:00
Loic Dachary
f193925aa5 autogen: test compare strings with != not -ne
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-19 13:40:06 +01:00
Haomai Wang
98af37d091 Fix segmentation fault when handler is NULL pointer
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2013-12-19 18:54:04 +08:00
Loic Dachary
b2a80004e6 Merge pull request #966 from dachary/ceph-master
crush: silence error messages in unit tests

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:31:41 -08:00
Loic Dachary
27f4d1f6bc Merge pull request #968 from ceph/wip-crush
make check is happy and the change is non controversial

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-12-19 02:28:30 -08:00
Yan, Zheng
44aacaed51 mds: fixes for coverity scan
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-12-19 14:27:21 +08:00
Sage Weil
8b38f10bc2 crush/mapper: fix crush_choose_firstn comment
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-18 21:22:24 -08:00
Sage Weil
ea3a0bb8b7 crush/mapper: attempts -> tries
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-18 21:22:24 -08:00
Sage Weil
0497db49e5 crush/mapper: finish adding choose_local_[fallback_]tries
Didn't finish this in d129e09e57fbc61cfd4f492e3ee77d0750c9d292.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-18 21:22:24 -08:00
Sage Weil
99f41de3be vstart.sh: NOTE, not WARNING, to make gitbuilder happy
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-18 21:22:08 -08:00
Joao Eduardo Luis
b3ee598f50 qa: workunit: cephtool: test osd pool create with erasure type
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:21:05 +00:00
Joao Eduardo Luis
250b44627e pybind: test_ceph_argparse: test 'ceph osd pool create' with pool type
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:21:05 +00:00
Joao Eduardo Luis
c996f66986 mon: OSDMonitor: add optional 'pool type' arg to 'osd pool create'
Allow specifying 'rep', 'raid4' and 'erasure'.
Only allow setting type 'erasure' if all up osds support erasure codes.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:21:05 +00:00
Joao Eduardo Luis
bdeaa84126 osd: OSDMap: add 'get_up_osds()' function
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:21:05 +00:00
Joao Eduardo Luis
5756c05a22 ceph_osd: add EC to OSD's supported features used by the messenger
Fixes: 6028

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:20:01 +00:00
Joao Eduardo Luis
59ad6da43a osd: OSD: reflect OSDMap EC flag being set by setting on-disk feature
If OSDMap has the EC feature set, then update our superblock to
reflect as such, making our on-disk format incompatible with previous
OSDs without EC support.

Fixes: 6028

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:19:52 +00:00
Joao Eduardo Luis
8ac84db050 mon: OSDMonitor: handle osd features on boot
Add the osd's features to the osd's extra info field in the OSDMap
so we can track which OSDs are able to deal with Erasure Codes.

This will allow us to decide whether or not we are ready to set EC
whenever the user asks us to set EC on a pool -- which shall be
handled by a subsequent commit.

Fixes: 6028

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-19 02:19:47 +00:00