Commit Graph

22596 Commits

Author SHA1 Message Date
Josh Durgin
8c961610a6 ObjectCacher: don't needlessly increment iterator
This iterator is now reset on each run through the loop,
so there's no point in incrementing it here.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
b948e4c91f ObjectCacher: retry reads when they are incomplete
Skipping these callbacks when there's a racing write or
a gap in the results causes the original reads they represent
to never be completed. If the read falls within the range
of a BufferHead, retry all waiters no matter what.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:27 -08:00
Yehuda Sadeh
29a96cf29c rgw: ops log can also go to socket
Adding a new ops log output (into a unix domain socket).
Configuration:
  rgw_enable_usage_log : master switch for ops log
  rgw ops log socket path : set socket path
  rgw ops log rados : whether ops should be logged in the rados
                      cluster
  rgw ops log data backlog : max size in MB to be accumulated
                             without flushing

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-16 15:17:20 -08:00
John Wilkins
7b96807dfc doc: Added links to APIs.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 15:13:57 -08:00
Greg Farnum
feb8451f05 doc: don't use 'pool=data' as an example
Signed-off-by: Greg Farnum <greg@inktank.com>
2012-11-16 14:28:51 -08:00
John Wilkins
c42af8030f doc: Added documentation for using CPU profiler.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:58:22 -08:00
John Wilkins
84eac86696 doc: Added documentation for installing CPU profiler.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:57:55 -08:00
John Wilkins
3ccaaa0923 doc: Added librbd to the RBD toc.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:57:15 -08:00
John Wilkins
9ea2609ea0 doc: Added libcephfs to the CephFS section and moved librbd to RBD section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:56:46 -08:00
Sage Weil
8da6ddeaec common/ceph_argparse: fix malloc failure check
CID 743418 (#1 of 1): Dereference before null check (REVERSE_INULL)
Null-checking "argv" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:19:25 -08:00
Sage Weil
e82ca0d5e0 mon/MonClient: initialize ptr in ctor
CID 743433 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
At (2): Non-static class member "authorize_handler_registry" is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:18:21 -08:00
Sage Weil
8f1f36d5a2 os/FileStore: fix fd leak in _rmattr
CID 743405 (#2 of 2): Resource leak (RESOURCE_LEAK)
At (16): Handle variable "fd" going out of scope leaks the handle.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:15:23 -08:00
Sage Weil
426b58da29 os/FileStore: fix fd leaks in _setattrs
CID 743406 (#3 of 3): Resource leak (RESOURCE_LEAK)
At (26): Handle variable "fd" going out of scope leaks the handle.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:14:17 -08:00
Sage Weil
1df38fd73d osdc/ObjectCacher: faux use-after-free
CID 743435 (#1 of 1): Use after free (USE_AFTER_FREE)
At (68): Passing freed pointer "rd" as an argument to function "std::basic_ostream<char, std::char_traits<char> >::operator <<(void const *)".

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:11:05 -08:00
Josh Durgin
9a10ebb2ef test: add ObjectCacher stress test that does not use a cluster
Use a fake writeback handler and respond to all requests with -ENOENT.
This tests that all operations will complete, and the cache doesn't
lose waiters or callbacks.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 13:23:26 -08:00
Josh Durgin
fd928b9b71 ObjectCacher: more debugging for BufferHeads
This is useful for checking for lost waiters.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 12:11:01 -08:00
Gary Lowell
8b187bd8ca build: update for boost_thread library.
There is a difference in naming conventions between debian and
rpm based distributions for this library.  In configure.ac we
check first for boost_thread-mt, then if it's not found check
for boost_thread.  A side effect of the AC_CEHCK_LIB macro is
to add the library to the $LIBS, so the explicit -llibboost_thread
in the Makefile has been removed.
(cherry picked from commit f0c7bb3630)
2012-11-16 10:51:42 -08:00
Joao Eduardo Luis
493049b4b6 mon: OSDMonitor: clarify some command replies
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-16 15:30:53 +00:00
Joao Eduardo Luis
0b28ef6a15 mon: OSDMonitor: fix spacing when outputting items on command reply
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-16 15:30:24 +00:00
Gary Lowell
27830e2c13 Merge branch 'master' of https://github.com/ceph/ceph 2012-11-16 00:56:36 -08:00
Gary Lowell
f0c7bb3630 build: update for boost_thread library.
There is a difference in naming conventions between debian and
rpm based distributions for this library.  In configure.ac we
check first for boost_thread-mt, then if it's not found check
for boost_thread.  A side effect of the AC_CEHCK_LIB macro is
to add the library to the $LIBS, so the explicit -llibboost_thread
in the Makefile has been removed.
2012-11-16 00:46:41 -08:00
Josh Durgin
c707568a1b doc: fix crush set syntax
No commas between parameters, pool is required.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-15 23:07:51 -08:00
Sage Weil
71cfaf1cc5 os/FileStore: only try BTRFS_IOC_SUBVOL_CREATE on btrfs
Only try to create a btrfs subvolume if the fs is btrfs.  Otherwise, just
create a directory.  Then we can error out on *any* ioctl error, and not
rely on the ioctl error code to determine if we failed because we are on
a non-btrfs or a real error.

Fixes: #3052
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2012-11-15 16:50:55 -08:00
Sage Weil
3ca947e018 mon: clean up 'ceph osd ...' list output
No more 'osd.0 is already inosd.1 is already in' crap.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 16:35:53 -08:00
Sage Weil
344c4fdc3f mon: correctly identify crush names
get_item_id() returns 0 if the name already exists; that's not what we
want here.  Verify the name exists before checking its id.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 16:32:50 -08:00
Sage Weil
592a89421f mon: use parse_osd_id() throughout
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 16:32:50 -08:00
Samuel Just
918c58c85e PrioritizedQueue: remove internal lock, not used
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-11-15 16:02:34 -08:00
Samuel Just
b53e06cac8 DispatchQueue: lock DispatchQueue when for get_queue_len()
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-11-15 15:59:18 -08:00
Alex Elder
659d4c25b2 run_xfstests.sh: activate more tests that now work
I've gone through the set of xfstests that were previously found to
not work.  Some of those now do work, and with the addition of an
option to pass to "mkfs.xfs" a large number of other tests now
produce expected output as well.

This patch updates the default list of tests to run to reflect
the result of this exercise.  The following 50 additional tests
are now run by default:

    029 074 078 084-087 100 105 117 121 124 126 129-134
    164 165 167 174 181 184 186 187 192 214-216 227 236
    237 241 243 245-249 257-259 261 277 278 280 285 286

Test 127 completed without error, but it took from 1-3 hours so I
kept that out of the list.

Signed-off-by: Alex Elder <elder@inktank.com>
2012-11-15 17:51:34 -06:00
Noah Watkins
22cb13565c doc: add Hadoop configuration parameters
fs.ceph.block.size and fs.ceph.replication.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2012-11-15 14:03:36 -08:00
Sage Weil
2fd9f4d74f Merge remote-tracking branch 'gh/next' 2012-11-15 11:36:20 -08:00
Samuel Just
63c512888b smalliobench: fix init-only, add don-not-init
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Mike Ryan <mike.ryan@inktank.com>
2012-11-15 10:55:05 -08:00
Sage Weil
b40387de23 msg/Pipe: fix leak of Authorizer
Reported-by: Joao Luis <joao.luis@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 10:06:07 -08:00
Sage Weil
0fb23cf8ce Merge remote-tracking branch 'gh/wip-3477' into next
Reviewed-by: Greg Farnum <greg@inktank.com>
2012-11-15 09:48:25 -08:00
Samuel Just
12c2b7fa20 msg/DispatchQueue: release throttle on messages when dropping an id
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-11-14 17:05:58 -08:00
Samuel Just
5f214b2938 PrioritizedQueue: allow remove_by_class to return removed items
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-11-14 17:05:55 -08:00
Sage Weil
98b93b5d3d librbd: use delete[] properly
==4986== Mismatched free() / delete / delete []
==4986==    at 0x4C2658C: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4986==    by 0x4ED8EA9: librbd::ImageCtx::~ImageCtx() (ImageCtx.cc:100)
==4986==    by 0x4EF3827: librbd::close_image(librbd::ImageCtx*) (internal.cc:1869)
==4986==    by 0x4EE8FB8: librbd::clone(librados::IoCtx&, char const*, char const*, librados::IoCtx&, char const*, unsigned long, int*, unsigned long, int) (internal.cc:900)
==4986==    by 0x4EC363C: rbd_clone2 (librbd.cc:553)
==4986==    by 0x404C85: do_clone (fsx.c:836)
==4986==    by 0x405639: test (fsx.c:1048)
==4986==    by 0x406369: main (fsx.c:1523)
==4986==  Address 0xd498b30 is 0 bytes inside a block of size 37 alloc'd
==4986==    at 0x4C26CF7: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4986==    by 0x4ED9B4D: librbd::ImageCtx::init_layout() (ImageCtx.cc:164)
==4986==    by 0x4ED9845: librbd::ImageCtx::init() (ImageCtx.cc:142)
==4986==    by 0x4EF3449: librbd::open_image(librbd::ImageCtx*, bool) (internal.cc:1828)
==4986==    by 0x4EE89E0: librbd::clone(librados::IoCtx&, char const*, char const*, librados::IoCtx&, char const*, unsigned long, int*, unsigned long, int) (internal.cc:871)
==4986==    by 0x4EC363C: rbd_clone2 (librbd.cc:553)
==4986==    by 0x404C85: do_clone (fsx.c:836)
==4986==    by 0x405639: test (fsx.c:1048)
==4986==    by 0x406369: main (fsx.c:1523)

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-14 17:05:39 -08:00
Sage Weil
4a7a81bb53 objecter: fix leak of out_handlers
The error paths don't use the handlers.  Make sure they get cleaned up.

Fixes: #3446
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-14 17:05:39 -08:00
Sage Weil
ef4e4c8287 mon: calculate failed_since relative to message receive time
Instead of looking at the current time we process the message, look at the
receive time.  This gives us a more real failure time given that messages
may be requeued.

It doesn't solve the problem when messages are forwarded between monitors
due to an election, but that's ok; this is still a net improvement.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-14 17:00:57 -08:00
Yehuda Sadeh
9267d8a42f rgw: update post policy parser
json parser semantics changed a little bit, so
needed to update the policy parser.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-14 16:45:56 -08:00
Sage Weil
f6cb0780ac mon: set default port when binding to random local ip
Fixes #3135
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-14 16:26:58 -08:00
Sage Weil
dfeb8ded6a Merge remote-tracking branch 'gh/wip-asok' into next
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-14 16:22:27 -08:00
Yehuda Sadeh
ce28455206 rgw: relax date format check
Don't try to parse beyond the GMT or UTC. Some clients use
special date formatting. If we end up misparsing the date
it'll fail in the authorization, so don't need to be too
restrictive.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-14 15:51:36 -08:00
John Wilkins
0a2a0c075c doc: config-cluser move to new IA.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-14 14:57:51 -08:00
John Wilkins
5c302ebbd7 doc: Move of cluster ops for new IA.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-14 14:55:29 -08:00
Joao Eduardo Luis
8430210cc9 doc: add-or-rm-mons.rst: fix typo
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-14 10:06:31 +00:00
Sage Weil
4a34965c56 client: register admin socket commands without lock held
Avoid a lock cycle.

existing dependency Client::client_lock (11) -> AdminSocket::m_lock (16) at:
 ceph version 0.54-578-g7926ef5 (7926ef5393)
 1: (Mutex::Lock(bool)+0x41) [0x831337]
 2: (AdminSocket::register_command(std::string, AdminSocketHook*, std::string)+0x40) [0x873a32]
 3: (Client::init()+0x454) [0x6f4c24]
 4: (main()+0x637) [0x6ea399]
 5: (__libc_start_main()+0xed) [0x7fd97bbca76d]
 6: ./ceph-fuse() [0x6e9c59]

    -4> 2012-11-13 18:14:48.619714 7fd97b1a3700  0 new dependency AdminSocket::m_lock (16) -> Client::client_lock (11) creates a cycle at
 ceph version 0.54-578-g7926ef5 (7926ef5393)
 1: (Mutex::Lock(bool)+0x41) [0x831337]
 2: (Objecter::RequestStateHook::call(std::string, std::string, ceph::buffer::list&)+0x7a) [0x90627e]
 3: (AdminSocket::do_accept()+0xb1b) [0x87318f]
 4: (AdminSocket::entry()+0x2fa) [0x8725fe]
 5: (Thread::_entry_func(void*)+0x23) [0x86b335]
 6: (()+0x7e9a) [0x7fd97d279e9a]
 7: (clone()+0x6d) [0x7fd97bc9ccbd]

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-13 18:18:24 -08:00
Sage Weil
4db9442bad objecter: separate locked and unlocked init/shutdown
We don't want to hold the lock while we register the admin socket commands
or else we create a lock cycle when we try to process them later.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-13 18:17:55 -08:00
Gary Lowell
7926ef5393 Merge branch 'next'
Conflicts:
	configure.ac
	src/rgw/rgw_common.cc
2012-11-13 17:29:47 -08:00
Samuel Just
d395131c7f osd/: add config helper for min_size and update build_simple*
min_size should never be set to 0 on a pool.  config.h
now has a helper to determine the correct default value.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-11-13 17:09:26 -08:00