Commit Graph

22768 Commits

Author SHA1 Message Date
Chen Baozi
68491afceb rgw: add -lresolv flags to Makefile.am
radosgw depends on libresolv since since the commit 951c6be. So we need to
add -lresolve flags, or it cannot link right library.

Signed-off-by: Chen Baozi <baozich@gmail.com>
2012-11-17 23:21:08 -08:00
Sage Weil
7903aabeb1 mon/MonClient: use thread-safe RNG for picking monitors
Avoid using shared-state rand() when picking monitors.  This way we don't
screw with library users like test_librbd_fsx that rely on srand() and
rand() being deterministic.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-17 16:30:00 -08:00
Sage Weil
f9fd0659cd Merge remote-tracking branch 'gh/wip-3431' into next 2012-11-16 21:26:30 -08:00
Sage Weil
07c831acd5 upstart: fix limit lines
Two arguments.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 17:59:08 -08:00
Sage Weil
b4a769df32 upstart: add ceph-osd-all-starter.conf
Starter helper will start all osds that appear in /var/lib/ceph/osd/*,
as we do with the mons and mdss.

This will only proceed if the 'ready' file is there, which is currently
only touched by ceph-disk-activate.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 17:28:14 -08:00
Sage Weil
ff0a44bb81 upstart: make ceph-osd-all, ceph jobs
This will let you start/stop all daemons.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 17:28:14 -08:00
Josh Durgin
48295a188f Merge branch 'next' 2012-11-16 17:14:28 -08:00
Josh Durgin
3610754a57 Makefile.am: fix LDADD for test_objectcacher_stress
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 17:14:06 -08:00
Sage Weil
e85c9e7b16 Merge branch 'wip-coverity' into next
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 17:36:34 -08:00
Sage Weil
12eb797fb8 client: fix lock leak in lazio_*() failure paths
CID 743400 (#1 of 1): Missing unlock (LOCK)
At (5): Returning without unlocking "this->client_lock._m".

CID 743399 (#1 of 1): Missing unlock (LOCK)
At (5): Returning without unlocking "this->client_lock._m".

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 17:36:16 -08:00
Josh Durgin
5909997243 Merge branch 'next' 2012-11-16 16:44:41 -08:00
Josh Durgin
78382fecaa Merge branch 'wip-oc-hang' into next
Reviewed-by: Sage Weil <sage.weil@inktank.com>
2012-11-16 16:43:07 -08:00
Sage Weil
be11c317bf upstart: set high open file limits
The default 1024 limit is easily hit on larger clusters.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 16:19:00 -08:00
Sage Weil
25f003ba5f msg/Accepter: only close socket if >= 0
It is possible for rebind() to fail, in which case the OSD will go through
it's shutdown procedure and call stop().  This is simpler than trying to
avoid calling stop() when rebind() fails.

Fixes: #3504
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 16:10:56 -08:00
Sage Weil
30373ce872 osd: default journal size to 5GB
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 16:04:13 -08:00
Josh Durgin
a562518b6b librbd: take cache lock when discarding data from cache
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:27:52 -08:00
Josh Durgin
2e862f4d18 ObjectCacher: fix off-by-one error in split
This error left a completion that should have been attached
to the right BufferHead on the left BufferHead, which would
result in the completion never being called unless the buffers
were merged before it's original read completed. This would cause
a hang in any higher level waiting for a read to complete.

The existing loop went backwards (using a forward iterator),
but stopped when the iterator reached the beginning of the map,
or when a waiter belonged to the left BufferHead.

If the first list of waiters should have been moved to the right
BufferHead, it was skipped because at that point the iterator
was at the beginning of the map, which was the main condition
of the loop.

Restructure the waiters-moving loop to go forward in the map instead,
so it's harder to make an off-by-one error.

Possibly-fixes: #3286
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
fdadefe331 ObjectCacher: begin at the right place when iterating over BufferHeads
If the desired offset overlaps a BH, data.lower_bound() will return
the element after it, since it's indexed by the start of a range.

The confusingly similarly named data_lower_bound() method fixes this,
and returns the correct starting element.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
20a0c56da6 ObjectCacher: add debug function to check BufferHead consistency
This isn't called because it's potentially expensive, but calling it
in various places can help future debugging.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
5d760b776a ObjectCacher: more debugging for read completions
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
c054ad6d48 ObjectCacher: assert lock is held everywhere
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
7570e6c894 ObjectCacher: debug read waiters
Now we can tell which ones will be called.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
8c961610a6 ObjectCacher: don't needlessly increment iterator
This iterator is now reset on each run through the loop,
so there's no point in incrementing it here.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:38 -08:00
Josh Durgin
b948e4c91f ObjectCacher: retry reads when they are incomplete
Skipping these callbacks when there's a racing write or
a gap in the results causes the original reads they represent
to never be completed. If the read falls within the range
of a BufferHead, retry all waiters no matter what.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 15:22:27 -08:00
Yehuda Sadeh
29a96cf29c rgw: ops log can also go to socket
Adding a new ops log output (into a unix domain socket).
Configuration:
  rgw_enable_usage_log : master switch for ops log
  rgw ops log socket path : set socket path
  rgw ops log rados : whether ops should be logged in the rados
                      cluster
  rgw ops log data backlog : max size in MB to be accumulated
                             without flushing

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-16 15:17:20 -08:00
John Wilkins
7b96807dfc doc: Added links to APIs.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 15:13:57 -08:00
Greg Farnum
feb8451f05 doc: don't use 'pool=data' as an example
Signed-off-by: Greg Farnum <greg@inktank.com>
2012-11-16 14:28:51 -08:00
John Wilkins
c42af8030f doc: Added documentation for using CPU profiler.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:58:22 -08:00
John Wilkins
84eac86696 doc: Added documentation for installing CPU profiler.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:57:55 -08:00
John Wilkins
3ccaaa0923 doc: Added librbd to the RBD toc.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:57:15 -08:00
John Wilkins
9ea2609ea0 doc: Added libcephfs to the CephFS section and moved librbd to RBD section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-16 14:56:46 -08:00
Sage Weil
8da6ddeaec common/ceph_argparse: fix malloc failure check
CID 743418 (#1 of 1): Dereference before null check (REVERSE_INULL)
Null-checking "argv" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:19:25 -08:00
Sage Weil
e82ca0d5e0 mon/MonClient: initialize ptr in ctor
CID 743433 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
At (2): Non-static class member "authorize_handler_registry" is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:18:21 -08:00
Sage Weil
8f1f36d5a2 os/FileStore: fix fd leak in _rmattr
CID 743405 (#2 of 2): Resource leak (RESOURCE_LEAK)
At (16): Handle variable "fd" going out of scope leaks the handle.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:15:23 -08:00
Sage Weil
426b58da29 os/FileStore: fix fd leaks in _setattrs
CID 743406 (#3 of 3): Resource leak (RESOURCE_LEAK)
At (26): Handle variable "fd" going out of scope leaks the handle.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:14:17 -08:00
Sage Weil
1df38fd73d osdc/ObjectCacher: faux use-after-free
CID 743435 (#1 of 1): Use after free (USE_AFTER_FREE)
At (68): Passing freed pointer "rd" as an argument to function "std::basic_ostream<char, std::char_traits<char> >::operator <<(void const *)".

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-16 14:11:05 -08:00
Josh Durgin
9a10ebb2ef test: add ObjectCacher stress test that does not use a cluster
Use a fake writeback handler and respond to all requests with -ENOENT.
This tests that all operations will complete, and the cache doesn't
lose waiters or callbacks.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 13:23:26 -08:00
Josh Durgin
fd928b9b71 ObjectCacher: more debugging for BufferHeads
This is useful for checking for lost waiters.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-16 12:11:01 -08:00
Gary Lowell
8b187bd8ca build: update for boost_thread library.
There is a difference in naming conventions between debian and
rpm based distributions for this library.  In configure.ac we
check first for boost_thread-mt, then if it's not found check
for boost_thread.  A side effect of the AC_CEHCK_LIB macro is
to add the library to the $LIBS, so the explicit -llibboost_thread
in the Makefile has been removed.
(cherry picked from commit f0c7bb3630)
2012-11-16 10:51:42 -08:00
Joao Eduardo Luis
493049b4b6 mon: OSDMonitor: clarify some command replies
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-16 15:30:53 +00:00
Joao Eduardo Luis
0b28ef6a15 mon: OSDMonitor: fix spacing when outputting items on command reply
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-16 15:30:24 +00:00
Gary Lowell
27830e2c13 Merge branch 'master' of https://github.com/ceph/ceph 2012-11-16 00:56:36 -08:00
Gary Lowell
f0c7bb3630 build: update for boost_thread library.
There is a difference in naming conventions between debian and
rpm based distributions for this library.  In configure.ac we
check first for boost_thread-mt, then if it's not found check
for boost_thread.  A side effect of the AC_CEHCK_LIB macro is
to add the library to the $LIBS, so the explicit -llibboost_thread
in the Makefile has been removed.
2012-11-16 00:46:41 -08:00
Josh Durgin
c707568a1b doc: fix crush set syntax
No commas between parameters, pool is required.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-15 23:07:51 -08:00
Sage Weil
71cfaf1cc5 os/FileStore: only try BTRFS_IOC_SUBVOL_CREATE on btrfs
Only try to create a btrfs subvolume if the fs is btrfs.  Otherwise, just
create a directory.  Then we can error out on *any* ioctl error, and not
rely on the ioctl error code to determine if we failed because we are on
a non-btrfs or a real error.

Fixes: #3052
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2012-11-15 16:50:55 -08:00
Sage Weil
3ca947e018 mon: clean up 'ceph osd ...' list output
No more 'osd.0 is already inosd.1 is already in' crap.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 16:35:53 -08:00
Sage Weil
344c4fdc3f mon: correctly identify crush names
get_item_id() returns 0 if the name already exists; that's not what we
want here.  Verify the name exists before checking its id.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 16:32:50 -08:00
Sage Weil
592a89421f mon: use parse_osd_id() throughout
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-15 16:32:50 -08:00
Samuel Just
918c58c85e PrioritizedQueue: remove internal lock, not used
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-11-15 16:02:34 -08:00
Samuel Just
b53e06cac8 DispatchQueue: lock DispatchQueue when for get_queue_len()
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-11-15 15:59:18 -08:00