Commit Graph

28066 Commits

Author SHA1 Message Date
Yehuda Sadeh
b5e74485d9 rgw: set bucket attrs are a bucket instance meta operation
Need to do the action through the bucket instance handler
and not through the bucket handler, otherwise it's wrongly
recorded (and wrongly replayed, ouch).

Fixes: #5791

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-29 15:06:56 -07:00
Yehuda Sadeh
68730d80c8 rgw: track bucket instance oid
We now keep the bucket instance oid in rgw_bucket. The reason
we need it is that the bucket might have been created before
the entrypoint / bucket instance separation.

Fixes: #5790

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-29 15:06:56 -07:00
Yehuda Sadeh
acd16d1aed rgw: read / write bucket attributes from bucket instance
and not from bucket entry point.

Fixes: #5770

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-29 15:06:56 -07:00
Sage Weil
f3eda63494 mon/PGMonitor: fix 'pg dump_[pools_]json'
Use the correct type for the dumpcontents arg.  Fixes the dump_pools_json
output and avoids these errors:

2013-07-29 13:09:14.089188 7fa0c5d21700 -1 0x7fa0c5d1e7a8
2013-07-29 13:09:16.306560 7fa0c5d21700 -1 bad boost::get: key dumpcontents is not type std::vector<std::string, std::allocator<std::string> >
2013-07-29 13:09:16.317104 7fa0c5d21700 -1 0x7fa0c5d1e7a8
2013-07-29 13:09:16.317136 7fa0c5d21700 -1 bad boost::get: key dumpcontents is not type std::vector<std::string, std::allocator<std::string> >

Fixes: #5786
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-29 13:13:24 -07:00
Sage Weil
099ac5184d mon: fix xml element name
Should be the type, not the name.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-29 13:01:30 -07:00
Loic Dachary
ee9f04c5f8 check_new_interval must compare old acting with old osdmap
When trying to establish if the old acting set is either empty or
smaller than the min_size of the osdmap,
pg_interval_t::check_new_interval compares with the min_size of the
new osdmap. Since the goal is to try to determine if the previous
interval may have been writeable, it should not enter the if when
there were not enough osds in the acting set ( i.e. < min_size ). But
it may enter it anyway if min_size was decremented in the new osdmap.

A complete set of unit tests were added to cover the logic of
check_new_interval. The parameters are prepared to describe a
situation where the function returns false (i.e. no new
interval). Each case is described in a separate bloc that introduces
the minimal changes to demonstrate the intended test case.

Because a number of cases have the same output while implementing a
different logic, the debug output is parsed to differentiate between them.

A test case demonstrating the problem ( check_new_interval must
compare old acting with old osdmap ) is added, with a link to the bug
number for future reference. The problem is fixed. The text of two
debug messages are slightly changed to make the maintenance of the
test that match them easier.

http://tracker.ceph.com/issues/5780 refs #5780

Signed-off-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-07-29 12:49:25 -07:00
Samuel Just
1f13d8ac5b OSD: suspend tp timeout while taking pg lock in OpWQ
If N op_tp threads are configured, and recovery_max_active
is set to a sufficiently large number, all N op_tp threads
might grab a MOSDPGPush op off of the queue for the same PG.
The last thread to get the lock will have waited
N*time_to_handle_push before completing its item and pinging
the heartbeat timeout.  If that time exceeds the timeout
and there are enough ops waiting, each thread subsequently
will end up exceeding the timeout before completeing an
item preventing the OSD from heartbeating indefinitely.

We prevent this by suspending the timeout while we try to
get the PG lock.  Even if we do block for an excessive
period of time attempting to get the lock, hopefully,
the thread holding the lock will cause the threadpool
to time out.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-29 12:49:16 -07:00
Samuel Just
f1bd4e5bdf WorkQueue: fix bracing on reset_tp_timeout
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-29 12:49:09 -07:00
Sage Weil
a6cd9fea50 osd: get initial full map after a map gap
If there is a gap in our map history, get the full range of maps that
the mon has.  Make sure the first one is a full map.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-07-29 12:12:14 -07:00
Sage Weil
e24b50225c osd: fix off-by-one in map gap logic
If we have map 250, and monitor's first is 251, but sends 260, we can
request the intervening range.

Fixes: #5784
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-07-29 12:11:35 -07:00
athanatos
1cb3216c5f Merge pull request #425 from dachary/wip-5527
unit tests for sharedptr_registry

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-07-29 11:00:59 -07:00
Thomas Bechtold
251a6a498d Use dh_installinit to install upstart job files
Signed-off-by: Thomas Bechtold <thomasbechtold@jpberlin.de>
2013-07-29 14:53:58 +02:00
Sage Weil
b62845e739 doc/changelog/v0.61.7
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 20:43:01 -07:00
Sage Weil
ee18322e5b Merge remote-tracking branch 'gh/next' 2013-07-28 16:17:55 -07:00
Sage Weil
a46f60afde doc/dev/repo-lab-access: notes
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 16:17:50 -07:00
Sage Weil
12c1f1157c ceph_test_rados: print version banner on startup
It is helpful when looking at qa run logs to see what version of the
tester is running.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 15:42:08 -07:00
Sage Weil
74c1bec2aa ceph-authtool: fix cli tests
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 15:29:49 -07:00
Sage Weil
7b683f7f6a test/system/*: parse CEPH_ARGS environment
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 15:27:57 -07:00
Sage Weil
be73215913 Merge pull request #474 from dalgaaf/wip-da-next-fix-help
Add missing commands to help/usage and manpage of ceph-authtool

Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-28 15:26:17 -07:00
Danny Al-Gaaf
347b5a2078 ceph-authtool.8: add missing commands to man page
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-28 23:39:09 +02:00
Danny Al-Gaaf
86b2632831 ceph_authtool.cc: update help/usage text
Added implemented but not listed commands to the help/usage text:
* -g  shortcut for --gen-key
* -a  shortcut for --add-key
* -u/--set-uid to set auid
* --gen-print-key
* --import-keyring

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-28 23:25:58 +02:00
Sage Weil
4b6c569a11 mon/DataHealthService: do not name xml key after mon
The name might be something like '0', which is illegal.  This should be
the *type*, not name.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 08:19:12 -07:00
Sage Weil
dd3b0865e0 Merge pull request #464 from toabctl/fuse-utils-obsolete
Remove fuse-utils from Recommends in debian/control

Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-27 17:34:12 -07:00
Sage Weil
6881ab3b39 debian, rpm: make python-ceph depend on python-requests
For ceph-rest-api.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-27 17:32:49 -07:00
Loic Dachary
2ec480b1ba replace in_method_t with a counter
A single counter ( waiting ) accurately reflects the number of
waiters, regardless of the method waiting. It is enough to allow
unit tests to synthetise all situations, including:

T1: x = lookup_or_create(0)
T1: release x part 1 (weak_ptrs now fail to lock)
T2: y = lookup_or_create(0)
T2: block in lookup_or_create (waiting == 1)
T1: z = lookup_or_create(1) (does not block because the key is different)
    while holding the lock it waiting++ and waiting == 2
    and before returning it waiting-- and waiting is back to == 1
T1: complete release x
T2: complete lookup_or_create(0) (waiting == 0)

The unit tests are modified to add a lookup on an unrelated key to
demonstrate that it does not reset waiting counter.

http://tracker.ceph.com/issues/5527 refs #5527

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-07-27 08:11:11 -07:00
Loic Dachary
6b16cd1aaa unit tests for sharedptr_registry
Covers 100% of the LOC and all the expected behavior, including thread
safety.

The sharedptr_registry is made friend of the test class so that it can
synthetize race conditions. The lookup and lookup_or_create methods
set the new in_method data member before calling cond.Wait() so that
the caller knows it is waiting.

http://tracker.ceph.com/issues/5527 refs #5527

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-07-27 08:11:11 -07:00
Sage Weil
b5250fdc70 Merge pull request #471 from dalgaaf/wip-da-fix-pylint
Cleanup minor issues from pylint

Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-27 07:33:21 -07:00
Danny Al-Gaaf
09ee092ffd pybind/rbd.py: remove unused import of 'pointer'
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-27 11:37:37 +02:00
Danny Al-Gaaf
246011807b cephfs.py: remove unused imports
Removed ctypes, c_uint64, c_ubyte, pointer, CFUNCTYPE since they
are not used in the code.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-27 11:37:34 +02:00
Danny Al-Gaaf
865d5e9255 rados.py: fix bad indentation
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-27 11:37:26 +02:00
Danny Al-Gaaf
6eadb870b4 ceph_argparse.py: add missing spaces after comma
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-27 11:36:35 +02:00
Danny Al-Gaaf
da4d23ec08 ceph_argparse.py: remove unnecessary semicolons
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-27 11:35:38 +02:00
Danny Al-Gaaf
a48b391fce ceph_rest_api.py: fix bad indentation
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-27 11:32:34 +02:00
Sage Weil
25948319c4 rgw: fix leak of RGWDataChangesLog::renew_thread
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 23:22:25 -07:00
Sage Weil
a31356338b rgw: free resolver on shutdown
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 23:22:25 -07:00
Sage Weil
59b13cebee rgw: fix up signal handling
OMG libfcgi is annoying with shutdown and signals.  You need to close
the fd *and* resend a signal to ensure that you kick the accept loop
hard enough to make it shut down.

Document this, and switch to the async signal handlers.  Put them
tightly around the runtime loop as we do with other daemons.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 23:22:20 -07:00
Sage Weil
bd0e35f2e1 test/admin_socket: fix admin_socket unit tests
Command is now JSON!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 22:28:40 -07:00
Sage Weil
94865368ca rgw: fix RGWFormatter_Plain
Needs dump_format_unqouted.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 22:14:47 -07:00
Sage Weil
535d8701b9 common/admin_socket: do not populate empty help strings
Hidden commands have no help string.  Make this consistent: the m_help
entry is always there, but may be empty.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 22:08:26 -07:00
Sage Weil
b4bde3cbc0 common/perf_counters: fix unit tests
The commands are now in json.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 21:47:34 -07:00
Sage Weil
2aa9afa5c5 common/perf_counters: fix missing decimal in time, quoting
We shouldn't quote integer or float values.  Also easier to use
dump_unsigned.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 21:47:33 -07:00
Sage Weil
a8c1a2a28b common/Formatter: add dump_format_unquoted()
Sometimes a savvy caller wants to format the output but doesn't want it
quoted.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 21:47:33 -07:00
Dan Mick
0041e9f8ff rados.py: missing parameter to make_ex()
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 21:38:12 -07:00
Dan Mick
8af47755af ceph-rest-api: clean up options/environment
ceph-rest-api:
* create app from wrapper by calling generate_app()
* pass args to generate_app() (early parsed in wrapper)
* parse -i/--id here as well
* set addr:port on returned app object
* handle only EnvironmentError exceptions; let others spew traceback
* turn off debug when running singlethreaded server

ceph_rest_api.py:
* put glob.* on app.ceph_* instead; pass around app in init code
* drop conf parsing (let librados do its job)

Documentation updated to match.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 21:38:12 -07:00
Sage Weil
629326aa92 qa/fs/.gitignore
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 21:06:14 -07:00
Sage Weil
803a1fdd08 ceph_test_admin_socket: fix unit test
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 21:06:06 -07:00
David Zafman
a419354120 message: Fix asserts that dont' trigger
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 18:31:04 -07:00
Sage Weil
a9ca6234c8 librados: EINVAL on a negative osd id
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 17:26:48 -07:00
Dan Mick
3f93691bf3 ceph.in: make osdids() (and mon, mds) work on old mons
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 17:24:22 -07:00
Sage Weil
4b739005a5 osd: humor coverity
CID 1058392: Out-of-bounds access (OVERRUN_DYNAMIC) [select issue]

CID 1058391 (#1 of 1): Out-of-bounds access (OVERRUN)
32. alloc_strlen: Allocating insufficient memory for the terminating null of the string.

CID 1058390 (#1 of 1): Unchecked return value from library (CHECKED_RETURN)
13. check_return: Calling function "this->class_handler->open_all_classes()" without checking return value. It wraps a library function that may fail and return an error code. [show details]
    	14. unchecked_value: No check of the return value of "this->class_handler->open_all_classes()".

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 17:14:44 -07:00