Commit Graph

22944 Commits

Author SHA1 Message Date
Sage Weil
17c627b5e4 Merge remote-tracking branch 'gh/wip-cephtool' into next 2012-12-20 11:04:29 -08:00
Sage Weil
f38d891138 Merge branch 'wip-build-fixes' into next 2012-12-20 10:49:34 -08:00
Yehuda Sadeh
a803159b02 rgw: configurable exit timeout
Fixes: #3638

rgw exit timeout secs : number of seconds to wait for process
to exit cleanly before forcing exit. If set to 0, it'l wait
indefinitely.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-12-20 10:13:16 -08:00
Yehuda Sadeh
92b59e9059 rgw: don't try to assign content type if not found
Fixes: #3648
Cannot assign a NULL pointer into stl string. This is only
relevant to swift, when uploading an object without specifying
content type, and when the suffix cannot be determined.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-12-20 09:45:35 -08:00
Sage Weil
c02e9062b9 Merge remote-tracking branch 'gh/wip-crushtool' into next
Reviewed-by: Caleb Miles <caleb.miles@inktank.com>
2012-12-20 08:53:19 -08:00
Yehuda Sadeh
08c64249eb rgw: don't initialize keystone if not set up
Fixes: #3653
No need to initialize keystone, including the keystone
revocation thread which was verbose if key stone was
not set up. This removes some unuseful errors from the
log.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-12-19 22:03:56 -08:00
Yehuda Sadeh
799c59ae89 rgw: remove useless configurable, fix swift auth error handling
Fixes: #3649
No need to have an extra configurable to use keystone. Use keystone
whenever keystone url has been specified. Also, fix a bad error
handling that turned a failure to authenticate into successfully
authenticating a bad user.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-12-19 22:03:56 -08:00
Samuel Just
9a9778fb9c Merge remote-tracking branch 'upstream/wip_pg_temp' into next
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Luis <joao.luis@inktank.com>
2012-12-19 16:51:25 -08:00
Samuel Just
6122a9f62f OSDMonitor: remove temp pg mappings with no up pgs
Otherwise, the pg won't be validly mapped until one of the temp
pgs comes back up.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-12-19 10:33:40 -08:00
Samuel Just
2395af9f7a OSDMap: make apply_incremental take a const argument
This requires us to copy bufferlists in two cases since bufferlist
does not have a const interator at this time.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-12-19 10:32:52 -08:00
Sage Weil
2e49d5c4b7 cephtool: add qa workunit
A few basic sanity checks, including a tell on a down osd.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-19 08:37:42 -08:00
Gary Lowell
d9c2396b55 ceph.spec.in: Improve finding location of jni.h for sles11.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
2012-12-18 21:00:15 -08:00
Sage Weil
b2eb8bd2ed osd: implement 'version' tell command
Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-18 20:08:42 -08:00
Gary Lowell
46344105e7 ceph.spec.in: Add packages for libcephfs-jni and libcephfs-java
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
2012-12-18 19:40:32 -08:00
Sage Weil
85763f09fe ceph: report error string to stderr, not stdout
If we return an error, send the message to stderr.  This makes things
more easily scriptable because error messages won't take the place of
expected output.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-18 19:21:24 -08:00
Sage Weil
5f24e23b19 ceph: fix error reporting when tell target is invalid or down
Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-18 19:20:06 -08:00
Sage Weil
b00eb6fd87 mon: 'ceph osd ls'
List osd ids that exist.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-18 19:11:49 -08:00
Samuel Just
212f6b56d1 OSDMap::dump: tag pg_temp mappings with pgid
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2012-12-18 17:00:45 -08:00
Sage Weil
6c7ec2d435 crushtool: nicer error message on extra args
Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-17 17:12:06 -08:00
Sage Weil
dbe6fb7281 crushtool: only dump usage on -h|--help
Instead, output a useful error message.

Fix error code to be a success.

Add test for the output usage.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-17 17:12:06 -08:00
Sage Weil
0dd1302543 Merge remote-tracking branch 'gh/testing' into next 2012-12-17 16:51:20 -08:00
Gary Lowell
fd482a2781 ceph.spec.in: Update pre-reqs for ceph-fuse pacakge. 2012-12-17 16:38:19 -08:00
Sage Weil
1b67a43891 Revert "objecter: don't use new tid when retrying notifies"
This reverts commit c3107009f66bc06b5e14c465142e14120f9a4412.

This appears to be causing problems in the objecter by corrupting
the stack.  Until that is resolved, let's revert.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-17 16:29:19 -08:00
Joao Eduardo Luis
bdc998ef4c mon: OSDMonitor: add option 'mon_max_pool_pg_num' and limit 'pg_num' accordingly
Instead of having a hardcoded default, use a configurable one. It is
limited to 65536 until future testing guarantees there is no side-effects
of increasing it past this value, but by being adjustable the user still
has the freedom to specify whatever maximum value he wants.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-12-17 14:41:48 -08:00
Sage Weil
3c246226a5 crushtool: add --set-chooseleaf-descend-once to help
We forgot to update this in 88f218181a9e6d2292e2697fc93797d0f6d6e5dc.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-17 11:14:44 -08:00
Gary Lowell
e9231fe6c1 Makefiles: Two new packages needed in the debian build depdencies.
The ceph test programs that are now being built by default require the junit
and libboost-program-options packages.  These have been added to the build
dependecies in the debian control file.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
2012-12-16 12:29:54 -08:00
James Page
bc9d9d8a2b Refactor rule file to separate arch/indep builds.
Prior to the ceph fs java bindings, all packages where
architecture depdendent so the packaging rules file
worked OK; this fixes up the binary-indep/arch targets
to split the builds of architecture dependent and
independent files.

Signed-off-by: James Page <james.page@ubuntu.com>
2012-12-16 12:29:30 -08:00
Sage Weil
4bf9078286 osdc/Objecter: prevent pool dne check from invalidating scan_requests iterator
We iterate over ops and, if the pool dne and other conditions are true,
we will immediately return ENOENT and cancel an op.  Increment the
iterator at the top of the loop to avoid invalidating it.

We also need to switch to a map<>, because hash_map<> mutations may
invalidate any/all iterators.

Fixes: #3613
Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-15 17:45:25 -08:00
Greg Farnum
1ec70aa0dd qa: add a workunit for fsync-tester
It turns out that our suites don't exercise fsync, at least not very much
(I couldn't find it in all the places I looked for it). This tester
was written by Ted T'so and updated by Chris Mason; I just made it
work on a smaller dataset (256MB) because 8GB against a small cluster takes
more time than we want to wait.

Signed-off-by: Greg Farnum <greg@inktank.com>
2012-12-14 15:24:36 -08:00
Noah Watkins
286dcbeb55 test: remove underscores from cephfs test names
Google Test documentation strongly suggests avoiding underscores from
unit test names to avoid accidental conflicts with their macro naming
scheme.

http://code.google.com/p/googletest/wiki/FAQ#Why_should_not_test_case_names_and_test_names_contain_underscore

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2012-12-14 15:31:00 -08:00
Sage Weil
79db5a40c7 Merge branch 'wip_watch' into next 2012-12-14 14:32:44 -08:00
Sam Lang
a7de975d93 lockdep: Decrease lockdep backtrace skip by 1
Skipping the top 4 (it starts at 0) calls in the
backtrace actually skips the call that does the lock.
Skip 3 instead.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-12-14 14:17:40 -08:00
Sage Weil
641b077f9e mkcephfs: fix == -> =
Another bashism.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-14 14:20:00 -08:00
Alex Elder
bf01b7b2e2 map-unmap.sh: use udevadm settle for synchronization
This script was heuristically using short sleep commands in order to
give udev activity time to complete.

There's a command "udevadm settle" which actually looks at the udev
queue and waits until its processing is done.  Much, much better.

This rearranges the get_id function a bit too, breaking it into one
function that gets the id and another that loops back and tries
again after a short delay in the event the get_id fails.

Signed-off-by: Alex Elder <elder@inktank.com>
2012-12-14 15:58:39 -06:00
Sage Weil
c728171b91 Merge branch 'wip-upstart' into next
Reviewed-by: Greg Farnum <greg@inktank.com>
2012-12-14 13:51:19 -08:00
Sage Weil
02aca6830a ceph-disk-activate: mark dir as upstart-managed
Mark the directory so that upstart will manage the daemon.  Eventually,
this should be generalized to allow ceph-disk-* usage with other init
systems.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-14 13:49:26 -08:00
Sage Weil
96f40b146b upstart: make starter jobs consistent
Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-14 13:49:26 -08:00
Sage Weil
e597482f29 upstart: only start when 'upstart' file exists in daemon dir
We need to distinguish between daemons managed by upstart and sysvinit
(and, eventually, systemd).  Only start daemons when 'upstart' is present.

Note that sysvinit will only start daemons when the 'host = ...' line is
in ceph.conf, so there is a similar "opt-in".

Signed-off-by: Sage Weil <sage@inktank.com>
2012-12-14 13:49:26 -08:00
Samuel Just
6ab7db6717 ReplicatedPG: use default priority for Backfill messages
Backfill messages modify the stats on the replica and therefore
must be sent with the same priority as sub_op_modify to ensure
ordering.  Using recovery_op_priority caused the following
sequence:

1) Primary(1) sends MOSDPGBackfill FINISH with updated stats (v1)
2) Primary(1) sends SubOp modify for new client op with stats (v2)
3) Replica(2) receives SubOp with stats (v2)
4) Replica(2) receives MOSDPGBackfill FINISH with stats (v1)
5) Replica(2) responds and Primary(1) resets pgtemp making
    Replica(2) Primary(2)
6) PG stats on Primary(2) several ops old.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-12-14 13:38:51 -08:00
Samuel Just
7e1335691d ReplicatedPG: do not use priority from client op
There are internal ordering requirements which may be sensitive
to assigned priority.  We don't want a mix of priorities from
old clients with priorities from new clients causing trouble.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-12-14 13:38:37 -08:00
Sam Lang
b63940caa1 Merge branch 'wip-3610' into next 2012-12-14 09:00:24 -10:00
Greg Farnum
8d73f3e946 Fix comment in sample.ceph.conf
Signed-off-by: Greg Farnum <greg@inktank.com>
2012-12-14 09:53:38 -08:00
Sam Lang
f16e571757 client: Add config option to inject sleep for tick
Testing the tick delay with a fork/suspend is causing
corruption in the lockdep code.  This approach uses
a config option to sleep the tick thread for a number
of seconds, avoiding the entire fork/suspend mess.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-12-13 17:49:43 -10:00
Josh Durgin
8cf367cb79 rbd.py: check for new librbd methods before use
This way attempting to use format 2 images works when you upgrade the
python bindings before librbd, and attempting to use functions
that librbd does not have results in more understandable errors.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-12-13 17:43:23 -08:00
Sage Weil
c9894ff0e5 osd: up != acting okay on mkpg
This can happen when:

 - mon sends create pg
 - it gets created
 - osd remaps the pg to a different osd
     but osd does not update pg status to the mon
 - mkpg resent to the new osd

or something along those lines.  It seems unusual, but in the end who
really cares why the mon doesn't know about the pg creation yet.

Note that this check was added in the initial commit where acting/up was
added; there is no specific condition of concern we are protecting against.

Instead, ignore the message.  We'll get a query soon anwyay.

This 'fixes' #3614.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-12-13 16:27:09 -08:00
Joao Eduardo Luis
e3ed28eb2d mon: OSDMonitor: don't allow creation of pools with > 65535 pgs
There are some limitations to the number of possible pg's per pool, and
by allowing the 'osd pool create' command to succeed, we were making room
to some anomalous behavior.

Fixes: #3617

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2012-12-13 15:38:55 -08:00
Dan Mick
8103414a45 rbd: handle images disappearing while in ls -l
rbd.list() returns a list of names, but nothing stops them from
going away before rbd.open(); check for ENOENT and ignore if that
happens; warn on other errors

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-12-13 14:46:53 -08:00
Samuel Just
97cc55d599 OSD: put connection in disconnect_session_watches as well as the session
obc->watchers now has a ref to the connection as well.  This piece of
disconnect_session_watchers essentially parallels remove_watcher and
should generally do the same thing.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-12-13 10:52:52 -08:00
Samuel Just
f2c083efd5 OSD: disconnect_session_watches obc might not be valid after we relock
If disconnect_session_watches races with watch removal, the session
might no longer have a valid obc ref.  In that case, move on to
the next obc.

Note, there is no danger of any obcs being *added* to the session
since the session/connection at this point is dead.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-12-13 10:52:52 -08:00
Greg Farnum
c17d628b52 clarify/correct some of sample.ceph.conf
Signed-off-by: Greg Farnum <greg@inktank.com>
2012-12-13 10:39:19 -08:00