Commit Graph

26959 Commits

Author SHA1 Message Date
Sage Weil
e97a2c86de Merge remote-tracking branch 'yan/wip-mds' into wip-mds 2013-06-20 15:55:41 -07:00
Dan Mick
31d221c3a4 ceph.in: remove some TAB chars
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-20 15:14:36 -07:00
Dan Mick
69e1a9121d ceph.in: fix ^C handling in watch (trap exception in while, too)
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-20 15:14:36 -07:00
Sage Weil
29f6f27729 ceph: --version as well as -v
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 15:04:51 -07:00
Sage Weil
4bf5b732cd Merge remote-tracking branch 'gh/next' 2013-06-20 12:30:54 -07:00
Yan, Zheng
714f2128bd osdc: re-calculate truncate_size for strip objects
Feed truncate_size through the striping algorithm so that it reflects the
correct per-object offset (as opposed to the file offset).

Fixes #5380
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-06-20 12:26:30 -07:00
Sage Weil
ebb46c452e qa/workunits/misc/multiple_rsync.sh: wtf
2013-06-15T12:55:29.808 INFO:teuthology.task.workunit.client.0.err:+ rsync -auv --exclude local/ /usr/ usr.1
2013-06-15T12:55:29.808 INFO:teuthology.task.workunit.client.0.err:+ tee a
2013-06-15T12:55:29.820 INFO:teuthology.task.workunit.client.0.out:sending incremental file list
2013-06-15T12:56:46.019 INFO:teuthology.task.workunit.client.0.out:
2013-06-15T12:56:46.020 INFO:teuthology.task.workunit.client.0.out:sent 1452634 bytes  received 7485 bytes  19086.52 bytes/sec
2013-06-15T12:56:46.020 INFO:teuthology.task.workunit.client.0.out:total size is 3205063225  speedup is 2195.07
2013-06-15T12:56:46.020 INFO:teuthology.task.workunit.client.0.err:+ wc -l a
2013-06-15T12:56:46.021 INFO:teuthology.task.workunit.client.0.out:4 a
2013-06-15T12:56:46.022 INFO:teuthology.task.workunit.client.0.err:+ wc -l a
2013-06-15T12:56:46.022 INFO:teuthology.task.workunit.client.0.err:+ grep 4
2013-06-15T12:56:46.023 INFO:teuthology.task.workunit.client.0.out:4 a
2013-06-15T12:56:46.024 INFO:teuthology.task.workunit.client.0.err:+ rsync -auv --exclude local/ /usr/ usr.2
2013-06-15T12:56:46.024 INFO:teuthology.task.workunit.client.0.err:+ tee a
2013-06-15T12:56:46.112 INFO:teuthology.task.workunit.client.0.out:sending incremental file list
2013-06-15T12:57:17.172 INFO:teuthology.task.workunit.client.0.out:
2013-06-15T12:57:17.174 INFO:teuthology.task.workunit.client.0.out:sent 1452634 bytes  received 7485 bytes  46352.98 bytes/sec
2013-06-15T12:57:17.174 INFO:teuthology.task.workunit.client.0.out:total size is 3205063225  speedup is 2195.07
2013-06-15T12:57:17.175 INFO:teuthology.task.workunit.client.0.err:+ wc -l a
2013-06-15T12:57:17.175 INFO:teuthology.task.workunit.client.0.out:3 a

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 21e85f90be)
2013-06-20 12:19:16 -07:00
Sage Weil
fd769c0f21 qa/workunits/cephtool/test.sh: fix and cleanup several tests
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 11:28:26 -07:00
Sage Weil
f420e5c614 mon: drop deprecated 'stop_cluster'
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 11:23:38 -07:00
Sage Weil
4977b88a7c mds: make 'mds compat rm_*compat' idempotent
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 11:23:11 -07:00
Sage Weil
4a038d6df5 mon: make 'log ...' command wait for commit before reply
Previously we would just dump the command argument to our local log client
and reply immediately, which could lose the message if we then restarted.
Instead, commit directly and wait before replying.

Also, log as the actual client, not as the monitor processing the message.

Fixes: #5409
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-06-20 11:11:50 -07:00
Sage Weil
5de54f6a79 a/workunits/cephtool/test.sh: --no-log-to-stderr when examining stderr
We can get random messages to stderror from socket reconnects and such;
discard those if we are looking at stderr in the test.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 11:04:26 -07:00
Sage Weil
d60534b8f5 mon: more fix dout use in sync_requester_abort()
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 09:46:42 -07:00
Sage Weil
8a4ed58e39 mon: fix raw use of *_dout in sync_requester_abort()
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 08:36:26 -07:00
Sage Weil
c8af85d451 doc/release-notes: v0.61.4
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-20 08:08:14 -07:00
Sage Weil
1fccfd8b46 Merge pull request #367 from ceph/wip-ceph-cli
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-06-19 20:02:27 -07:00
Samuel Just
84444d0be6 Merge remote-tracking branch 'upstream/next' 2013-06-19 19:54:31 -07:00
Samuel Just
c39b99cdec FileStore: handle observers in constructor/destructor
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-06-19 19:53:55 -07:00
Sage Weil
021158306e init-radosgw: use radosgw --show-config-value to get config values
This way we get the correct default values, as reflected by radosgw
itself.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 19:44:46 -07:00
Sage Weil
88f73c5a63 ceph: fix ceph-conf call to get admin socket path for 'daemon <name> ...'
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 19:44:45 -07:00
Sage Weil
716fc3501a ceph-conf: make --show-config-value reflect daemon defaults
We want DAEMON defaults, but we don't want global_init to do anything else
daemonish like print a banner or mkdir /var/run/ceph.  This lets us use

 ceph-conf -n osd.0 --show-config-value log_file

to get the default, while

 ceph-conf -n osd.0 log_file

only reflects what is in the config file.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 19:44:45 -07:00
Samuel Just
cf3bc25197 FileStore: apply changes after disabling m_filestore_replica_fadvise
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit ed8b0e65bd)
2013-06-19 19:15:53 -07:00
Samuel Just
ed8b0e65bd FileStore: apply changes after disabling m_filestore_replica_fadvise
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-06-19 19:14:40 -07:00
Sage Weil
2a4953b697 ceph-disk: use unix lock instead of lockfile class
The lockfile class relies on file system trickery to get safe mutual
exclusion.  However, the unix syscalls do this for us.  More
importantly, the unix locks go away when the owning process dies, which
is behavior that we want here.

Fixes: #5387
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-06-19 17:58:31 -07:00
Sage Weil
cb6bc95a9c Merge remote-tracking branch 'gh/next' 2013-06-19 17:55:20 -07:00
Alexandre Maragone
8c0daafe00 ceph-disk: make list_partition behave with unusual device names
When you get device names like sdaa you do not want to mistakenly conclude that
sdaa is a partition of sda.  Use /sys/block/$device/$partition existence
instead.

Fixes: #5211
Backport: cuttlefish
Signed-off-by: Alexandre Maragone <alexandre.maragone@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-06-19 16:11:45 -07:00
athanatos
c830b7040c Merge pull request #364 from dachary/wip-5213
unit tests for PGLog::proc_replica_log

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-06-19 15:59:39 -07:00
athanatos
9056402578 Merge pull request #366 from dachary/wip-5398
PGLog::rewind_divergent_log must not call mark_dirty_from on end()

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-06-19 15:57:28 -07:00
Samuel Just
8a5d989c39 FileStore: get_index prior to taking fdcache_lock in lfn_unlink
We take the fdcache_lock while holding onto index objects
elsewhere in the code.

Fixes: #5389
Reviewed-by: David Zafman <david.zafman@inktank.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-06-19 15:24:51 -07:00
Loic Dachary
09e869a4c4 PGLog::rewind_divergent_log must not call mark_dirty_from on end()
PGLog::rewind_divergent_log is dereferencing iterator "p" though it is
already past the end of its container. When entering the loop for the
first time, p is log.log.end() and must not be dereferenced.

mark_dirty_from must only be called after p--. It
will not rewind past begin() because of the

  if (p == log.log.begin())

test above.

http://tracker.ceph.com/issues/5398 fixes #5398

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-06-19 22:50:30 +02:00
Loic Dachary
4d77443d87 unit tests for PGLog::proc_replica_log
The tests covers 100% of the LOC of proc_replica_log. It is broken down
in 7 cases to enumerate all the situations it must address. Each case
is isolated in a independant code block where the conditions are
reproduced.

All tests are done on omissing and oinfo because they are the only
data structures that can be modified by proc_replica_log.

The first case is a noop and checks that only last_complete gets
updated when there are no logs.

The following case includes entries that are supposed to be ignored (
x7, x8 and xa ), however this is not an actual proof that the code
ignoring them is actually run : it only shows in the code coverage
report.

   The log entry (1,3) modifies the object x9 but the olog entry
   (2,3) deletes it : log is authoritative and the object is added
   to missing. x7 is divergent and ignored. x8 has a more recent
   version in the log and the olog entry is ignored. xa is past
   last_backfill and ignored.

The other cases are a variation of the first case with minimal changes
to make them easier to understand and adapt. For instance most of them
start with a tail that is the same ( object with hash x5 and both at
version 1,1 ).

http://tracker.ceph.com/issues/5213 refs #5213

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-06-19 20:52:23 +02:00
Loic Dachary
e11cc1c8c3 add constness to PGLog::proc_replica_log
The function is made const by replacing a single call to log.objects[]
with log.objects.find. The olog argument is also a const and does not
require any change.

http://tracker.ceph.com/issues/5213 refs #5213

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-06-19 20:52:23 +02:00
Sage Weil
02b3c55265 Merge pull request #342 from ceph/wip-mon
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-06-19 11:33:28 -07:00
Sage Weil
392a8e21f8 mon/PaxosService: not active during paxos UPDATING_PREVIOUS
Treat this as an extension of the recovery process, e.g.

 RECOVERING -> ACTIVE
or
 RECOVERING -> UPDATING_PREVIOUS -> ACTIVE

and we are not active until we get to "the end" in both cases.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
ee34a21960 mon: simplify states
- make states mutually exclusive (an enum)
- rename locked -> updating_previous
- set state prior to begin() to simplify things a bit

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
ec2ea86ed5 mon/Paxos: not readable when LOCKED
If we are re-proposing a previously accepted value from a previous quorum,
we should not consider it readable, because it is possible it was exposed
to clients as committed (2/3 accepted) but not recored to be committed, and
we do not want to expose old state as readable when new state was
previously readable.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
7b7ea8e30e mon/Paxos: cleanup: drop unused PREPARING state bit
This is never set when we block, and nobody looks at it.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
f985de28f8 mon/PaxosService: simplify is_writeable
Recast this in terms of paxos check + our conditions, and make it
match wait_for_writeable().

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
3aa61a0beb mon/PaxosService: simplify readable check
Recast this in terms of the paxos check and our additional conditions,
which match wait_for_readable().

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
a9e9c58f32 mon/MonmapMonitor: remove some dead code
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:05 -07:00
Sage Weil
7229da8a4d mon/MonmapMonitor: fix typo in check to remove mkfs info
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Sage Weil
e56580234a mon: fix whitespace
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Sage Weil
439e79969e mon: debug proposal timers
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Sage Weil
e832e76a4a mon: simplify Monitor::init_paxos()
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Sage Weil
e68b1bd36e mon/Paxos: go active *after* refreshing
The update_from_paxos() methods occasionally like to trigger new activity.
As long as they check is_readable() and is_writeable(), they will defer
until we go active and that activity will happen in the normal callbacks.

This fixes the problem where we active but is_writeable() is still false,
triggered by PGMonitor::check_osd_map().

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Sage Weil
dc83430124 mon: safely signal bootstrap from MonmapMonitor::update_from_paxos()
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Sage Weil
a42d7582f8 mon/Paxos: do paxos refresh in finish_proposal; and refactor
Do the paxos refresh inside finish_proposal, ordered *after* the leader
assertion so that MonmapMonitor::update_from_paxos() calling bootstrap()
does not kill us.

Also, remove unnecessary finish_queued_proposal() and move the logic inline
where the bad leader assertion is obvious.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:04 -07:00
Joao Eduardo Luis
2fccb300bd mon/PaxosService: cache {first,last}_committed
Refresh the in-memory values when we are told the on-disk paxos state
may have changed.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-06-19 11:27:03 -07:00
Sage Weil
d941363d6e mon: no need to refresh from _active
The refresh is done explicitly by the monitor, independent of the more
fragile PaxosService callbacks.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:03 -07:00
Sage Weil
03014a4ecc mon: remove unnecessary update_from_paxos calls
The refresh() will do this when the state changes; no need to
opportunistically call this method all of the time.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-19 11:27:03 -07:00