Commit Graph

34654 Commits

Author SHA1 Message Date
Sage Weil
f80ed26d24 ceph-disk: move fs mount probe into a helper
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-12 17:25:42 -07:00
Sage Weil
6c77f5f2f9 ceph-disk: use partition type UUIDs, and blkid
Use blkid to give us the GPT partition type.  This lets us distinguish
between dmcrypt and non-dmcrypt partitions.  Fake it if blkid doesn't
give us what we want and try with sgdisk.  This isn't perfect (it can't
tell between dmcrypt and not dmcrypt), but such is life, and we are better
off than before.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-12 17:25:10 -07:00
Sage Weil
1088d6cd11 ceph-disk: fix log syntax error
File "/usr/sbin/ceph-disk", line 303, in command_check_call
    LOG.info('Running command: %s' % ' '.join(arguments))
TypeError: sequence item 2: expected string, NoneType found

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-12 13:53:16 -07:00
Sage Weil
2edf01ffa4 Revert "Fix for bug #6700"
This reverts commit 673394702b.

This appears to break things when the journal and data disk are *not* the same.
And I can't seem to reproduce the original failure...

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-11 15:58:15 -07:00
Sage Weil
d6e6ba198e ceph-disk: fix verify_no_in_use check
We only need to verify that partitions aren't in use when we want to
consume the whole device (osd data), not when we want to create an
additional partition for ourselves (osd journal).

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-11 15:57:56 -07:00
Greg Farnum
84c9654704 OSD: add require_same_peer_inst(OpRequestRef&,OSDMap&) helper
Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit e99acf9810)
2014-08-08 22:59:25 -07:00
Greg Farnum
4a0c9417c1 OSD: introduce require_self_aliveness(OpRequestRef&,epoch_t) function
Take the self-aliveness checks out of require_same_or_newer_map() and use
the new function for that and for require_up_osd_peer().

Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit e179e9227b)
2014-08-08 22:59:17 -07:00
Greg Farnum
48c9b38414 OSD: use OpRequestRef& for a few require_* functions
Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit eb2f1ea2c3)
2014-08-08 22:59:14 -07:00
Greg Farnum
f86bf1d8be OSD: introduce require_up_osd_peer() function for gating replica ops
This checks both that a Message originates from an OSD, and that the OSD
is up in the given map epoch.
We use it in handle_replica_op so that we don't inadvertently add operations
from down peers, who might or might not know it.

Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit ccd0eec501)
2014-08-08 22:59:09 -07:00
Sage Weil
790f4c2345 Merge pull request #2228 from ceph/wip-pg-epoch
osd: fix pg epoch floor tracking

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-08 18:17:27 -07:00
Sage Weil
a52a855f6c osd: fix pg epoch floor tracking
If you call erase() on a multiset it will delete all instances of a value;
we only want to delete one of them.  Fix this by passing an iterator.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-07 17:42:06 -07:00
Samuel Just
dceab8dc49 Merge pull request #2198 from ceph/wip-8998
fix OSD SEGV in heartbeat()

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-04 15:12:25 -07:00
Sage Weil
8dcfbd8e3b osd: simplify dout_prefix macros
Use a get_osdmap_epoch() helper that is a bit lighter weight (by avoiding
copying around an OSDMapRef).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-04 15:01:15 -07:00
Sage Weil
80829d7805 osd: reorder OSDService methods under proper dout_prefix macro
The dout_prefix for OSDService uses get_osdmap() to grab a shared_ptr for
the epoch printout.  The OSD one does not, and is not safe to run in all
thread contexts.

In particular, update_osd_stat() is run by the heartbeat thread and can
race with the shared_ptr itself being updated with a new map.

Ironically, if this were simply an OSDMap*, there would be no race since
the pointer is a single word and updates atomically.

Fix this, and any similar issues, by moving the OSDService methods up in
OSD.cc so that they use the safe dout macro.

Fixes: #8998
Backport: firefly (in a minimal form, I think!)
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-04 14:57:28 -07:00
Samuel Just
3897f09a70 Merge pull request #2192 from ceph/wip-8891
msg/SimpleMessenger: drop msgr lock when joining a Pipe

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-04 10:30:25 -07:00
Yehuda Sadeh
7b2c8b3310 cls_rgw: fix object name of objects removed on object creation
Fixes: #8972
Backport: firefly, dumpling

Reported-by: Patrycja Szabłowska <szablowska.patrycja@gmail.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 0f8929a68a)
2014-08-04 10:04:55 -07:00
Yehuda Sadeh
8519e9ab06 rgw: need to pass need_to_wait for throttle_data()
need_to_wait wasn't passed into processor->throttle_data(). This was
broken in fix for #8937.

CID 1229541:    (PW.PARAM_SET_BUT_NOT_USED)

Backport: firefly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e93818df33)
2014-08-04 10:03:40 -07:00
Yehuda Sadeh
062062479a rgw: call processor->handle_data() again if needed
Fixes: #8937

Following the fix to #8928 we end up accumulating pending data that
needs to be written. Beforehand it was working fine because we were
feeding it with the exact amount of bytes we were writing.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 0553890e79)
2014-08-04 09:52:51 -07:00
Loic Dachary
b22a8a9e60 Merge pull request #2193 from ceph/wip-ceph-conf
ceph-conf: flush log on exit

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-08-04 18:28:49 +02:00
Sage Weil
3230060f07 ceph-conf: flush log on exit
This makes it deterministic whether we output

2014-08-03 20:59:45.482614 4036c80 -1 did not load config file, using default settings.

or not, and will make the unit tests stop intermittently failing.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-03 21:00:51 -07:00
Sage Weil
98997f3b22 msg/SimpleMessenger: drop msgr lock when joining a Pipe
Avoid this deadlock:

- a fault
- delay thread entry gets a fast dispatch message
 - drops delay_lock
 - calls into fast_dispatch
- reaper tries to reap the pipe
 - pipe->join()
  - delay_thread->join()
   - blocks waiting for delay_thread to exit
- delay thread / fast dispatch blocks on msgr->lock trying to mark_down

The solution is to drop the msgr lock while joining the thread.  This will
allow the join() to complete.  Adjust the reaper thread to recheck the
exit condition since the lock may have been dropped.  The other two callers
do not care.

Fixes: #8891
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-03 18:26:34 -07:00
Sage Weil
e36babc825 os/MemStore: fix lock leak
CID 1228868 (#2-1 of 2): Missing unlock (LOCK)
12. missing_unlock: Returning without unlocking oc->lock.L.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-03 11:23:33 -07:00
Samuel Just
f335c73b12 Merge pull request #2172 from ceph/wip-8714
osd: prevent old clients from using tiered pools

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-01 09:57:24 -07:00
Gregory Farnum
a00777f428 Merge pull request #2190 from ceph/wip-osd-leaks
osd: do not leak Session* ref in _send_boot()

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-08-01 10:08:16 -04:00
Gregory Farnum
fc2d18bb1e Merge pull request #2182 from ceph/wip-round
use llrintl when converting double to micro

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-08-01 08:11:08 -04:00
Sage Weil
04037ef9dd osd: do not leak Session* ref in _send_boot()
The get_priv() call returns a ref; make sure we drop it if it exists.
This doesn't happen on every run because usually it is NULL and we take
the other path; it's only after the OSD has been marked down that we reach
the second path.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 20:59:49 -07:00
Sage Weil
80911736bd use llrintl when converting double to micro
This avoids rounding error (noticeable on i386).

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 15:39:40 -07:00
Samuel Just
fe99b25317 Merge pull request #2179 from ceph/wip-8982
osd: fix cache pools without hit_sets configured

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-07-31 11:28:04 -07:00
Sage Weil
383536a2ef mon/OSDMonitor: warn when cache pools do not have hit_sets configured
Give users a clue when cache pools are enabled but the hit_set is not
configured.  Note that technically this will work, but not well, so for
now let's just steer them away.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:19 -07:00
Sage Weil
caf554bf2f osd/ReplicatedPG: improve agent_choose_mode args
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:19 -07:00
Sage Weil
ce4e559d29 vstart.sh: limit open files
Most users only get 1024 fds.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:19 -07:00
Sage Weil
ea4996d3ef osd/ReplicatedPG: evict blindly if there is no hit_set
If there is no hit set for a PG, blindly evict objects.  This avoids an
assert(hit_set) in agent_estimate_atime_temp().

Fixes: #8982
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:18 -07:00
Sage Weil
5d1c76f641 osd/ReplicatedPG: check agent_mode if agent is enabled but hit_sets aren't
It is probably not a good idea to try to run the tiering agent without a
hit_set to inform its actions, but it is technically possible.  For
example, one could simply blindly evict when we reach the full point.
However, this doesn't work because the agent mode is guarded by a hit_set
check, even though agent_setup() is not.  Fix that.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:18 -07:00
Gregory Farnum
f3d1f396f9 Merge pull request #2173 from ceph/wip-8969
fix i386 atomics

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-07-31 08:59:00 -04:00
Sage Weil
5c6a051220 Merge pull request #2162 from ceph/wip-8586
rgw: fix crash in swift CORS preflight request

Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-30 20:57:06 -07:00
Sage Weil
9df9d28009 mon/OSDMonitor: fix i386 floating point rounding error
This would round .123 to .122999 on i386; now it does not.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-30 17:50:39 -07:00
Josh Durgin
9eda0bbe1c Merge pull request #2171 from ceph/wip-cli-integration
fix rbd cli-integratoin tests

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-07-30 16:57:38 -07:00
Sage Weil
96863128e6 atomic: fix read() on i386, clean up types
Among other things, fixes #8969

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-30 14:52:06 -07:00
Sage Weil
0190df5305 osd: prevent old clients from using tiered pools
If the client is old and doesn't understand tiering, don't let them use a
tiered pool.  Reply with EOPNOTSUPP.

Fixes: #8714
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-30 13:57:34 -07:00
Sage Weil
605064dc68 test/cli-integration/rbd: fix trailing space
Newer versions of json.tool remove the trailing ' ' after the comma.  Add
it back in with sed so that the .t works on both old and new versions, and
so that we don't have to remove the trailing spaces from all of the test
cases.

Backport: firefly
Fixes: #8920
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-30 13:41:16 -07:00
Sage Weil
6e6fc23c7e v0.83
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIcBAABAgAGBQJT2AdNAAoJEH6/3V0X7TFt+5EP/iLOUg5o6aqBa/7yUNwtgZEb
 6cm1h8bWJpigP51oHGNyoeS4PnYWQ7DfvwKL/TDP8268g/o/w0DRvSnCZopHFos9
 j6Ci/VE9ag9uQNqW+NOph13k3fjf5KetzM5g/q/Ay4dKVCS2+9uABfosql9RNZa6
 ojhGKf2BtMgswfemq/0XWc49Ptimox5G/ntR+/xYm0s906q5wB1Y9Tvh2PNZo1Y1
 wL2qy9UnmonBLGIu0BIStKnub57VHCYbNqV6fl3W+Oct9f0znYPCqnRVUb2lw3Ie
 4KciilzteQCfurCCI7CQFmNEKCVpPlujiKo/q8CKIDgbwkMcUntCmW9QcmH3BzC5
 czYr695aokE+dt+MICRY+sIREY5achXynb7wnSon9JI8qrCUQ0o4fHQ1AZOio7V6
 +zcCdussqSeEXOoVUlNS5eGrzbY1kqrFcXY18WiCy6nadLapuUQTtJ5QALQyJ5TW
 8TidkkU6h0V4sworwpM6tiDLfq2UQwZ5NuP8MGz9DtOjMDbLHSE6TrRug7Irjj41
 4AKdkSWMhuXljm/rEsOi54ZGRUhq2VZ2xpnUD0WR8r/3lAP1d2UnefFlrSZaCN4z
 bNcrCncK7wre2UUyDQ/qJ+S808XPUPQaohbmb3Eg+Hr0mbkiJXYdNNyrUzL3wnkr
 E3YL+8sapzZKn9zOxNQ3
 =ujRc
 -----END PGP SIGNATURE-----

Merge tag 'v0.83'

v0.83
2014-07-29 16:23:12 -07:00
John Spray
440c820cce Merge pull request #2161 from ceph/wip-jcsp-test
Reviewed-by: Greg Farnum greg@inktank.com
2014-07-29 23:55:30 +01:00
Yehuda Sadeh
18ea2a8697 rgw: fix crash in swift CORS preflight request
Fixes: #8586

This fixes error handling, in accordance with commit 6af5a537 that fixed
the same issue for the S3 case.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-07-29 15:33:03 -07:00
John Spray
6bb3aeafcf mds: remove some rogue "using namespace std;"
Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:43 +01:00
John Spray
c283ad4ba5 mds: handle replaying old format journals
To get back to the reformatting procedure that otherwise
occurs during MDLog::open, introduce an MDLog::reopen call
that MDS can use in the standbyreplay->standby transition
for the special case where the journal is old.

Fixes: #8869

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:43 +01:00
John Spray
07665ec4b3 mds: introduce explicit DaemonState instead of int
Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:43 +01:00
John Spray
6b004f19da mds: refactor MDS boot
* Make boot_start private.
* Define boot stages in enum, replace int with type.
* Merge steps 0 and 1, 0 always fell through to 1.
* starting_done was only ever reached by a fall through
  from the previous step, so call it directly from there.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:18 +01:00
John Spray
6832ec041a mds: make MDS::replay_done clearer
... and add some assertions.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:05 +01:00
John Spray
e587088918 mds: remove unused purge_prealloc_ino
Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:05 +01:00
John Spray
6be80873c3 mds: separate inode recovery queue from MDCache
Refactor to:
* have somewhere to put some logic for doing
  background recovery in future.
* trim a few lines from the oversized MDCache.cc
  whereever we can.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-29 22:32:05 +01:00