Commit Graph

18397 Commits

Author SHA1 Message Date
Yehuda Sadeh
e73ab2cce1 Merge branch 'master' into wip-swift-acls 2012-02-28 13:31:09 -08:00
Yehuda Sadeh
bc80ba1f26 rgw: fix swift bucket acl verification
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
2012-02-28 13:29:30 -08:00
Yehuda Sadeh
cc93518024 rgw: implement swift public group
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
2012-02-28 12:37:27 -08:00
Greg Farnum
d10e1f46df mon: fix slurp_latest to fill in any missing incrementals
Fixes #1789.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-02-28 12:29:05 -08:00
Sage Weil
7b48cca184 test_osd_types: fix unit test for new pg_t::is_split() prototype
Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-28 09:33:18 -08:00
Sage Weil
edd35c04b4 osd: drop useless ENOMEM check
new throws exception; doesn't return NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-28 09:26:04 -08:00
Sage Weil
f317028f42 doc: beginnings of documentation of stuck pgs and pg states
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
Reviewed-by: Sage Weil <sage@newdream.net>
2012-02-27 15:41:57 -08:00
Sage Weil
f02195b40f Merge branch 'wip-split2'
Reviewed-by: Samuel Just <samuel.just@dreamhost.com>
2012-02-27 14:37:41 -08:00
Sage Weil
b6a04174bc osd: pg_t::is_split(): make children out param a pointer, and optional
Also unit test it.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 14:35:21 -08:00
Sage Weil
85ed06e973 osd: bypass split code
Until it is fully implemented.  It's also disabled in the monitor
currently, but just in case it gets into the OSDMap, do nothing for now.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 14:18:21 -08:00
Sage Weil
15d5324904 osd: fix pg locking flags
Two things we need to handle:

 - callers who already hold map_lock (split_pg())
 - callers who already hold another pg->lock, and want to skip the lockdep
   check for this one.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 14:16:27 -08:00
Sage Weil
fc7b11a9ee osd: partially refactor pg split
This partially refactors the OSD split code to do the split synchronously
when processing a new OSDMap.  It is incomplete in that it does not yet
do anything useful for the PG.  The full solution needs to:

- Do the split synchronously when applying the map update.
- Reset the parent pg so that it repeers.  This will cause problems until
  we consistently consider this a new interval when looking backwards in
  time; this needs to be fixed.  Anybody doing generate_past_intervals()
  or similar will need to consider a split/merge event as an interval
  boundary.
- The recovery state machine should trigger appropriately when this
  happens.
- The old PG that was split should probably be handle identically to the
  new children.  That means deleting the old PG instance and creating a new
  PG object for the newly-split child.  Ditto for merge.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 14:04:22 -08:00
Sage Weil
d9cf33223e osd: implement pg_t::is_split()
Test to determine if a pg has split between two pool sizes, and if so,
what its children are.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 11:44:24 -08:00
Sage Weil
6a081888fd osd: factor hobject key into child pgid calc during split
When we calculate the object's new pg, take the locator key into
consideration, to avoid a crash like

osd/OSD.cc: In function 'void OSD::split_pg(PG*, std::map<pg_t, PG*>&,ObjectStore::Transaction&)' thread 7fe3df8c4700 time 2012-02-20 18:22:19.900886
osd/OSD.cc: 4066: FAILED assert(child)

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 11:44:24 -08:00
Sage Weil
ee4d99099f journaler: log on unexpected objecter error
This will help with #2110, #1796, #1640.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-27 11:39:53 -08:00
Sage Weil
91b119a064 osd: fix recursive map_lock via check_replay_queue()
Also drop activate_pg() helper while we're at it, so it's clear that we
are the only user.

recursive lock of OSD::map_lock (33)
 ceph version 0.42-146-g7ad35ce (commit:7ad35ce489cc5f9169eb838e1196fa2ca4d6e985)
2012-02-24 12:30:16.541416 1: (PG::lock(bool)+0x2a) [0xa09348]
2012-02-24 12:30:16.541424 2: (OSD::_lookup_lock_pg(pg_t)+0xbd) [0x84b8df]
2012-02-24 12:30:16.541431 3: (OSD::activate_pg(pg_t, utime_t)+0x9f) [0x87463b]
2012-02-24 12:30:16.541442 4: (OSD::check_replay_queue()+0x12f) [0x87452d]
2012-02-24 12:30:16.541450 5: (OSD::tick()+0x23c) [0x8535ea]
2012-02-24 12:30:16.541456 6: (OSD::C_Tick::finish(int)+0x1f) [0x881671]
2012-02-24 12:30:16.541462 7: (SafeTimer::timer_thread()+0x2d5) [0x8f8211]
2012-02-24 12:30:16.541468 8: (SafeTimerThread::entry()+0x1c) [0x8f923c]
2012-02-24 12:30:16.541475 9: (Thread::_entry_func(void*)+0x23) [0x9c8109]
2012-02-24 12:30:16.541485 10: (()+0x68ba) [0x7f9dbed838ba]
2012-02-24 12:30:16.541491 11: (clone()+0x6d) [0x7f9dbd66f02d]
2012-02-24 12:30:16.541495 common/lockdep.cc: In function 'int lockdep_will_lock(const char*, int)' thread 7f9db9d98700 time 2012-02-24 12:30:16.541504

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Reviewed-by: Sam Just <samuel.just@dreamhost.com>
2012-02-27 09:56:21 -08:00
Sage Weil
402ece5e31 init-ceph: stick with /var/run for the time being
/run isn't present on older systems.  Stick with the old location until it
is more pervasive, or we add an autoconf option to control it.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-26 20:56:05 -08:00
Laszlo Boszormenyi
41295b584a debian: /var/run/ceph -> /run/ceph
/run/ceph should exists for creating UNIX domain sockets
ceph uses UNIX domain sockets for internal communication. Create their
directory on startup as /run is on a virtual filesystem.

Last-Update: <2012-02-26>
Bug-Debian: http://bugs.debian.org/660238
Forwarded: <ceph-devel@vger.kernel.org>
Signed-off-by: Laszlo Boszormenyi (GCS) <gcs@debian.hu>
2012-02-26 20:47:53 -08:00
Laszlo Boszormenyi
0d8b5756e1 debian: build-{indep,arch}
Signed-off-by: Laszlo Boszormenyi <gcs@debian.hu>
2012-02-26 20:45:52 -08:00
Laszlo Boszormenyi
3ad6ccb4a6 debian: sdparm|hdparm, new standards version
Signed-off-by: Laszlo Boszormenyi <gcs@debian.hu>
2012-02-26 20:45:06 -08:00
Yehuda Sadeh
266902a993 rgw: initialize bucket_id in bucket structure
might make valgrind a little bit less noisy.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
2012-02-24 17:01:32 -08:00
Sage Weil
f8f6e4d850 rgw: _exit(0) on SIGTERM
We need to do something a bit smarter to get coverage information, but this
is a start.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 15:32:07 -08:00
Sage Weil
708be0a5ab Merge remote branch 'gh/wip-crush-adjust'
Reviewed-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
2012-02-24 13:52:32 -08:00
Sage Weil
b0feba56bd Merge remote branch 'gh/wip-mds-resetter'
Reviewed-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-02-24 13:48:06 -08:00
Sage Weil
5c6e8b3795 Merge branch 'wip-pg-query'
Reviewed-by: Samuel Just <samuel.just@dreamhost.com>
2012-02-24 13:43:43 -08:00
Sage Weil
008ce6b2d4 Merge branch 'stable' 2012-02-24 13:22:49 -08:00
Sage Weil
732f3ec94e v0.42.2 2012-02-24 13:00:39 -08:00
Sage Weil
321ba67f1b Merge remote-tracking branch 'gh/stable' into stable 2012-02-24 13:00:33 -08:00
Sage Weil
be761149f5 Merge branch 'stable' 2012-02-24 12:54:41 -08:00
Sage Weil
d85ed91cc7 osd: fix array index
Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 12:40:50 -08:00
Sage Weil
722e9e59cb lockdep: don't make noise on startup
Who cares!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 12:40:34 -08:00
Sage Weil
fdaed0a7a2 formatter: fix trailing dump_stream()
Flush a previous dump_stream() if it was the last thing prior to a
close_section().

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 12:40:13 -08:00
Sage Weil
7ad35ce489 osd: include timestamps in state json dumps
Include the time we entered this state in the dump.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 12:04:29 -08:00
Sage Weil
185c6b1fd8 Merge branch 'wip-2007'
Reviewed-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-02-24 12:00:00 -08:00
Sage Weil
e22adac224 osd: use blocks for readability in list_missing
Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 11:59:20 -08:00
Sage Weil
6d90a6ddc8 osd: dump recovery_state states in json
Use a formatter.  Present a vector of states, inner to outer.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 11:33:57 -08:00
Sage Weil
a7c8bfbea0 osd: query Peering substates
Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 11:33:57 -08:00
Sage Weil
e22a45a18d osd: query recovery state machine
For now, just append this to the end of the pg <pgid> query json dump.
We definitely want to do something smarter here, but I'm not sure whether
json or plaintext is the way to go.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 11:33:57 -08:00
Sage Weil
d3b203af16 osd: add tunable for number of records in osd command replies
e.g., 'pg <pgid> list_missing [offset]'.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 11:24:45 -08:00
Sage Weil
91fbc687eb osd: 'pg <pgid> list_missing <json hobject_t offset>'
Dump missing objects in json.  If more key is non-zero, user should ask for
more by passing the last object as the offset for the next request.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 11:24:44 -08:00
Sage Weil
6c257c4d12 hobject_t: decode json
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 11:24:44 -08:00
Sage Weil
2677c72fde add libjson_spirit.la
This is lightweight and relies on boost spirit, which we already use, so
there are no new dependencies.

There were some other libraries that also looked good, but they weren't
already packages for existing Debian distros like squeeze or even wheezy.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 11:24:44 -08:00
Sage Weil
0361a3c431 osd: pass in data to do_command
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 11:24:44 -08:00
Sage Weil
c9416e6184 osd: 'tell osd.N mark_unfound_lost revert' -> 'pg <pgid> mark_unfound_lost revert'
More consistent interface.

Fixes: #2030
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Reviewed-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-02-24 11:24:04 -08:00
Sage Weil
6403852426 lockdep: warn on stderr (via derr), not stdout
Otherwise we screw up ceph-conf output and the like.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-24 11:15:12 -08:00
Sage Weil
804f243b2a do_autogen.sh: -T for --without-tcmalloc
Signed-off-by: Sage Weil <sage@newdream.net>
2012-02-24 11:15:04 -08:00
Yehuda Sadeh
5efa821c22 rgw: swift read acls allow bucket listing
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
2012-02-23 20:22:03 -08:00
Yehuda Sadeh
f09fb870c6 rgw: fix swift acl enforcement
we'll also need to make it so that swift read acls allow bucket listing

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
2012-02-23 20:11:08 -08:00
Sage Weil
7c7349ef5d ceph: fix help.t
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-02-23 18:58:35 -08:00
Sage Weil
c3e1291da9 v0.42.1 2012-02-23 18:48:57 -08:00