Commit Graph

15552 Commits

Author SHA1 Message Date
Colin Patrick McCabe
7cb1edd904 ceph_context: remove set_module_type
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 16:38:12 -07:00
Colin Patrick McCabe
15e3cb789a Merge branch 'wip-proflogger'
Conflicts:
	src/common/config.cc
	src/common/config.h
2011-07-20 16:26:18 -07:00
Colin Patrick McCabe
a2627f4a08 Client.cc: rename client_logger -> client_counters
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 16:23:35 -07:00
Colin Patrick McCabe
1b4733295a Rename ProfLogger to Perfcounters: part 2
Part 2: class names, variable names

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 16:18:58 -07:00
Greg Farnum
1b0d9ff228 Journaler: weaken a few asserts to let it pass errors back up.
Adjust MDLog to handle the possibility of errors appearing.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 15:58:40 -07:00
Greg Farnum
3d0c314f21 C_Gather: fix comment.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 15:58:40 -07:00
Greg Farnum
1212ceb013 Journaler: return errors from probing back to callers.
Adjust Dumper to handle the error codes. The other callers already do
so!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 15:58:40 -07:00
Greg Farnum
3bb63130b1 mds: handle EBLACKLISTED in MDCache::do_file_recover; assert on errors
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 15:58:40 -07:00
Greg Farnum
387e46b445 Filer: return error codes from probing up to the calling layer.
This is pretty limited; if you get multiple errors in one batch
of probes it'll only return the last one to get sent back (ENOENT is
excluded). But it's better than nothing.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 15:58:40 -07:00
Colin Patrick McCabe
1cf5211e72 Rename ProfLogger to PerfCounters
Part 1: file names

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 15:58:13 -07:00
Colin Patrick McCabe
9b1ad7b99e re-enable proflogger test
Re-enable the proflogger test. This time, it doesn't test the basic
socket functionality (that is what test/admin_socket.cc is for).

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 15:48:00 -07:00
Yehuda Sadeh
ffd92d20ca rgw: dump is_truncated when listing bucket 2011-07-20 15:41:00 -07:00
Colin Patrick McCabe
8d1193b9d3 test/admin_socket: test message 0
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 15:33:23 -07:00
Colin Patrick McCabe
b2a3899640 Move ProfLogger socket code into admin_socket
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 14:07:07 -07:00
Greg Farnum
73d66f677f mds: cinode: Add a little debugging and fix an output
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 13:22:19 -07:00
Greg Farnum
b2b44d47c9 mds: Drop locks and auth pins when waiting for freezingtree
In most cases we don't end up in this branch because there's an escape
if you already have an auth_pin on the ref in question. But when
you've got snapshots going on, you can process the request, block on
a lock state change, commit a snapshot change to the inode, and then
try to process the request again. On this second attempt, though,
you end up with a different ref which you don't have a pin on.
Deadlock and breakage ensues!
To fix, drop all locks and auth pins when you wait -- either
you don't have any to drop anyway, or you've got them on a previous
version of the inode that isn't useful anymore.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 13:22:19 -07:00
Greg Farnum
022c526180 uclient: correctly initialize mseq in flush_snaps.
Previously we set mseq=0 unconditionally; this was a mistake that
creeped in via bitrot. Instead, set mseq from the auth cap.
This bug meant that any inodes which had gotten migrated between
MDSes would have their flush_snap messages dropped by the MDS.
Resolves #1324.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-07-20 13:22:19 -07:00
Sage Weil
4adfd46630 mds: mark rmdir with slaves as a master in journal
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-20 13:17:23 -07:00
Sage Weil
573e972098 mds: fix rmdir slave commit, rollback
We need to set the slave commit callback to journal a commit or rollback
as needed.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-20 13:06:37 -07:00
Sage Weil
8692ed7df7 mds: simplify journaled subtree_map
We may have subtrees split locally due to migrations that are just getting
started or stopped.  Simplify the map we journal to disk.  Among other
things this makes the replay check simple (it can compare against the
current "live" map).

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-20 12:49:02 -07:00
Sage Weil
a3e47311b5 mds: add mds_debug_subtrees option
Verify that replayed ESubtreeMap events match what we have.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-20 12:49:02 -07:00
Sage Weil
4fe09c1cc9 mds: set LogEvent _start_off, drop unused _end_off
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-20 12:49:02 -07:00
Sage Weil
a25374e40a mds: witness rmdir when subtrees are on other hosts
If there is an rmdir with an empty subtree on another mds, we need to witness/
journal that on the dirfrag's auth mds so that replay correctly updates the
subtree map.

This is simpler than the rename witnesses (and the link/unlink ones) because
we aren't actually journaling a modification to any actual metadata; it's just
the subtree map that is changing.

The projection of the subtree map update needs work still, but that is also the
case for renames.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-20 12:46:57 -07:00
Yehuda Sadeh
0c913a8721 Merge branch 'wip-rgw-multithreaded' 2011-07-20 10:35:55 -07:00
Yehuda Sadeh
820b3f3684 rgw: setup as external server is configurable 2011-07-20 10:35:06 -07:00
Yehuda Sadeh
9eed12f57f rgw: clearer logging context 2011-07-20 10:01:17 -07:00
Colin Patrick McCabe
41f7acf5bc config.cc: fix comments
Fix comments that were reverted to an older version by commit
a51b983a8b.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-20 09:31:31 -07:00
Samuel Just
019955a1f4 PGMonitor.cc: add force_create_pg command to monitor
ceph pg force_create_pg <pgid> will now reset that pg's status to
creating to handle corner cases where no osd has that pg.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-19 17:59:41 -07:00
Yehuda Sadeh
43575c7ac7 rgw: fix concurrency issue 2011-07-19 14:02:11 -07:00
Colin Patrick McCabe
bd89bd9ef3 ProfLogger: add request codes
The client now starts by sending a 4-byte request code.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-19 12:08:39 -07:00
Colin Patrick McCabe
77fc561984 ProfLogger: use double quotes, add commas
Technically, JSON always uses double quotes, even though Javascript
often uses single quotes for strings. Also add a comma that had been
omitted.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-19 11:48:21 -07:00
Sage Weil
b3b06d02ab Merge branch 'stable' 2011-07-19 11:26:25 -07:00
Sage Weil
62e6f36a35 mds: clean up file flags to file mode translation
There was some seriously wrong and ancient cruft in there.  open(2)
specifies that one of O_RDONLY, O_WRONLY, and O_RDWR must always be
specified.  Drop all the crazy.

Linux VFS interprets O_WRONLY|O_RDWR as read+write, so we'll do the same.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-19 11:25:46 -07:00
Colin Patrick McCabe
f412942f1f ProfLogger: enable by default for daemons
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-19 11:08:34 -07:00
Colin Patrick McCabe
c3fa22faaf Put each ProfLogger's data into its own object
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-19 11:04:45 -07:00
Sage Weil
82faf3d6dd client: check for invalid file mode
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-19 11:03:22 -07:00
Sage Weil
ff82b60bd0 mds: check for invalid file mode
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-19 11:03:20 -07:00
Yehuda Sadeh
5e988b8e99 rgw: fix i386 compilation 2011-07-19 10:28:55 -07:00
Yehuda Sadeh
a51b983a8b config: fix compilation in certain environments 2011-07-19 09:43:50 -07:00
Sage Weil
626f0b1ffe mon: fix dump_json quoting
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-19 08:41:58 -07:00
Sage Weil
650f9f315b mds: fix O_APPEND file mode calculation
Currently

  fd = open("aaa",O_RDWR|O_CREAT|O_APPEND,0666);

gives us fmode of 1 instead of 3.  We should only infer O_WRONLY if no
other mode is explicitly specified.

Reported-by: Fyodor Ustinov <ufm@ufm.su>
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-19 08:33:37 -07:00
Colin Patrick McCabe
80c206427d ProfLogger: don't start threads until it's safe
Introduce a new configuration variable, internal_safe_to_start_threads.
This will be set by common_init_finish once it is safe to start threads.
That will trigger callbacks for any configuration observers listening
for this event.

This is used by ProfLogger to hold off on starting the UNIX domain
socket thread until it is safe to do so.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 17:38:43 -07:00
Colin Patrick McCabe
3286048b18 vstart.sh: use out/proflog.$name
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 17:25:48 -07:00
Colin Patrick McCabe
daeffbd5c2 proflogger: Fix compiler warnings
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 17:21:37 -07:00
Colin Patrick McCabe
64d3314237 proflogger: test multiple profloggers
Test multiple profloggers, and removing profloggers.

Add a clear function which removes all profloggers.

Make the order in which profloggers are output to JSON determininstic.

Const cleanup on get() methods.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 17:18:52 -07:00
Yehuda Sadeh
51039da1a6 rgw: use thread pool for rgw work 2011-07-18 17:14:25 -07:00
Colin Patrick McCabe
48769e2244 test/profloger: test fl_avg variables
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 16:53:48 -07:00
Colin Patrick McCabe
6454ecc9bc test/proflogger: test JSON formatting a bit
Test JSON formatting a bit. Use single quotes in our JSON because that
seems to make things easier.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 16:49:04 -07:00
Colin Patrick McCabe
cbeedeb176 proflogger: Unlink our UNIX domain sockets on exit
When we're exiting normally, we ought to unlink our UNIX domain sockets.

Also fix a bug in ProfLogThread, and enable some more tests.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-18 16:33:49 -07:00
Sage Weil
59a8e65e47 mon: pg dump_json
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-07-18 16:31:24 -07:00