Commit Graph

12525 Commits

Author SHA1 Message Date
Colin Patrick McCabe
6722b0c85d rpm: add pkgconfig to BuildRequires
You can't build without pkgconfig.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-09 11:18:32 -08:00
Colin Patrick McCabe
9df18d1984 rpm: set files-attr for radosgw
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-09 10:28:39 -08:00
Sage Weil
b4264fbbdc filejournal: reset last_commited_seq if we find journal to be invalid
If we read an event that's later than our expected entry, we set read_pos
to -1 and discard the journal.  If that happens we also need to reset
last_committed_seq to avoid a crash like

2010-12-08 17:04:39.246950 7f269d138910 journal commit_finish thru 16904
2010-12-08 17:04:39.246961 7f269d138910 journal committed_thru 16904 < last_committed_seq 37778589
os/FileJournal.cc: In function 'virtual void FileJournal::committed_thru(uint64_t)':
os/FileJournal.cc:854: FAILED assert(seq >= last_committed_seq)
 ceph version 0.24~rc (commit:fe10300317383ec29948d7dbe3cb31b3aa277e3c)
 1: (FileJournal::committed_thru(unsigned long)+0xad) [0x588e7d]
 2: (JournalingObjectStore::commit_finish()+0x8c) [0x57f2ec]
 3: (FileStore::sync_entry()+0xcff) [0x5764cf]
 4: (FileStore::SyncThread::entry()+0xd) [0x506d9d]
 5: (Thread::_entry_func(void*)+0xa) [0x4790ba]
 6: /lib/libpthread.so.0 [0x7f26a2f8373a]
 7: (clone()+0x6d) [0x7f26a1c2569d]

Fixes #631

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-08 18:10:49 -08:00
Yehuda Sadeh
cc78bbf16e objecter: create a new op for resending lingering requests 2010-12-08 16:02:52 -08:00
Colin Patrick McCabe
027d5bfdca logger: tweak cmon log output a bit
Make the output of cmon on stderr a little bit less verbose.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-08 13:51:00 -08:00
Colin Patrick McCabe
fdc7414ecd logging: DoutStreambuf: handle daemonizing better
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-08 12:25:17 -08:00
Yehuda Sadeh
5cba1e6333 objecter: a few lingering fixes 2010-12-08 11:36:18 -08:00
Sage Weil
a9c098df47 mon: use helper for clock drift check; log relative instead of absolute time
Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-08 11:12:51 -08:00
Colin Patrick McCabe
986c2af4ae logging: debug.h: use DoutStreambuf
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-08 10:54:51 -08:00
Colin Patrick McCabe
c53ffafb5c logging: Remove _dout_check_log
_dout_check_log is unneeded, since every invocation of dout makes the
same check.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 23:27:19 -08:00
Colin Patrick McCabe
8fdd0f44a5 logging: debug.h: minor cleanup
Don't put std::ostream into the global namespace. Copyright update.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 23:17:48 -08:00
Colin Patrick McCabe
aeba6bca5d logging: eliminate dbeginl
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 22:52:21 -08:00
Colin Patrick McCabe
0f0cb46ad7 logging: Implement rank symlinks
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 19:09:18 -08:00
Colin Patrick McCabe
6c4a7d5810 logging: Support isym_path
Support instance symlinks, which are activated when we are using
g_conf.log_per_instance.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:54:03 -08:00
Colin Patrick McCabe
e597d02d42 logging: rename_output_file -> handle_pid_change
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:51:35 -08:00
Colin Patrick McCabe
116478a3be logging: _calculate_opath: use g_conf.log_dir
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:49:14 -08:00
Colin Patrick McCabe
ea3414d4dd logging: DoutStreambuf: better debug output
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:47:28 -08:00
Colin Patrick McCabe
627399f760 logging: create_symlink:sometimes use rel symlinks
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:45:21 -08:00
Colin Patrick McCabe
b00baab159 logging: implement get_dirname, move get_basename
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:43:49 -08:00
Colin Patrick McCabe
ef22366455 logging: fix normalize_relative
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 18:41:23 -08:00
Sage Weil
fe10300317 mds: sync->mix replica state is sync->mix(2)
When auth first moves to sync->mix,
 - auth sends AC_MIX to replicas
 - replicas go to sync->mix
 - replicas finish gather, send AC_SYNCACK, move to sync->mix(2)
 - auth gets all acks, sends AC_MIX again
 - replica moves to MIX

So any new replica should just get sync->mix(2), so that it is not confused
by the second AC_MIX.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:19 -08:00
Sage Weil
2000f69e99 mds: no not choose lock state on replicas
The lock state has already been set during rejoin.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:19 -08:00
Sage Weil
3825c4b87b mds: small rejoin cleanup
Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
9b9b86935e mds: rev mds cluster internal protocol
The lock encoding changed with the dirty bit on scatterlocks.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
2ea9b2d7db mds: fix replay of already-journaled requests
Check for already-completed tids for both retried and replayed requests.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
b5fd2e4d4e mds: open undef dirfrags during rejoin
Any invented dirfrags have a version of 0.  This will cause problems later
if we pre_dirty() anything in that dir because the dir version won't be
in sync (it'll be way too small).  Also, we can do that at any point,
e.g. when flushing dirty caps, and aren't allowed to delay, so we need to
load those dirfrags now.

In theory we could read only the fnode and not all the dentries, but we
may as well.  We should be more careful about memory that this patch is,
though.

Fixes #15.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
39c5933db0 mds: add missing try_clear_more() to scatterlock
Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
c681ed752f mds: explicitly pass scatterlock dirty flag to auth on gather
This ensures that if the replica is thinks it is flushing something the
auth will always do a scatter_writebehind.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
9bbb33b436 mds: send LOCKFLUSHED to trigger finish_flush on replicas
Since f741766a we have triggered start_flush and finish_flush on replicas.
The problem is that the finish_flush didn't always happen for the mix->lock
case: we sould start_flush when we sent the AC_LOCKACK, but could only
finish_flush if/when we got another SYNC or MIX.  If the primary stayed in
the LOCK state, we would keep our flushing flag.  That in turn causes
problems later when we try to eval_gather() (esp if we are auth at that
point?).

Fix this by sending an explicit AC_LOCKFLUSHED message to replicas after
we do a scatter_writebehind.  The replica will only set flushing if it
flushed dirty data, which forces scatter_writebehind, so we will always
get the LOCKFLUSHED to match.  Replicas that didn't flush will also get
it, but oh well.  We'd need to keep track which ones sent dirty data to
do that properly, though.

TODO: still need to verify that this is correct for rejoin.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
681b010fdb mds: clear EXPORTINGCAPS on export_reverse
We need to reverse the effects of encode_export_inode_caps(), which is just
the pin and state bit.

The original problem can be reproduced with
 - ceph tell mds 0 injectargs '--mds-kill-import-at 5'
 - restart mds
 - recovery completes successfully
 - wait for the subtree to be reexported
 - fail with bad EXPORTINGCAPS get in encode_export_inode_caps

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
f97660ff40 mds: fix LOOKUPHASH to avoid creating bogus replica CDir
We can't create the CDir if we are non-auth.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Sage Weil
4f6439945b mds: introduce rejoin_invent_dirfrag() helper
Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-07 16:44:18 -08:00
Colin Patrick McCabe
42464fb72a logging: Add symlink helper functions
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 15:47:24 -08:00
Colin Patrick McCabe
e2ba601be1 logger: fix EINTR handling
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 14:00:01 -08:00
Colin Patrick McCabe
bacdd49352 logging: rename_output_file: fix bug
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 13:57:38 -08:00
Colin Patrick McCabe
d70851ef01 logging: DoutStreambuf: Implement log-to-file
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 13:55:54 -08:00
Colin Patrick McCabe
952111451c logging: Add log_to_file option
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 13:46:40 -08:00
Colin Patrick McCabe
df5d4e6297 logging: DoutStreambuf improvements
Write to stdout_fileno directly rather than using a buffer, which we
would then have to flush. Fix a bug in the buffering of priorities.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 12:11:18 -08:00
Colin Patrick McCabe
1e2e4aa0f4 automake: in scripts, use sysconfdir as-is
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 10:56:05 -08:00
Colin Patrick McCabe
10b6887eae automake: in deb pkg, use --syconfdir=/etc
When building the debian packages, use --sysconfdir=/etc.

Also, don't fudge sysconfdir in the init-ceph script.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-07 10:48:19 -08:00
Colin Patrick McCabe
d4043e818f logging: add DoutStreambuf::set_prio
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 23:46:07 -08:00
Colin Patrick McCabe
6c7735f692 logging: DoutStreambuf must handle stdout + stderr
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 23:14:27 -08:00
Colin Patrick McCabe
12544a4912 logging: Add log_to_syslog option
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 23:03:43 -08:00
Sage Weil
57bcdc54d5 mkcephfs: require -k; update man page
Force users to specify keyring location; update man page accordingly.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-12-06 22:18:16 -08:00
Colin Patrick McCabe
5ac581df02 Rename SyslogStreambuf -> DoutStreambuf
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 15:57:56 -08:00
Colin Patrick McCabe
d1e0a2ae15 logging: debug.h: move some debug functions
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 15:38:14 -08:00
Colin Patrick McCabe
c94e0d2d38 logging: optimize with likely/unlikely macros
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 15:38:14 -08:00
Colin Patrick McCabe
9811fbd047 logging: Replace derr with dout
derr was really just an alias for STDERR. Unfortunately, after we call
daemonize, STDERR is connected to /dev/null. So just replace calls to
derr with dout so that our important messages don't get lost.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 15:38:14 -08:00
Colin Patrick McCabe
ab18aaec4a logging: add g_conf.clog_to_syslog
Add a new configuration option that allows you to send central log
messages to syslog.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 15:38:12 -08:00
Colin Patrick McCabe
ab61823e29 logging: LogEntry: don't pass enums by reference
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-12-06 15:35:52 -08:00