Commit Graph

22510 Commits

Author SHA1 Message Date
Yehuda Sadeh
0beeb47c43 rgw: document ops logging setup
Fixes: #3530

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-26 15:55:00 -08:00
Yehuda Sadeh
6bc32b2008 rgw: usage REST api handles cateogories
Fixes: #3528
The usage REST api was missing the categories filter.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-26 15:55:00 -08:00
Alexandre Oliva
b1c71088bb logrotate on systems without invoke-rc.d
The which command doesn't output anything to stdout when it can't find
the given program name, and then [ -x ] passes.  Use the exit status
of which to tell whether the command exists, before testing whether
it's executable, to fix it.

Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br>
2012-11-26 15:04:08 -08:00
Alexandre Oliva
a37c34debd Search for srcdir/.git in check_version
Support srcdir != . looking for .git in srcdir when computing the ceph
release and git tag.

Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br>
2012-11-26 15:04:08 -08:00
Yehuda Sadeh
74b2a2d964 rgw: POST requests not default to init multipart upload
Fixes: #3516
We don't default to init multipart upload request when
getting S3 POST. This way when the request is not really
init multipart upload we'd end up sending a 405 response
instead of 500. Also, it's cleaner this way.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-26 12:29:40 -08:00
Noah Watkins
1f8c32347b java: add ceph_open_layout interface
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2012-11-26 11:15:47 -08:00
Noah Watkins
f0c608c0d6 client: add ceph_open_layout interface
Adds an interface identical to ceph_open() that takes additional
parameters specifying a file layout to use on new files.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-11-26 11:15:47 -08:00
Josh Durgin
365ba0600b qa: add script to run objectcacher tests
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-26 10:37:43 -08:00
Sage Weil
30669d6d87 Merge remote-tracking branch 'gh/wip-upstart' into next 2012-11-26 08:38:25 -08:00
Sage Weil
525f942edc init-ceph: do not make noise about missing devs
It is pretty normal not to include the devs line in the ceph.conf.  Do not
print/warn about it.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-26 08:37:45 -08:00
Sage Weil
bc32fc42d2 syncfs: check for __NR_syncfs too
Also make the filestore startup tell us *all* variants that are
supported, not just the first one.

Tested-by: Stefan Priebe <s.priebe@profihost.ag>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-25 13:29:52 -08:00
Sage Weil
6890675b87 monmap: fix crash from dup initial seed mons
Fix bug reproduced by

	-m hostname,ip_that_hosthname_resolves_to

Backport: argonaut
Reported-by: Drunkard Zhang <gongfan193@gmail.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-25 09:34:02 -08:00
Sage Weil
7602a05576 osdc/ObjectCacher: fix BufferHead leak on ENOENT
This was detected by fsstress over ceph-fuse under valgrind.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-24 10:05:52 -08:00
Sage Weil
8a03d50146 Merge remote-tracking branch 'gh/wip-mon-misc-fixes' into next 2012-11-24 09:16:13 -08:00
Danny Al-Gaaf
df550c9cce make mkcephfs and init-ceph osd filesystem handling more flexible
Remove btrfs specific keys and replace them by more generic
keys to be able to replace btrfs with e.g. xfs or ext4 easily.

Add new key to define the osd fs type: 'osd mkfs type', which can
get defined in the [osd] section for all OSDs.

Replaced config keys:
- 'btrfs devs' -> 'devs'
- 'btrfs path' -> 'fs path'
- 'btrfs options' -> 'osd mount options $fstype'

New config key:
- 'osd mkfs options $fstype': file system specific options for mkfs
- 'osd mkfs type': to define the filesystem for mkfs and also mount

Replaced in mkcephfs: --mkbtrfs with --mkfs

Replaced in init-ceph:
- --btrfs with --fsmount
- --nobtrfs with --nofsmount
- --btrfsumount with --fsumount

NOTE: old options from mkcephfs and init-ceph will still work, but
      get may removed in the future from the scripts.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2012-11-23 19:14:52 -08:00
Joao Eduardo Luis
96b82ebf87 mon: Monitor: wake up contexts based on paxos machine's state
When recovering the leader, only wake up a paxos machine's contexts if
the paxos machine is in a state that can handle said contexts.

Fixes: #3495

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-23 19:13:07 +00:00
Joao Eduardo Luis
3b061ab9d3 mon: AuthMonitor: increase log levels when logging secrets
Fixes: #3361

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-23 19:13:05 +00:00
Joao Eduardo Luis
7527a1ea6c auth: Keyring: increase log levels when logging secrets
Fixes: #3361

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-23 19:13:03 +00:00
Joao Eduardo Luis
deabdc8a10 auth: cephx: increase log levels when logging secrets
We understand that logging secrets may be useful when debugging the root
causes for auth issues. However, logging secrets is far from a good idea.
Therefore, just increase the log levels to a high enough value so that
most other debug infos can be obtained without even logging the secrets.
If one really wants to log the secrets, then setting --debug-auth 30 should
do the trick.

Fixes: #3361

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-23 19:13:01 +00:00
Joao Eduardo Luis
d6cf77dcbb crush: CrushWrapper: don't add item to a bucket with != type than wanted
We take little consideration about the type of the bucket we are adding
an item to. Although this works for the vast majority of cases, it was
also leaving room for silly little mistakes to become problematic and
leading a monitor to crash.

For instance, say that we ran:
  'ceph osd crush set 0 osd.0 1 root=foo row=foo'

If root 'foo' exists, then this will work and 'row=foo' will be ignored.
However, if there is no bucket named 'foo', then we would (in order)
create a bucket for row 'foo', adding osd.0 to it, and would then add
osd.0 to bucket 'foo' again -- remember, little consideration regarding
the bucket type was given.

This would trigger a monitor crash due to the recursion done in
'adjust_item_weight'. A solution to this problem is to make sure that we
do not allow specifying multiple buckets with the same name when adding
an item to crush. Not only solves our crash problem, but will also render
invalid any mistake when specifying the wrong bucket type (say, using
'row=bar' when in fact 'bar' is a rack).

Fixes: #3515

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-23 19:12:57 +00:00
Joao Eduardo Luis
95e1fe8822 mon: PGMonitor: check if pg exists when handling 'pg map <PG>'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-11-23 19:12:48 +00:00
Yehuda Sadeh
ab8327fec0 Merge remote-tracking branch 'origin/next' into next 2012-11-22 14:59:25 -08:00
Sage Weil
1c715a11f7 mds: child directory inherits SGID bit
Update the inode, not the local variable.

Reported-by: Giorgos Kappes <geokapp@gmail.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-22 13:53:29 -08:00
Yehuda Sadeh
3110e5ca42 Merge remote-tracking branch 'origin/next' into next 2012-11-22 12:57:33 -08:00
Yehuda Sadeh
a0e8452a09 Merge branch 'wip-opslog-socket2' into next
Conflicts:
	src/rgw/rgw_main.cc
2012-11-22 12:55:35 -08:00
Sage Weil
55081c2bea crush: prevent loops from insert_item
If the insertion would create a loop, return -EINVAL.

Fixes: #3515
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-22 09:17:34 -08:00
Dan Mick
b706945ae9 Try using syscall() for syncfs if not supported directly by glibc
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2012-11-22 08:50:44 -08:00
Yehuda Sadeh
189d073b25 rgw: fix usage accounting
bytes_sent and bytes_received are no longer accounted
in the req_state, needed to get them out of the ClientIO
object.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-21 14:19:36 -08:00
Josh Durgin
5126bd7085 rbd: make snap purge fail if removing a snap fails
Otherwise we report 100% removed when some may have failed.

Reported-by: Andrey Korolyov <andrey@xdel.ru>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-11-21 13:27:27 -08:00
Greg Farnum
820911b265 msgr: use get_myaddr and set_myaddr to set the nonce
We can't set it directly via get_myaddr as its a const ref. Dur.

Signed-off-by: Greg Farnum <greg@inktank.com>
2012-11-21 12:00:37 -08:00
Yehuda Sadeh
5c27bc42ba rgw: fix cleanup
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-21 12:09:06 -08:00
Yehuda Sadeh
f40f715ca8 rgw: signal shuts down fcgi socket
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-21 12:09:03 -08:00
Yehuda Sadeh
24fde54700 crypto: fix nss related leak
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-21 12:08:58 -08:00
Yehuda Sadeh
7f221a581b rgw: fix xml parser leak
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-21 12:08:54 -08:00
Yehuda Sadeh
b3b77b6ce3 rgw: fix memory leaks
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-11-21 12:08:39 -08:00
Greg Farnum
c31f94d152 set the nonce unconditionally on bind 2012-11-21 10:55:35 -08:00
Dan Mick
0dbf6e8906 test_librbd_fsx: Add OP_FLATTEN 2012-11-21 08:34:50 -08:00
Dan Mick
7021f1a27b test_librbd_fsx: consume saved-image files as test runs
Avoid consuming lots of disk space by holding only as many file
copies as needed (compare the n-2 file as we make clone n).
2012-11-21 08:34:47 -08:00
Sage Weil
b35e37fb73 osdc/Striper: fix handling for sparse reads in add_partial_sparse_result()
If bl_map begins *after* the first item in buffer_extents, we want to
skip only the first buffer extent before doing 'continue' to loop to the
next one.

This fixes a crash caused by underflow with a pattern like:

2012-11-20 13:54:30.347861 7f9404ed6700 10 striper add_partial_sparse_result(0x1efa088) 192 covering {12288=192} (offset 2906) to [0,5286,38054,4288]
2012-11-20 13:54:30.347863 7f9404ed6700 20 striper   t 0~5286 bl has 192 off 2906
2012-11-20 13:54:30.347866 7f9404ed6700 20 striper   s gap 9382, skipping
2012-11-20 13:54:30.347867 7f9404ed6700 20 striper   s has 192, copying
2012-11-20 13:54:30.347872 7f9404ed6700 20 striper   t 9574~18446744073709547328 bl has 0 off 12480
2012-11-20 13:54:30.347874 7f9404ed6700 20 striper   s at end
2012-11-20 13:54:30.347876 7f9404ed6700 20 striper   t 38054~4288 bl has 0 off 12480
2012-11-20 13:54:30.347877 7f9404ed6700 20 striper   s at end

Dan reproduced this with

 ./test_librbd_fsx -d -W -R -p 10 -t 1 -S 4 -N 300 rbd fsx

(although I was unable to do so).

Re-fixes #3428.

Reported-and-tested-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-20 16:41:28 -08:00
Alex Elder
700b5c0029 qa/run_xfstests.sh: drop tests 174 and 181
These tests are showing intermittent failures so we'll drop them
from the default list for the time being.

Signed-off-by: Alex Elder <elder@inktank.com>
2012-11-20 15:53:55 -06:00
Sage Weil
f8f452f324 Merge remote-tracking branch 'gh/wip-mon-parsing' into next 2012-11-18 21:20:36 -08:00
Sage Weil
34e5f9bbfc Merge branch 'wip-mon-leaks-fix' into next 2012-11-18 14:37:22 -08:00
Sage Weil
288db95aa9 mon: shutdown async signal handler sooner
Before the mon, and lockdep, in particular.

#0  __pthread_mutex_lock (mutex=0x30) at pthread_mutex_lock.c:50
#1  0x0000000000816092 in ceph::log::Log::submit_entry (this=0x0, e=0x2f4a270) at log/Log.cc:138
#2  0x00000000007ee0f8 in handle_fatal_signal (signum=11) at global/signal_handler.cc:100
#3  <signal handler called>
#4  0x00000000008e1300 in lockdep_will_lock (name=0x959aa7 "SignalHandler::lock", id=17) at common/lockdep.cc:163
#5  0x00000000008867fc in Mutex::_will_lock (this=0x2f20428) at ./common/Mutex.h:56
#6  0x0000000000886605 in Mutex::Lock (this=0x2f20428, no_lockdep=false) at common/Mutex.cc:81
#7  0x00000000007eeb95 in SignalHandler::entry (this=0x2f20300) at global/signal_handler.cc:198
#8  0x00000000008b0bd1 in Thread::_entry_func (arg=0x2f20300) at common/Thread.cc:43
#9  0x00007f36fefd6b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#10 0x00007f36fd80b6dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#11 0x0000000000000000 in ?? ()

#0  0x00007f36fefd7e75 in pthread_join (threadid=139874129766144, thread_return=0x0) at pthread_join.c:89
#1  0x00000000008b11ec in Thread::join (this=0x2f20300, prval=0x0) at common/Thread.cc:130
#2  0x00000000007eeae7 in SignalHandler::shutdown (this=0x2f20300) at global/signal_handler.cc:186
#3  0x00000000007ee9cf in SignalHandler::~SignalHandler (this=0x2f20300, __in_chrg=<optimized out>) at global/signal_handler.cc:175
#4  0x00000000007eea58 in SignalHandler::~SignalHandler (this=0x2f20300, __in_chrg=<optimized out>) at global/signal_handler.cc:176
#5  0x00000000007ee643 in shutdown_async_signal_handler () at global/signal_handler.cc:324
#6  0x00000000006de9d2 in main (argc=7, argv=0x7fffbfb8a1e8) at ceph_mon.cc:439

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:34:35 -08:00
Sage Weil
45c652d772 mon/AuthMonitor: refactor assign_global_id
Move the failure logic into the caller, where we easier to do something
about it and return the right value to the caller.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:01 -08:00
Sage Weil
92d6b8e636 mon/AuthMonitor: reorder session->put()
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:01 -08:00
Sage Weil
82042adfe0 msg/Pipe: remove useless reader_joining
We set it but do not read it.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:01 -08:00
Sage Weil
c07c93e01d msg/Pipe: join previous reader threads
We may stop and then restart the reader thread.  Join previous threads
before we create new ones.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:00 -08:00
Sage Weil
c4caf871aa msg/DispatchQueue: fix message leak from discard_queue()
We need to drop the Message ref() here; the msgr owns one ref
independent of those from the intrusive_ptr's in the queue itself.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:00 -08:00
Sage Weil
183953e14b msg/SimpleMessenger: use put() on local_connection
This aids leak debugging; not much else.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:00 -08:00
Sage Weil
3e2eb3a16b mon: clean up Subsription xlists
Signed-off-by: Sage Weil <sage@inktank.com>
2012-11-18 08:29:00 -08:00