Commit Graph

34512 Commits

Author SHA1 Message Date
Andrey Kuznetsov
8e5c92139e [RGW, memory leak] Memory leak in RGW GC (losing pointer during allocating Ceph-context) has been fixed.
Memory leaks detector report:
...
==117947== 11,725 (200 direct, 11,525 indirect) bytes in 25 blocks are definitely lost in loss
record 82 of 82
==117947==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==117947==    by 0x687CC1: RGWGC::process(int, int) (in /usr/bin/radosgw)
==117947==    by 0x6884D1: RGWGC::process() (in /usr/bin/radosgw)
==117947==    by 0x68875C: RGWGC::GCWorker::entry() (in /usr/bin/radosgw)
==117947==    by 0x55F890A: Thread::_entry_func(void*) (in /usr/lib64/librados.so.2.0.0)
==117947==    by 0x34D46079D0: start_thread (in /lib64/libpthread-2.12.so)
==117947==    by 0x34D42E8B6C: clone (in /lib64/libc-2.12.so)
...

Signed-off-by: Andrey Kuznetsov <Andrey_Kuznetsov@epam.com>
2014-06-29 22:26:24 +04:00
Andrey Kuznetsov
8706b74e03 [RGW, memory leaks] Memory leak in RGW initialization (Inserting new connection into connections map w/o check) has been fixed.
Memory leaks detector report:

$ valgrind  --leak-check=full /usr/bin/radosgw -c /etc/ceph/ceph.conf -n
client.radosgw.gateway -f
...
=16986== 1,262 (48 direct, 1,214 indirect) bytes in 1 blocks are definitely lost in loss record 81
of 83
==16986==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16986==    by 0x618F0D: RGWRados::init_complete() (in /usr/bin/radosgw)
==16986==    by 0x618FE6: RGWRados::initialize() (in /usr/bin/radosgw)
==16986==    by 0x63BB23: RGWRados::initialize(CephContext*, bool) (in /usr/bin/radosgw)
==16986==    by 0x634D20: RGWStoreManager::init_storage_provider(CephContext*, bool) (in
/usr/bin/radosgw)
==16986==    by 0x5B8970: RGWStoreManager::get_storage(CephContext*, bool) (in /usr/bin/radosgw)
==16986==    by 0x5B6D5D: main (in /usr/bin/radosgw)
...

Signed-off-by: Andrey Kuznetsov <Andrey_Kuznetsov@epam.com>
2014-06-29 22:26:16 +04:00
Ilya Dryomov
9bcc19de58 map-unmap.sh: fail if 'rbd rm' fails
Fail if 'rbd rm' fails - most probably it'd fail with "image still has
watchers" and in that case it's a bug in the kernel client which we do
want to notice.  Also nuke the trap-based error handling - cleanup() is
half-baked and not really necessary here.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-06-29 21:58:15 +04:00
Ilya Dryomov
16b14ea123 map-unmap.sh: drop the get_id() logic
Take advantage of the fact that 'rbd map' will now talk to udev and
output the device that got assigned by the kernel to the newly created
mapping.  Drop the get_id() cruft, udevadm settle and chown calls.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-06-29 21:58:15 +04:00
Ilya Dryomov
abdb168cce test_librbd_fsx: use private RNG context
It is at the core of fsx to be able to reproduce the exact op sequence
that lead to a failure.  Use reentrant glibc RNG functions to make sure
that nothing else has to chance to mess up the sequence.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-06-29 21:55:11 +04:00
John Spray
9517ceab98 os/FileStore: put SUPER usage in ifdef __linux__
This is an update to take account of 9312c5e wherein
the FS_* enum was replaced by direct use of SUPER_MAGIC
macros.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-06-29 17:32:16 +01:00
John Spray
b066e160c8 common: move #include syscall into ifndef DARWIN
Signed-off-by: John Spray <john.spray@redhat.com>
2014-06-29 17:32:06 +01:00
John Spray
2db500b186 client: handle missing O_RSYNC constant
Although this is part of POSIX, it's missing on
OS X.  #define it to zero in this case so that checks
for the flag will always be false.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-06-29 17:31:59 +01:00
John Spray
4b36074ce2 os/FileStore: move ZFS_SUPER_MAGIC in linux ifdef
The __SWORD_TYPE for this line is linux-specific.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-06-29 17:31:48 +01:00
John Spray
656bc04d2e common/Thread: add missing #include for pid_t
On linux this worked by side effect from
pthread.h, but POSIX header for pid_t is
sys/types.h -- this is needed on OS X.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-06-29 17:31:38 +01:00
John Spray
b2bcf52a32 ceph.in: handle DYLD_LIBRARY_PATH on OS X
On OS X, the environment variable to override
to .libs is DYLD_LIBRARY_PATH instead of
LD_LIBRARY_PATH.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-06-29 17:31:15 +01:00
Joao Eduardo Luis
12079a7c7c doc: dev: cache-pool.rst: fix typo s/objets/objects/
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-06-28 14:44:09 +01:00
Somnath Roy
4689467a4a PG: pass OpRequestRef by ref to avoid refcounting overhead
Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2014-06-27 13:33:35 -07:00
Samuel Just
1f59e405ef Merge remote-tracking branch 'upstream/master' into wip-8438-master
Conflicts:
	src/osd/ECMsgTypes.cc
2014-06-27 13:27:47 -07:00
Samuel Just
d48a737a6f osd/: in scrub, check for and remove obsolete rollback objs
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 13:25:52 -07:00
Samuel Just
953c33edb6 osd/: plumb rollback_trim_to through interfaces and messages
We simply trim the roll back info up to min_last_complete_ondisk
since we cannot correctly rollback past that point anyway!

ReplicatedPG
-> PGBackend::submit_transaction
-> Backend messages
-> PGBackend::Listener::log_operation
-> append_log

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 13:25:52 -07:00
Samuel Just
ac11ca40b4 PGLog: add rollback_info_trimmed_to_riter to IndexedLog and use in PG
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 13:25:49 -07:00
Sage Weil
62027ec9f4 doc/release-notes: v0.82
Signed-off-by: Sage Weil <sage@inktank.com>
2014-06-27 13:25:41 -07:00
Sage Weil
eb3963f849 Merge branch 'next' 2014-06-27 12:55:44 -07:00
Yehuda Sadeh
7fae941299 rgw: allocate enough space for bucket instance id
Fixes: #8608
Backport: dumpling, firefly
Bucket instance id is a concatenation of zone name, rados instance id,
and a running counter. We need to allocate enough space to account zone
name length.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-06-27 12:55:39 -07:00
Samuel Just
2207ed12fa PGLog: fix logging whitespace in trim
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 11:09:35 -07:00
Somnath Roy
af4970caa5 OSDMap: avoid passing pg_t by value
Passing pg_t param by cons ref for avoiding copies.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2014-06-27 11:06:22 -07:00
Samuel Just
f48a0333eb Merge remote-tracking branch 'upstream/master' into wip-7999-master
Conflicts:
	src/os/KeyValueStore.h
2014-06-27 10:54:23 -07:00
Samuel Just
d72eec0341 PG: block outgoing peering messages until the last interval is durable
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 10:51:29 -07:00
Samuel Just
f9f89be383 PG: add machinery to temporarily buffer outgoing recovery messages
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 10:51:26 -07:00
Samuel Just
f749812432 os/: add async flush_commit() method to Sequencer and implementations
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-06-27 10:50:50 -07:00
Somnath Roy
0debfe1186 OSD: pass param by ref to avoid copying overrhead
passing OpRequestRef, OSDMapRef by ref to avoid refcounting overhead
and in some cases passing structures by const ref.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2014-06-27 10:48:28 -07:00
Yan, Zheng
1f3fbc9062 mds: print sequence number of log segment 2014-06-27 15:45:56 +08:00
Yan, Zheng
55ed85b194 mds: introduce sequence number for log events
Use the sequence number to index log segments. Later commit will defer
encoding/submitting log events to separate thread, log events' offsets
are unknown before they are encoded.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2014-06-27 15:45:56 +08:00
Yan, Zheng
a17462c84b mds: add get_metablob() to log events
later commits will use it to get EMetaBlob and update EMetaBlob's
event sequence number.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2014-06-27 15:45:55 +08:00
Yan, Zheng
0690078cf8 mds: pass finish context to MDlog::submit_entry()
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2014-06-27 15:45:55 +08:00
Somnath Roy
21955d72a5 MOSDOpReply: Constructor initialization list is used to instantiate members
constructor initialization list is modified to initialize ops,oid,pgid with
copy constructor instead of assignment operator in the constructor body.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2014-06-26 18:57:16 -07:00
Somnath Roy
44db3d04ae MOSDOpreply: The functions are returned by const ref and parameters passed by ref
get_oid,get_pg,get_replay_version,get_user_version are returning by reference and
corresponding setter is accepting param by reference.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2014-06-26 18:53:11 -07:00
Somnath Roy
1c93c61c11 MOSDOp: The functions are returned by const ref and parameters passed by ref
get_snapid(),get_snap_seq,get_pg,get_object_locator,get_version are returning
by reference and the corresponding setter is accepting the param by reference.
MOSDOp constructor is is accepting pg_t  by reference now.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2014-06-26 18:35:35 -07:00
Sage Weil
94e1b77624 Merge pull request #2035 from ceph/wip-da-SCA-20140623
Fix for SCA and CID issues

Reviewed-by: Sage Weil <sage@redhat.com>
2014-06-26 12:43:08 -07:00
Sage Weil
ad81a98e09 Revert "ceph-disk: Enable creating multiple osds per dev"
This reverts commit 200d0ae9c6.

This breaks ceph-disk.  See #8644.  Reverting for now.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-06-26 11:37:15 -07:00
Greg Farnum
e02957dbf7 test: use (unsigned)0 with gtest to avoid spurious warnings
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2014-06-26 11:14:22 -07:00
Samuel Just
4aadddc30b Merge pull request #2034 from ceph/wip-osd-ints
osd: fix int -> int32_t in misc encoded structures

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-06-26 11:05:49 -07:00
Danny Al-Gaaf
f8df9bd269 scratchtool.c: cleanup do_rados_getxattrs()
Cleanup error path to reduce code duplication.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2014-06-26 18:51:19 +02:00
Sage Weil
4e9c2c1b38 osd: fix pg_stat_t int -> int32_t
Still 4 bytes on i386 and x86_64.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-06-26 09:13:47 -07:00
Sage Weil
238b1a3fa4 osd: fix pg_shard_t int -> int32_t
Still 4 bytes on i386 and x86_64.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-06-26 09:13:30 -07:00
Sage Weil
709f0c47c4 osd: fix pg_interval_t int -> int32_t
Still 4 bytes on i386 and x86_64.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-06-26 09:11:53 -07:00
Sage Weil
959d664ba4 Merge pull request #2032 from sponce/wip-fixstriperoni386
Wip fixstriperoni386

Reviewed-by: Sage Weil <sage@redhat.com>
2014-06-26 08:47:06 -07:00
Sebastien Ponce
a5f9a091f8 Fixed build on 32 bits platforms
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-06-26 17:40:08 +02:00
Sebastien Ponce
f5a72b75d5 Added a couple of ASSERTS for avoiding coverity to complain about the possibility of a division by 0
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-06-26 17:40:08 +02:00
Sage Weil
cf16dfc7e3 Merge pull request #2030 from ceph/wip-xattr-fix
mds: set xattr_version to 0 if replay doesn't contain xattrs

Reviewed-by: Sage Weil <sage@redhat.com>
2014-06-26 08:33:11 -07:00
Yan, Zheng
93df7daf18 mds: set xattr_version to 0 if replay doesn't contain xattrs
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2014-06-26 13:33:55 +08:00
Sage Weil
c2ffad3494 Merge pull request #2019 from ceph/wip-refactor-cephtool-test
qa/workunits/cephtool/test.sh: refactor and make it modular (yey! \o/)

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-06-25 22:06:58 -07:00
Danny Al-Gaaf
046c9769fc common/fd.cc: fix possible out-of-bounds write
Read max 'sizeof(target) - 1' to not write out of bound
later on the 'target[r] = 0;' call in case we read the
full PATH_MAX.

CID 1128416 (#1 of 1): Out-of-bounds write (OVERRUN)
 overrun-local: Overrunning array target of 4096 bytes
 at byte offset 4096 using index r (which evaluates to 4096).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2014-06-26 05:22:02 +02:00
Danny Al-Gaaf
605d180569 mount.ceph.c: fix strdup related memory leak
CID 1219611 (#1 - 5): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable saw_name going out of scope
 leaks the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2014-06-26 04:40:09 +02:00