Commit Graph

42218 Commits

Author SHA1 Message Date
shawn
5c395ff856 doc: add bucket object version description.
bucket object version has been supported, but do not have description in the docs,
so add this part.

Signed-off-by: shawn chen <cxwshawn@gmail.com>
2015-07-30 10:09:16 +08:00
Yan, Zheng
905c31e78b qa: use "sudo rmdir" to remove test data of multiple_rsync.sh
Fixes: #12019
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-06-16 15:06:22 +08:00
Sage Weil
81605bdd1d Merge pull request #4959 from ceph/wip-crush-compat
Add crush_compat.h to eliminate ad hoc diff between kernel and userspace

Reviewed-by: Sage Weil <sage@redhat.com>
2015-06-15 12:40:14 -07:00
Kefu Chai
80e29f19f1 Merge pull request #4956 from oritwas/fix-cmake-2
cmake: add missing RadosDump.cc, RadosImport.cc and PoolDump.cc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-06-16 00:28:58 +08:00
Ilya Dryomov
1db1abc832 crush: eliminate ad hoc diff between kernel and userspace
- map->choose_tries is not in the kernel
- 64-bit/64-bit in the kernel needs a special helper for 32-bit
  architectures, crush_compat.h provides a stub
- INT64_MIN is not in the kernel, crush_compat.h provides S64_MIN
- use dprintk inside DEBUG_INDEP sections

No functional changes.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-06-15 18:13:40 +03:00
Ilya Dryomov
69316a9387 crush: add crush_compat.h
Move all the cruft that is necessary to compile shared files both in
kernel and userspace into a new crush_compat.h.  Also add a stub for
div64_s64() and limits for linux fixed width types.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-06-15 18:13:40 +03:00
Kefu Chai
c1761c0f57 Merge pull request #4826 from xinxinsh/wip-fix
os/DBObjectMap:move iterator to first record of object otherwise iter->valid is always false

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-06-15 22:48:14 +08:00
Ilya Dryomov
ecee227b61 crush: shared files should use kernel-doc
This reverts mapper.c hunks of 830752a528 ("doc: fix doxygen
warnings").  It is shared with the kernel and uses kernel-doc for
in-code documentation.  If doxygen warnings are an issue, we should
probably just exclude the 7 shared files in Doxyfile - we don't have
@file in these files or EXTRACT_ALL/EXTRACT_STATIC in Doxyfile, which
means these files are essentially skipped as there is no C++ stuff in
them.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-06-15 14:26:51 +03:00
Ilya Dryomov
9fde186411 crush: fix style issues in shared files
Fix some recently introduced style issues in crush_ln_table.h,
crush.[ch], hash.[ch], and mapper.[ch].  These files are shared with
the kernel, so we need to at least not introduce anything new -
checkpatch is pretty unhappy as it is.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-06-15 14:26:51 +03:00
Ilya Dryomov
9a8ed8b11c crush: move safe arithmetic functions to buider.c
Given that crush_{addition,multiplication}_is_unsafe() are only used
for compiling maps, they have no business in crush.c which is shared
with the kernel.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-06-15 14:26:51 +03:00
Orit Wasserman
6ef40d7c9e cmake: add missing RadosDump.cc, RadosImport.cc and PoolDump.cc
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2015-06-15 11:39:17 +02:00
xinxin shu
05e95f75d0 objectstore/store_test.cc : add tests for DBObjectMap::get_keys
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2015-06-15 02:46:39 +08:00
xinxin shu
ce5ba43d69 os/DBObjectMap : fix recurcive lock in get_keys
1. get_keys obtain header lock of oid
2. get iterator with get_iterator(oid), in get_iterator, it also want to get header lock of oid

Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2015-06-15 02:45:42 +08:00
xinxin shu
440ef3a230 os/DBOjectMap : iter->valid is always false if we do not initialize the iterator with init(), seek_to_first will initialize this iterator
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2015-06-15 02:45:27 +08:00
Kefu Chai
ac4477926f Merge pull request #4690 from branch-predictor/wip-radosbench-memjuggle
tools: improve the performance of rados and rest_bench

Don't delete, recreate and re-fill buffers in rados bench. Fixes the high CPU usage by rados bench on fast SSDs 
and ramdisks/memstore.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-06-15 00:03:45 +08:00
Sage Weil
a756a32d09 Merge pull request #4950 from SUSE/wip-osd-spelling
osd_types.{cc,h}: fix three typos in comments

Reviewed-by: Sage Weil <sage@redhat.com>
2015-06-13 11:59:39 -07:00
Nathan Cutler
724c139cca osd_types.{cc,h}: fix three typos in comments
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2015-06-13 20:36:09 +02:00
Haomai Wang
e7c91c729e Merge pull request #4758 from xinxinsh/wip-kv-dump-transaction
dump keyvaluestore transaction to a file

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>
2015-06-13 22:49:44 +08:00
Loic Dachary
aa8a56391d Merge pull request #4940 from swamireddy/wip-v094-mailmap
mailmap: Yuri Weinstein affiliation

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-06-13 11:26:43 +02:00
Yehuda Sadeh
df4d4b42ea Merge pull request #4947 from oritwas/fix-cmake
cmake: Remove rados_sync.cc, rados_import.cc and rados_export.cc deleted files

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-06-12 18:45:46 -07:00
Orit Wasserman
f620a8a648 Remove rados_sync.cc, rados_import.cc and rados_export.cc deleted files
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2015-06-13 03:25:20 +02:00
Michal Jarzabek
55d9747e81 mon: add 'PGs' column to 'ceph osd df'
Fixes: #11882

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-06-12 16:00:51 -04:00
Yehuda Sadeh
69989ffa3c Merge pull request #4752 from wuxingyi/wip-11749
rgw: fix data corruption when race condition

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-06-12 11:19:50 -07:00
Yehuda Sadeh
21f850eb68 Merge pull request #4900 from abhidixit/trans_id
Transaction Id added in response for S3 and Swift requests

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-06-12 10:17:31 -07:00
Swami Reddy
45392a2de2 mailmap: Yuri Weinstein affiliation
Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>
2015-06-12 20:19:58 +05:30
Kefu Chai
95ba9677ba Merge pull request #4726 from tchaikov/wip-11680-check-empty-crushmap
mon: check new crush for unknown name/type

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-06-12 09:19:51 +08:00
Loic Dachary
0698bde96a Merge pull request #4932 from dzafman/wip-cot-import
test, tools: Improve ceph-objectstore-tool import error handling and …

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-06-12 00:35:21 +02:00
Sage Weil
f784db0b74 Merge pull request #4933 from dachary/wip-releases
doc: add v0.94.2 to the release timeline

Reviewed-by: Sage Weil <sage@redhat.com>
2015-06-11 13:54:07 -07:00
Loic Dachary
5b29a57b9d doc: add v0.94.2 to the release timeline
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-06-11 22:35:25 +02:00
David Zafman
ddc4d52782 test, tools: Improve ceph-objectstore-tool import error handling and add tests
Signed-off-by: David Zafman <dzafman@redhat.com>
2015-06-11 12:57:10 -07:00
David Zafman
0db3ba157e Merge pull request #4926 from dzafman/wip-omaptool
ceph-osdomap-tool: Fix argument handling

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-06-11 10:56:56 -07:00
Sage Weil
7594963378 Merge pull request #4914 from liewegas/wip-mon-pg-by-osd
mon: fix pg_by_osd in PGMap

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-06-11 10:01:07 -07:00
Yan, Zheng
8b47a62583 Merge pull request #4930 from ceph/wip-11959
qa: use "sudo cp" in multiple_rsync.sh
2015-06-11 21:28:11 +08:00
John Spray
832f33ad5e qa: use "sudo cp" in multiple_rsync.sh
VirtualBox has some files with weird
permissions in its /usr/lib, which was
tripping up this usually-safe operation
when run as an unprivileged user.

Fixes: #11959
Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-11 11:11:44 +01:00
Kefu Chai
393c624189 Merge pull request #4921 from dachary/wip-11952-test-ceph-disk
tests: ceph-disk tests need to install pip > 6.1

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-06-11 16:42:36 +08:00
David Zafman
f23ff731ed ceph-osdomap-tool: Fix argument handling
Signed-off-by: David Zafman <dzafman@redhat.com>
2015-06-10 18:49:00 -07:00
Sage Weil
b4ad260128 mon/PGMap: fix pgmap stat adjustment during map_pg_creates()
PGMonitor::map_pg_creates() is the only caller that passes sumonly=true,
which is necessary to avoid modifying creating_pgs while it is iterating
over it. However, everything else we *do* want to update--especially the
osd counts.

Rename the arg and skip only the creating bit.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-06-10 17:41:57 -07:00
David Zafman
2671775437 Merge pull request #4863 from ceph/wip-9964-nosharding
rados import/export (minus pgls sharding)

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2015-06-10 16:57:15 -07:00
David Zafman
c0c6efb9ce test: Add --dry-run and --no-overwrite testing of rados import
Signed-off-by: David Zafman <dzafman@redhat.com>
2015-06-10 23:50:49 +01:00
John Spray
14819507e1 tools/rados: dry-run/no-overwrite args for import
These were supported by the old objectstoretool
hook into RadosImport: wire them up here too.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 23:50:49 +01:00
John Spray
d1d6196346 tools: better error message in objectstoretool
...for the case where someone exports a pool
with "rados export" and then tries to import
it into a PG.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 23:50:49 +01:00
David Zafman
83a42203c4 test: Switch ceph-objectstore-tool import-rados to rados import
Signed-off-by: David Zafman <dzafman@redhat.com>
2015-06-10 23:50:46 +01:00
John Spray
460200c7bd tools: remove objectstoretool's rados-import
Same functionality now exposed as "rados import".

This removes objectstoretool's librados dependency.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:42 +01:00
John Spray
288331468c tools: remove old rados_sync code
This is the code that used to sit behind the
old import/export commands that have been
disabled for some time.

Replaced by the new import/export commands.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:42 +01:00
John Spray
391c72f38c tools: implement rados import/export
New PoolDump (export) class to dump pool contents
in format compatible with ceph-objectstore-tool,
and wire up RadosImport class (import).

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:41 +01:00
John Spray
a45f8fe11b tools: refactor RadosImport to pass ioctx
Callers from rados.cc will already have an ioctx,
whereas objectstoretool passes in a pool name.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:41 +01:00
John Spray
2991beebad tools: remove RadosDump::debug in favour of dout
This attribute was a hangover from objectstore-tool

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:41 +01:00
John Spray
f477b4458d tools: add support for pool exports in RadosImport
Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:41 +01:00
John Spray
8e5594da29 tools: clean up errors in ceph-objectstore-tool
We use negative error codes everywhere else, should
use them here too.  Also add a couple of strategically
placed prints.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:41 +01:00
John Spray
d26086d35a tools: refactor objectstore tool
...to expose the definitions of the object dump
format (RadosDump) for use in rados import/export.

The serialized->RADOS code is now RadosImport, and
the serialized->ObjectStore code is ObjectStoreTool.

This is a step toward #9964, which should use the
same on serialization format as the existing
objectstore-tool dumps.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-10 22:37:41 +01:00