Commit Graph

38274 Commits

Author SHA1 Message Date
Jianpeng Ma
397b261630 common: Don't call ioctl(BTRFS_IOC_SYNC) in sync_filesystem.
From the manual of syncfs, it first appeared in Linux 2.6.39. At this
point, btrfs didn't as a stable production. So remove this.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-07 15:28:37 +08:00
Jianpeng Ma
3f7faa4d8d common: Directly return the result of syncfs().
In commit 808c644248, it will try sync() if syncfs() return error.
No evidence prove this way can work. And sync() don't return result
so make this function always return zero which cause filestore omit the
error.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-07 15:26:08 +08:00
Josh Durgin
6fd348fe43 Merge remote-tracking branch 'origin/wip-10270' into master
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Conflicts:
	src/librados/IoCtxImpl.cc
	src/librados/IoCtxImpl.h
2015-01-06 15:24:14 -08:00
Sage Weil
1a13d92b34 Merge branch 'wip-fast-txn'
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-06 13:48:37 -08:00
Sage Weil
95d2174ece osd/ECBackend: make sure localt uses tbl is ec txn does
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-06 13:29:41 -08:00
Sage Weil
5d66525209 ceph-object-corpus: drop compat with old ObjectStore::Transaction
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-06 13:29:41 -08:00
Dong Yuan
0354305db7 osd: fix Transaction::get_data_offset bug when map layout used
add following offset:
  sizeof(__u8) +      // encode struct_v
  sizeof(__u8) +      // encode compat_v
  sizeof(__u32);      // encode len

Change-Id: I5b6662eb42aeeae64baa8699da6ce65e0b1d58c3
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:41 -08:00
Dong Yuan
fda601976b osd: add feature CEPH_FEATURE_OSD_TRANSACTION_MAY_LAYOUT (1ULL<<47)>>)
This feature determine whether we use tbl encode for transaction of use
the new map layout.

The primary uses peer_features to determine whether transaction should
use tbl, while the replica just follow the primary.

Change-Id: I92ca6e5b59bd1acde6007ad0dffc085be17accab
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:41 -08:00
Dong Yuan
5da6b41f87 osd: build fields for Transaction::iterator when tbl is used
When tbl is used (for compatibility), the Transaction::begin method need
to build all fields used by iterator. That includes: coll_index,
object_index, data_bl, op_bl, etc.)

Change-Id: I48ea74fec8d052f50da254a726a9c0dffead19bc
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Sage Weil
1889d390e0 ceph_perf_objectstore: fix warning
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
d427ca3540 osd: Transaction::append & Transaction::swap
Finish append and swap for new Transaction encode/decode layout.

Since append will modify the op_bl now, we changed the order of append
and swap in ReplicatedBackend::sub_op_modify and
ReplicatedBackend::submit_transaction to avoid append call on op_t, so
the op_t can be encode in message.

Change-Id: I6fb421e0defdb092fb9732eef818e90291b039f5
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
6b2103b728 osd: new Transaction::iterator interface
This patch add new Transaction::iterator interface according to new
encode/decode layout. The new iterator give the whole Op struct in a
single decode_op method.

All ObjectStore Impl (FileStore/MemStore/KeyValueStore) is also changed
to use the new interface.

Change-Id: I1900a6ec302890df2c4357b071e4966c26d7f037
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
a4575d1018 osd: add encode/deocde impl for new layout
When use_tbl is true, Transaction::encode will give the same result as
before, while when use_tbl is false, Transaction::encode will use new
field and logic to encode and all related methods such as
get_encoded_bytes, get_data_offset will do the same.

Change-Id: Ia5864e489d47f37cf496fe3fb825b21977d2d938
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
3db4f696e5 osd: new format for Transaction encode/decode
This patch add a new fixed size struct Transaction::Op to represent
all actions.

All coll and ghobject used by the transaction are keeped in two maps:
  coll:   map<coll_t, __le32> coll_index;
  object: map<ghobject_t, __le32> object_index;

And the Op struct use the map value(__le32) to refer coll and object,
so each coll and object is only need to encode once in the transaction.

Other variable-size fields(key/value/data) is encoded in bufferlist
data_bl.

Change-Id: I52b2fcd3217a6cb35de7b309a6dd74a99478feb2
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
dbae92253f osd: Add Transaction::TransactionData for fast encode/decode
TransactionData wrap the following fields:
      __le64 ops;
      __le32 largest_data_len;
      __le32 largest_data_off;
      __le32 largest_data_off_in_tbl;
      __le32 pad; //make TransactionData multiple of uint64_t

This struct can encode/decode just by a single memcpy instead of many
encode/decode operations.

Change-Id: I56df78def43bd2b80b77be0825756e133434a6e6
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
3bd0042154 osd: remove unused Transaction fields
We don't need sobject and pool_override anymore since we don't need to
support anything older than dumpling.

Change-Id: I22c01d4b5c6bf99765bf6bc13aecadc997d6750c
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Loic Dachary
ebfe230680 Merge pull request #3272 from trociny/tell-mon-version
cli: make ceph tell mon.* version work

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:35:36 +01:00
Loic Dachary
34c0e17d53 Merge pull request #3196 from tchaikov/shadow-cct
client, librados, osdc: do not shadow Dispatcher::cct

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:34:20 +01:00
Loic Dachary
815bb9c237 Merge pull request #3298 from majianpeng/fix5
TestLFNIndex.cc: For root, dont do permission operations.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:32:27 +01:00
Loic Dachary
de0213457d tests: resolve ceph-helpers races
Some tests were racing against the monitor. On a fast machine it worked
but slower machines (or sometime when running in parallel), the monitor
is lagging behind. Use wait_for_clean to make sure the monitor is in the
desired state for the test to succeed.

http://tracker.ceph.com/issues/10384 Fixes: #10384

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:10:34 +01:00
Loic Dachary
d7142d9a70 Merge pull request #3283 from dachary/wip-make-debs
debian: create a repository from sources

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 19:07:04 +01:00
Sage Weil
386ab7cef3 Merge pull request #3301 from ktdreyer/rm-tiobench
#10152: qa: drop tiobench suite

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 08:26:04 -08:00
Sage Weil
6051321059 Merge pull request #3300 from ceph/wip-10412
client: fix use-after-free bug in unmount()

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 08:20:19 -08:00
Ken Dreyer
bea2d4a8f7 qa: drop tiobench suite
The tiobench software has been abandoned upstream for years. Fedora and
Debian are no longer shipping the tiobench package, so we've had to
carry the package ourselves in the Ceph project, and we're trying to
slim down our dependencies where it makes sense to do so.

Nuke the tiobench suite.

http://tracker.ceph.com/issues/10152 Refs: #10152

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-01-06 09:00:16 -07:00
Sage Weil
9330ad6042 Merge pull request #3276 from ceph/wip-repop
Add MOSDRepOp and MOSDRepOpReply

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-06 07:11:53 -08:00
Sage Weil
68c27cfba8 Merge pull request #3299 from wonzhq/compile-warn
cleanup: fix a compile warning

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 06:50:49 -08:00
Yan, Zheng
55f8fee2c9 client: fix use-after-free bug in unmount()
If there is error, _release_fh() already outputs error message.
No need to print extra error message

Fixes: #10412
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-01-06 20:22:36 +08:00
Zhiqiang Wang
93d39a849c cleanup: fix a compile warning
warning: 'pi' may be used uninitialized in this function
[-Wmaybe-uninitialized]

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-01-06 16:36:03 +08:00
Jianpeng Ma
ea3ed5bbf0 TestLFNIndex.cc: For root, dont do permission operations.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-06 16:23:40 +08:00
Sage Weil
8a178392db Merge pull request #3293 from ktdreyer/help-missing-keyutils-libs-devel
configure: show package name when libkeyutils is missing
2015-01-05 18:24:11 -08:00
Sage Weil
4a42a81c69 Merge pull request #3294 from ceph/wip-fix-copyfrom
osd: fix copyfrom

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-05 16:54:20 -08:00
Josh Durgin
0f83a181ba Merge pull request #3226 from ceph/wip-10319-wusui
If trusty, use older version of qemu

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 15:39:22 -08:00
Sage Weil
488355cc5a osd: fix object_copy_data_t compat encoding
If the omap_data map is empty, we want an empty bufferlist.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 14:37:33 -08:00
Josh Durgin
6ccdce561e Merge pull request #3189 from ceph/wip-10329
Wip 10329

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 13:28:19 -08:00
Josh Durgin
d5e2ca1620 Merge pull request #3277 from ceph/wip-watch-leak
librados: fix a memory leak in watch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 12:54:22 -08:00
Sage Weil
ed503776b5 osd/PG: print last_update_applied if != last_update
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 12:53:58 -08:00
Xiaoxi Chen
78d2d310ba Add MOSDRepOp and MOSDRepOpReply
Add the two new message type and change the corresponding code flow as well.

Basically the idea to have MOSDRepOp is to seperate subop(read/write)
out of other subop(pull/push,etc), so that we can cleanup some unused fields in
the message type, then save some encoding/decoding overhead.

The backward compatibility is also remian, if talking with old version OSD who
doesn't support osd_client_subop/subopreply, will fall back to osd_subop/subopreply.

Sage: rename MOSDClientSubOp -> MOSDRepOp

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 12:53:58 -08:00
Josh Durgin
f9f64e190e Merge pull request #3206 from ceph/wip-librados-init
librados: init last_objver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 12:34:17 -08:00
Ken Dreyer
6ca2eb86be configure: show pkg names when libkeyutils is missing
Prior to this commit, when ./configure can't find libkeyutils, it would
bail out with a terse error message.

Some of the other library checks helpfully print the DEB and RPM package
names in parentheses. Add the DEB and RPM package names to the
libkeyutils check.

Reported-by: Pankaj Garg <Pankaj.Garg@caviumnetworks.com>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-01-05 12:19:40 -07:00
Sage Weil
d253badb8d Merge remote-tracking branch 'gh/next' 2015-01-05 11:16:57 -08:00
Sage Weil
3a2e2df79c Merge pull request #3247 from ceph/wip-10422
mon: provide encoded canonical full OSDMap from primary

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
2015-01-05 11:16:49 -08:00
Sage Weil
5cf84e6393 librados: fix leak of WatchContext on unwatch
The lifecycle matches that of the watch linger_op.  Note that it is NULL
for notify.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 09:36:43 -08:00
Mykola Golub
b5c24e45a9 qa: add test_tell, which currently is used only for testing 'ceph tell mon version'.
The test requires at least two monitors run.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-01-05 15:53:54 +02:00
Loic Dachary
f95022e88c Merge pull request #3291 from majianpeng/fix2
test/bufferlist: Remove chmod operation.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-05 14:28:50 +01:00
Jianpeng Ma
244777d825 test/bufferlist: For root, don't do permission operation for read_file
case.

For root user, it meet those error:
    test/bufferlist.cc:1880: Failure
    Value of: bl.read_file("testfile", &error)
      Actual: 0
    Expected: -13
    test/bufferlist.cc:1884: Failure
    Value of: bl.length()
      Actual: 8
    Expected: (unsigned)4
    Which is: 4
    test/bufferlist.cc:1886: Failure
    Value of: actual
      Actual: "ABC
    ABC
    "
    Expected: "ABC\n"
    Which is: "ABC

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-05 20:51:21 +08:00
Mykola Golub
3b4e021cdc qa: refactor 'ceph -w' magic in test_mon_misc by introducing helper
functions ceph_watch_start and ceph_watch_wait so they can be reused
in other tests.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-01-05 12:01:15 +02:00
Mykola Golub
11891383ae mon: make ceph tell mon.* version work
Fixes: #6767
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-01-05 12:01:03 +02:00
Haomai Wang
c37abeaf31 Merge pull request #3261 from tchaikov/async-msgr
AsyncMessenger: a trivial refactory and a fix

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
2015-01-05 10:14:40 +08:00
Sage Weil
f5e3b9a610 Merge pull request #3286 from ceph/wip-10351
mount.ceph: avoid spurious error message

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-03 10:28:21 -08:00
Yan, Zheng
bdd0e3c4bd mount.ceph: avoid spurious error message
/etc/mtab in most modern distributions is a symbol link to
/proc/self/mounts.

Fixes: #10351
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-01-03 15:35:00 +08:00