Commit Graph

38098 Commits

Author SHA1 Message Date
Dong Yuan
5da6b41f87 osd: build fields for Transaction::iterator when tbl is used
When tbl is used (for compatibility), the Transaction::begin method need
to build all fields used by iterator. That includes: coll_index,
object_index, data_bl, op_bl, etc.)

Change-Id: I48ea74fec8d052f50da254a726a9c0dffead19bc
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Sage Weil
1889d390e0 ceph_perf_objectstore: fix warning
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
d427ca3540 osd: Transaction::append & Transaction::swap
Finish append and swap for new Transaction encode/decode layout.

Since append will modify the op_bl now, we changed the order of append
and swap in ReplicatedBackend::sub_op_modify and
ReplicatedBackend::submit_transaction to avoid append call on op_t, so
the op_t can be encode in message.

Change-Id: I6fb421e0defdb092fb9732eef818e90291b039f5
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
6b2103b728 osd: new Transaction::iterator interface
This patch add new Transaction::iterator interface according to new
encode/decode layout. The new iterator give the whole Op struct in a
single decode_op method.

All ObjectStore Impl (FileStore/MemStore/KeyValueStore) is also changed
to use the new interface.

Change-Id: I1900a6ec302890df2c4357b071e4966c26d7f037
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
a4575d1018 osd: add encode/deocde impl for new layout
When use_tbl is true, Transaction::encode will give the same result as
before, while when use_tbl is false, Transaction::encode will use new
field and logic to encode and all related methods such as
get_encoded_bytes, get_data_offset will do the same.

Change-Id: Ia5864e489d47f37cf496fe3fb825b21977d2d938
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
3db4f696e5 osd: new format for Transaction encode/decode
This patch add a new fixed size struct Transaction::Op to represent
all actions.

All coll and ghobject used by the transaction are keeped in two maps:
  coll:   map<coll_t, __le32> coll_index;
  object: map<ghobject_t, __le32> object_index;

And the Op struct use the map value(__le32) to refer coll and object,
so each coll and object is only need to encode once in the transaction.

Other variable-size fields(key/value/data) is encoded in bufferlist
data_bl.

Change-Id: I52b2fcd3217a6cb35de7b309a6dd74a99478feb2
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
dbae92253f osd: Add Transaction::TransactionData for fast encode/decode
TransactionData wrap the following fields:
      __le64 ops;
      __le32 largest_data_len;
      __le32 largest_data_off;
      __le32 largest_data_off_in_tbl;
      __le32 pad; //make TransactionData multiple of uint64_t

This struct can encode/decode just by a single memcpy instead of many
encode/decode operations.

Change-Id: I56df78def43bd2b80b77be0825756e133434a6e6
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Dong Yuan
3bd0042154 osd: remove unused Transaction fields
We don't need sobject and pool_override anymore since we don't need to
support anything older than dumpling.

Change-Id: I22c01d4b5c6bf99765bf6bc13aecadc997d6750c
Signed-off-by: Dong Yuan <yuandong1222@gmail.com>
2015-01-06 13:29:06 -08:00
Loic Dachary
ebfe230680 Merge pull request #3272 from trociny/tell-mon-version
cli: make ceph tell mon.* version work

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:35:36 +01:00
Loic Dachary
34c0e17d53 Merge pull request #3196 from tchaikov/shadow-cct
client, librados, osdc: do not shadow Dispatcher::cct

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:34:20 +01:00
Loic Dachary
815bb9c237 Merge pull request #3298 from majianpeng/fix5
TestLFNIndex.cc: For root, dont do permission operations.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-06 19:32:27 +01:00
Loic Dachary
d7142d9a70 Merge pull request #3283 from dachary/wip-make-debs
debian: create a repository from sources

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 19:07:04 +01:00
Sage Weil
386ab7cef3 Merge pull request #3301 from ktdreyer/rm-tiobench
#10152: qa: drop tiobench suite

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 08:26:04 -08:00
Sage Weil
6051321059 Merge pull request #3300 from ceph/wip-10412
client: fix use-after-free bug in unmount()

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 08:20:19 -08:00
Ken Dreyer
bea2d4a8f7 qa: drop tiobench suite
The tiobench software has been abandoned upstream for years. Fedora and
Debian are no longer shipping the tiobench package, so we've had to
carry the package ourselves in the Ceph project, and we're trying to
slim down our dependencies where it makes sense to do so.

Nuke the tiobench suite.

http://tracker.ceph.com/issues/10152 Refs: #10152

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-01-06 09:00:16 -07:00
Sage Weil
9330ad6042 Merge pull request #3276 from ceph/wip-repop
Add MOSDRepOp and MOSDRepOpReply

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-06 07:11:53 -08:00
Sage Weil
68c27cfba8 Merge pull request #3299 from wonzhq/compile-warn
cleanup: fix a compile warning

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-06 06:50:49 -08:00
Yan, Zheng
55f8fee2c9 client: fix use-after-free bug in unmount()
If there is error, _release_fh() already outputs error message.
No need to print extra error message

Fixes: #10412
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-01-06 20:22:36 +08:00
Zhiqiang Wang
93d39a849c cleanup: fix a compile warning
warning: 'pi' may be used uninitialized in this function
[-Wmaybe-uninitialized]

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-01-06 16:36:03 +08:00
Jianpeng Ma
ea3ed5bbf0 TestLFNIndex.cc: For root, dont do permission operations.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-06 16:23:40 +08:00
Sage Weil
8a178392db Merge pull request #3293 from ktdreyer/help-missing-keyutils-libs-devel
configure: show package name when libkeyutils is missing
2015-01-05 18:24:11 -08:00
Sage Weil
4a42a81c69 Merge pull request #3294 from ceph/wip-fix-copyfrom
osd: fix copyfrom

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-05 16:54:20 -08:00
Josh Durgin
0f83a181ba Merge pull request #3226 from ceph/wip-10319-wusui
If trusty, use older version of qemu

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 15:39:22 -08:00
Sage Weil
488355cc5a osd: fix object_copy_data_t compat encoding
If the omap_data map is empty, we want an empty bufferlist.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 14:37:33 -08:00
Josh Durgin
6ccdce561e Merge pull request #3189 from ceph/wip-10329
Wip 10329

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 13:28:19 -08:00
Sage Weil
ed503776b5 osd/PG: print last_update_applied if != last_update
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 12:53:58 -08:00
Xiaoxi Chen
78d2d310ba Add MOSDRepOp and MOSDRepOpReply
Add the two new message type and change the corresponding code flow as well.

Basically the idea to have MOSDRepOp is to seperate subop(read/write)
out of other subop(pull/push,etc), so that we can cleanup some unused fields in
the message type, then save some encoding/decoding overhead.

The backward compatibility is also remian, if talking with old version OSD who
doesn't support osd_client_subop/subopreply, will fall back to osd_subop/subopreply.

Sage: rename MOSDClientSubOp -> MOSDRepOp

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-05 12:53:58 -08:00
Josh Durgin
f9f64e190e Merge pull request #3206 from ceph/wip-librados-init
librados: init last_objver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-05 12:34:17 -08:00
Ken Dreyer
6ca2eb86be configure: show pkg names when libkeyutils is missing
Prior to this commit, when ./configure can't find libkeyutils, it would
bail out with a terse error message.

Some of the other library checks helpfully print the DEB and RPM package
names in parentheses. Add the DEB and RPM package names to the
libkeyutils check.

Reported-by: Pankaj Garg <Pankaj.Garg@caviumnetworks.com>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-01-05 12:19:40 -07:00
Sage Weil
d253badb8d Merge remote-tracking branch 'gh/next' 2015-01-05 11:16:57 -08:00
Sage Weil
3a2e2df79c Merge pull request #3247 from ceph/wip-10422
mon: provide encoded canonical full OSDMap from primary

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
2015-01-05 11:16:49 -08:00
Mykola Golub
b5c24e45a9 qa: add test_tell, which currently is used only for testing 'ceph tell mon version'.
The test requires at least two monitors run.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-01-05 15:53:54 +02:00
Loic Dachary
f95022e88c Merge pull request #3291 from majianpeng/fix2
test/bufferlist: Remove chmod operation.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-05 14:28:50 +01:00
Jianpeng Ma
244777d825 test/bufferlist: For root, don't do permission operation for read_file
case.

For root user, it meet those error:
    test/bufferlist.cc:1880: Failure
    Value of: bl.read_file("testfile", &error)
      Actual: 0
    Expected: -13
    test/bufferlist.cc:1884: Failure
    Value of: bl.length()
      Actual: 8
    Expected: (unsigned)4
    Which is: 4
    test/bufferlist.cc:1886: Failure
    Value of: actual
      Actual: "ABC
    ABC
    "
    Expected: "ABC\n"
    Which is: "ABC

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-05 20:51:21 +08:00
Mykola Golub
3b4e021cdc qa: refactor 'ceph -w' magic in test_mon_misc by introducing helper
functions ceph_watch_start and ceph_watch_wait so they can be reused
in other tests.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-01-05 12:01:15 +02:00
Mykola Golub
11891383ae mon: make ceph tell mon.* version work
Fixes: #6767
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-01-05 12:01:03 +02:00
Haomai Wang
c37abeaf31 Merge pull request #3261 from tchaikov/async-msgr
AsyncMessenger: a trivial refactory and a fix

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
2015-01-05 10:14:40 +08:00
Sage Weil
f5e3b9a610 Merge pull request #3286 from ceph/wip-10351
mount.ceph: avoid spurious error message

Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-03 10:28:21 -08:00
Yan, Zheng
bdd0e3c4bd mount.ceph: avoid spurious error message
/etc/mtab in most modern distributions is a symbol link to
/proc/self/mounts.

Fixes: #10351
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-01-03 15:35:00 +08:00
Loic Dachary
dced90ba87 Merge pull request #3284 from ktdreyer/doc-rados-preflight-os
doc: rm reference to old Ubuntu release

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-02 18:54:47 +01:00
Ken Dreyer
274b989fec doc: rm reference to old Ubuntu release
Remove the reference to "Ubuntu 12.10" since this is EOL.

Clarify that we only recommend Ubuntu LTS releases.

(Since this information has a tendancy to become stale, perhaps this
whole paragraph should be removed here and we should simply point at the
main OS Recommendations page.)

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-01-02 10:37:10 -07:00
Loic Dachary
0471be6fe3 debian: create a repository from sources
Create a debian repository suitable for inclusion in
/etc/apt/sources.list, for the current distribution and using the git
clone from the current working directory.

It is meant to quickly provide teuthology with the packages to run a
job.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-01-02 12:14:39 +01:00
Loic Dachary
020983b0a5 Merge pull request #3223 from objoo/master
mailmap updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-01-01 22:52:24 +01:00
Kefu Chai
c8f56ab2a2 AsyncMessender: use IOV_MAX instead of IOV_LEN
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2014-12-31 10:45:00 +08:00
Kefu Chai
d49b694064 AsyncMessenger: s/sended/sent/
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2014-12-31 10:45:00 +08:00
Kefu Chai
c9da51de48 AsyncMessenger: should retry in case of EINTR
* if EINTR is returned, do_send() should return 0 or `continue`.
* reword the comment a little bit

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2014-12-31 10:45:00 +08:00
Sage Weil
169e8bd2cf msg: fix factory error message
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-30 14:08:49 -08:00
Sage Weil
ec1f58a30d unittest_msgr: enable experiemntal async msgr
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-30 14:08:38 -08:00
Sage Weil
51f5f12ca4 Merge pull request #3219 from yuyuyu101/wip-10383
Wip 10383: Fix unittest_msgr with AsyncMessenger bug

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-30 10:46:58 -08:00
Sage Weil
ecdc9fb2ec Merge pull request #3279 from ceph/wip-mon-janitorial-list
src/doc: add monitor janitorial list
2014-12-30 10:44:17 -08:00