Commit Graph

69268 Commits

Author SHA1 Message Date
Kefu Chai
edceabbd47 qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for workunit repo
if we run upgrade test, where, for example, "jewel" is not in
ceph-ci.git repo, we should check ceph.git to clone the workunits.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-02-27 17:36:05 +08:00
Haomai Wang
a6108dc4b5 msg/async/rdma: accelerate tx/rx buffer ownership lookup
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-02-27 16:33:39 +08:00
Haomai Wang
a9b13b21e8 msg/async/rdma: cleanup
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-02-27 16:22:32 +08:00
Loic Dachary
31bf2e9acf doc: crush typo in algorithm description
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2017-02-27 09:18:04 +01:00
Brad Hubbard
bc458d3963 rgw: set dumpable flag after setuid post ff0e521
ff0e521 resolved the issue for the other daemons but not for rgw since
it calls setuid (via civetweb) after the new code sets PR_SET_DUMPABLE.
Add another prctl call before wait_shutdown.

Fixes: http://tracker.ceph.com/issues/19089

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-02-27 16:08:41 +10:00
Haomai Wang
bd527486f6 msg/async/rdma: cleanup Infiniband, remove unused functions
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-02-27 13:55:37 +08:00
Haomai Wang
ef19e90f51 Merge pull request #13603 from Adirl/fix_compile
msg/async/rdma: Fix broken compilation

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-02-27 13:55:13 +08:00
Sage Weil
da06227c7f Merge pull request #13652 from liupan1111/wip-cleanup-rbd-nbd
rbd-nbd: fix a typo "moudle"

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-02-26 15:07:19 -06:00
Kefu Chai
87e0759bf2 auth/cephx: print challenges in hex
this matches the CephxServiceHandler::handle_request(), hence would be
easier for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-02-27 00:23:07 +08:00
Kefu Chai
c0f1600666 mon/MonClient: discard stray messages from non-acitve conns
Fixes: http://tracker.ceph.com/issues/19015
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-02-27 00:23:07 +08:00
John Spray
477ddea8a8 Merge pull request #13227 from ukernel/wip-multimds-misc
mds: misc multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>
2017-02-26 11:16:40 +00:00
Kefu Chai
259eac52aa Merge pull request #13648 from tchaikov/wip-doc-osd-agent-options
doc: document `osd-agent-{max,low}-ops` options

Reviewed-by: Zhiqiang Wang <zhiqiang@xsky.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-02-26 17:35:54 +08:00
Pan Liu
5cc7771447 rbd-nbd: fix a typo "moudle"
Signed-off-by: Pan Liu <liupan1111@gmail.com>
2017-02-26 12:14:15 +08:00
Kefu Chai
0d858b9789 doc: document osd-agent-{max,low}-ops options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-02-26 11:12:22 +08:00
Pan Liu
0daaa3c63b rbd: allow to remove snapshots of an image when Ceph cluster becomes full.
Signed-off-by: Pan Liu <liupan1111@gmail.com>
2017-02-26 08:23:55 +08:00
Pan Liu
25eaf0a662 rbd: allow users to remove rbd image when Ceph cluster becomes full.
Signed-off-by: Pan Liu <liupan1111@gmail.com>
2017-02-26 08:22:17 +08:00
Pan Liu
ad8d0e4881 librados: add interface to set osdmap fulll try flag
Signed-off-by: Pan Liu <liupan1111@gmail.com>
2017-02-26 08:20:38 +08:00
Radoslaw Zarzynski
19ca16b856 bluestore: remove CephContext* from BmapEntry.
Each BmapEntry instance stores a pointer to the same CephContext.
As we expect to have thousands of instances the overhead might
be too high. For instance, serving 1 TiB SSD disk on x86-64,
while using the default settings, results in 32 MiB of extra
memory consumption:

  # assuming sizeof(unsigned long) * CHAR_BIT == 64
  >>> 1024 * 1024 * 1024 * 1024 / 4096 / 64
  4194304
  >>> 4194304 * 8 / 1024
  32768

Although memory is cheap, CPU's caches are not.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-02-25 22:21:57 +01:00
Radoslaw Zarzynski
d76f222462 bluestore: the exhausted check in BitMapZone can be lock-less.
Before the patch BitMapZone::is_exhausted() required from its
callers to acquire appropriate lock. However, fulfilling this
condition is not really necessary to use the method correctly
while it can significantly hurt performance.

The change allows BitMapAreaLeaf::child_check_n_lock() to not
acquire the lock while examining zones for being exhausted.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-02-25 20:47:24 +01:00
Sage Weil
af5dab0613 Merge pull request #13649 from liewegas/wip-ceph-scrub-debug
qa/tasks/ceph.py: debug which pgs aren't scrubbing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-02-25 13:15:06 -06:00
Mykola Golub
44e8c873e2 msg: end parameter in entity_addr_t::parse is optional
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2017-02-25 17:42:42 +01:00
Sage Weil
3cc48278bf osd: restructure op_shardedwq
This is difficult to break into pieces, so one big fat commit it is.

A few trivial bits

- include epoch in PGQueueable.
- PGQueuable operator<<
- remove op_wq ref from OSDService; use simple set of queue methods instead

The big stuff:

- Fast dispatch now passes messages directly to the queue based on an
spg_t.  The exception is MOSDOp's from legacy clients.  We add a
waiting_for_map mechanism on the front-side that is similar to but simpler
than the previous one so that we can map those legacy requests to an
accurate spg_t.
- The dequeue path now has a waiting_for_pg mechanism.  It also uses a
much simpler set of data structures that should make it much faster than
the previous incarnation.
- Shutdown works a bit differently; we drain the queue instead of trying
to remove work for individual PGs.  This lets us remove the dequeue_pg
machinery.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-02-24 23:22:19 -05:00
Sage Weil
24d70aff6b msg/Dispatcher: pass const Message* to ms_can_fast_dispatch
Signed-off-by: Sage Weil <sage@redhat.com>
2017-02-24 23:22:18 -05:00
Sage Weil
f777d849e7 qa/tasks/ceph.py: debug which pgs aren't scrubbing
Signed-off-by: Sage Weil <sage@redhat.com>
2017-02-24 23:07:34 -05:00
Kefu Chai
09ab644c8d Merge pull request #13478 from xiaoxichen/fix_osdc_perfcounter
osdc: fix osdc_osd_seesion perf counter.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-02-25 11:00:12 +08:00
Kefu Chai
f5a3cb501a Merge pull request #13626 from wjwithagen/wip-wjw-freebsd-enodata-fixed
do_freebsd.sh: Remove ENODATA requirement

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-02-25 10:28:08 +08:00
Zack Cerza
829b4ac649 Merge pull request #13640 from zmc/wip-fs-openstack
qa/suites/fs: Add openstack volume configuration
2017-02-24 17:22:04 -07:00
Brad Hubbard
e68393aa48 Merge pull request #13619 from kylinstorage/cleanup_bitallocator
BitAllocator: delete useless codes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-02-25 09:30:28 +10:00
Zack Cerza
b076d89a3f qa/suites/fs: Add openstack volume configuration
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-02-24 15:25:19 -07:00
Samuel Just
44b26f6ab4 Merge pull request #13594 from athanatos/wip-snap-trim-sleep
osd: add snap trim reservation and re-implement osd_snap_trim_sleep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-02-24 14:09:17 -08:00
Samuel Just
4f856fe9c9 Merge pull request #13570 from athanatos/wip-18937
osd: don't use ORDERSNAP for flush; always request/send ondisk ack

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-02-24 14:08:40 -08:00
Willem Jan Withagen
1827c690a6 test/librbd/CMakeLists.txt: ceph_test_librbd_fsx requires linux includes/libs
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-02-24 22:45:15 +01:00
Zack Cerza
6f7f331ea8 Merge pull request #13637 from zmc/wip-openstack-volumes
qa/suites/{knfs,hadoop,samba}: OpenStack volume configuration
2017-02-24 14:43:56 -07:00
Zack Cerza
820833e7a0 Merge pull request #13634 from zmc/wip-kcephfs-openstack
qa/suites/kcephfs: Openstack volume configuration
2017-02-24 14:43:26 -07:00
David Zafman
830cc7aa7b mon: Bump min in ratio to 75%
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-02-24 13:02:00 -08:00
David Zafman
50cfe03fcb osd: Fix ENOSPC crash message text
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-02-24 13:02:00 -08:00
Zack Cerza
e0296d7064 qa/suites/samba: Openstack volume configuration
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-02-24 13:46:54 -07:00
Zack Cerza
3fef0a49da qa/suites/hadoop: Openstack volume configuration
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-02-24 13:45:18 -07:00
Zack Cerza
62c6fd3371 qa/suites/knfs: Add openstack volume configuration
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-02-24 13:44:33 -07:00
Zack Cerza
ec6fb28eaf qa/suites/kcephfs: Openstack volume configuration
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-02-24 13:37:23 -07:00
Samuel Just
0c0feca36d osd,osdc: eliminate FLAG_ONDISK and helpers
The objecter actually always needs to get a response in order to
be able to not continually resend ops (even if the caller didn't
provide a callback).  Thus, it makes no sense for an MOSDOp to
ever not have FLAG_ONDISK set.  Therefore, we'll just remove the
helper and assume it's always there (it's safe to send a response
the client didn't ask for, the error paths already do that).  On
the Objecter side, we'll just unconditionally fill in ONDISK for
the benefit of pre-luminous OSDs.

Fixes: http://tracker.ceph.com/issues/18961
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-02-24 11:27:02 -08:00
Samuel Just
48cc5d2643 PrimaryLogPG::start_flush: don't use ORDERSNAP, eliminate the second delete
I think that whole thing was a misguided attempt to avoid deleting head
if it exists in the base tier (in reality it doesn't matter since head
would have to be logically dirty and anything we actually care about
would be preserved by sending a new enough seq to cause a clone).

Introduced in 4843fd510b, but the real
logical error happened in f3df50188b.

I suggest never backporting this patch.  If you want to try, keep in
mind that the last version didn't turn up as busted for 2 years.

Fixes: f3df50188b
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-02-24 11:26:25 -08:00
Zack Cerza
7ac25217ea Merge pull request #13631 from ceph/wip-krbd-openstack
qa/suites/krbd: Add openstack volume configuration
2017-02-24 12:23:23 -07:00
Sage Weil
674ae80a28 Merge pull request #13534 from liewegas/wip-objecter-fixes
osdc/Objecter: fix bugs in explicit naming of op spg_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-02-24 12:55:16 -06:00
Zack Cerza
201b4d0d1e qa/suites/krbd: Add openstack volume configuration
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-02-24 11:17:45 -07:00
Sage Weil
b0a3a2cb69 Merge pull request #13545 from liewegas/wip-18933
osd: constify OpRequest::get_req(); fix a few cases of operator<< vs mutated message races

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-02-24 12:01:54 -06:00
Kefu Chai
8e36dadb71 Merge pull request #13625 from tchaikov/wip-suite-repo-for-workunit
qa/tasks/workunit: use the suite repo for cloning workunit

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-02-24 23:44:27 +08:00
Matt Benjamin
de98df6814 Merge pull request #13614 from linuxbox2/wip-valid-dirname
rgw_file: ensure valid_s3_object_name for directories, too
2017-02-24 09:09:25 -05:00
Jason Dillaman
3cd180b28b Merge pull request #12102 from yangdongsheng/async_remove
librbd: asynchronous image removal state machine

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-02-24 08:50:41 -05:00
Willem Jan Withagen
d92de30724 do_freebsd.sh: the EMBEDDED target is not (yet) working for FreeBSD
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-02-24 11:51:25 +01:00