Commit Graph

71076 Commits

Author SHA1 Message Date
John Spray
3fccc2372f mds: ignore ENOENT on writing backtrace
We get ENOENT when a pool doesn't exist.  This can
happen because we don't prevent people deleting
former cephfs data pools whose files may not have
had their metadata flushed yet.

http://tracker.ceph.com/issues/19401
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 12:20:25 -04:00
Ali Maredia
b31b84529e rgw multisite: use get_config_master_client for radosgw_admin task
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-04-13 12:15:50 -04:00
Ali Maredia
c5956790e6 rgw: multisite enabled over multiple clusters
Added '--cluster' to all necessary commands
ex: radosgw-admin, rados, ceph, made sure
necessary checks were in place so that clients
can be read with our without a cluster_name
preceeding them

Made master_client defined in the config for
radosgw-admin task

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-04-13 12:15:50 -04:00
Kefu Chai
dc7ff92e52 mgr: always free allocated MgrPyModule
use unique_ptr to manage the lifecycle of MgrPyModule and ServeThread,
it's easier and safer. without this chance, we don't free allocated
MgrPyModule if it fails to load().

Fixes: http://tracker.ceph.com/issues/19590
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-13 23:58:43 +08:00
Matt Benjamin
007b7451c2 rgw_file: don't expire directories being read
If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-13 11:30:31 -04:00
John Spray
01f79a748b mon: emit cluster log messages on MDS health changes
Previously, when we got a beacon that updated the health
metrics for an MDS, the user would just see mysterious-looking
cluster log messages indicating a rising fsmap epoch number.

It would be good to do this for health messages in general at
some point, but for now just do it for the MDS ones.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 11:06:13 -04:00
John Spray
fc28d29c0c mds: fix mgrc shutdown
Were previously only tearing MgrClient down when not
holding a rank, leading to it trying to continue
to run after monclient was shut down.

Fixes: http://tracker.ceph.com/issues/19566
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 11:00:46 -04:00
Matt Benjamin
e0191d74e3 rgw_file: chunked readdir
Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-13 10:58:22 -04:00
Matt Benjamin
2e66c7a7cc rgw_file: introduce rgw_lookup type hints
The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-13 10:35:37 -04:00
Amir Vadai
972c7416de msg/async/rdma: Use RDMA resources only after fork
Thanks to previous patch [1], no need to access RDMA resources before
the fork. Initialize Infiniband class only before a connection is
established or a listener is created. [1] is making sure that the call
to RDMAWorker::listen() is postponed till after the fork.

[1] - 7393db45644d ("msg/async: Postpone bind if network stack is not ready")

Issue: 995322
Change-Id: I8ea246b2e03c8c9533bc324b2b8d142eb3d1ed4d
Signed-off-by: Amir Vadai <amir@vadai.me>
2017-04-13 14:29:03 +00:00
Haomai Wang
a5b87e2fb8 msg/async: Postpone bind if network stack is not ready
RDMAStack shouldn't access hardware from the parent process.
The only reason to do so, is because bind is called before the fork.
After this patch the bind is postponed until the NetworkStack reports
that it is ready to bind.
For NetworkStack types will always return true, except the RDMAStack
which will return true only after the fork (after
AsyncMessenger::ready() is called).

This patch is based on a patch by Haomai Wang <haomai@xsky.com>

Issue: 995322
Change-Id: I1d0d0d52db0a339b9319680c18ee05cde87b2b64
Signed-off-by: Amir Vadai <amir@vadai.me>
2017-04-13 14:28:48 +00:00
Haomai Wang
22b0312b4f msg/async: remove duplicate NetworkStack::start which exists in constructor
Issue: 995322
Change-Id: Ib8f7f25b9b360bea97147ce27c369c6baefcc6f3
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-04-13 14:28:36 +00:00
Kefu Chai
8442cbb521 Merge pull request #14468 from tchaikov/wip-show-me-fake-plugin
qa/workunits/ceph-helpers: display rejected string

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-13 22:24:03 +08:00
Sage Weil
20e5bff7fc mon/OSDMonitor: fix initial map when require_luminous_osds not set on mkfs
If we don't set the luminous flag, we should not set the new luninous
fields or else we'll get a crc mismatch.  (Funnily that happens in the
epoch where the flag is eventually set and the encoded map finally includes
the field we have set in memory.)

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 10:10:26 -04:00
Sage Weil
10ea8e1d7d mon/OSDMonitor: ignore health checks about require_luminous if debug flag is set
This makes it tedious for teuthology health checks to proceed when we
deliberately run luminous osds without this flag.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 10:10:26 -04:00
Yang Honggang
eb915d0eec cephfs: fix write_buf's _len overflow problem
After I have set about 400 64KB xattr kv pair to a file,
mds is crashed. Every time I try to start mds, it will crash again.
The root reason is write_buf._len overflowed when doing
Journaler::append_entry().

This patch try to fix this problem through the following changes:

 1. limit file/dir's xattr size
 2. throttle journal entry append operations

Fixes: http://tracker.ceph.com/issues/19033
Signed-off-by: Yang Honggang joseph.yang@xtaotech.com
2017-04-13 20:09:07 +08:00
John Spray
b81e4b70a4 Merge pull request #14234 from jcsp/wip-19406-dirhash
tools/cephfs: set dir_layout when injecting inodes

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-13 12:01:28 +01:00
Willem Jan Withagen
291815aae1 ceph-disk/tests/test_main.py: Some partition tests are no longer valid under FreeBSD
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-04-13 12:53:35 +02:00
Willem Jan Withagen
c8fda3f04c src/ceph-disk/ceph_disk/main.py: Make ceph-disk list work on FreeBSD
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-04-13 11:59:06 +02:00
Jianpeng Ma
29d8c4fe73 os/bluestore: Record l_bluestore_state_kv_queued_lat for sync_submit_transaction mode.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-04-13 17:26:13 +08:00
Kefu Chai
051f9138df Merge pull request #14472 from joscollin/wip-warning-test
test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-13 16:28:12 +08:00
Kefu Chai
097d00cb81 Merge pull request #14495 from tchaikov/wip-cmake-ctags-without-repo
cmake: do not try to add submodule to exclude list if .git is not around

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-13 15:35:09 +08:00
Kefu Chai
9acd868857 Merge pull request #14404 from tchaikov/wip-another-mgr-command
mon,mgr: move reweight-by-* to mgr

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-13 13:57:42 +08:00
Kefu Chai
4847a88556 cmake: do not try to add submodule to exclude list if .git is not around
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-13 13:49:54 +08:00
fang yuxiang
dcaaea5dc8 rgw: bucket stats display bucket index type
Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com
2017-04-13 13:47:02 +08:00
Mykola Golub
b9522ef915 Merge pull request #14445 from dillaman/wip-rbd-import-diff
rbd: import-diff should discard any zeroed extents

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-04-13 08:00:05 +03:00
Matt Benjamin
335cf9a23e Merge pull request #14474 from Wilhelmshaven/fix_non-exist_string
rgw: avoid using null pointer in rgw_file.cc
2017-04-12 21:00:59 -04:00
Brad Hubbard
465ab063ad mds: Return error message instead of asserting
For "session evict" admin socket command return an error message when we
receive an invalid/missing client_id parameter rather than asserting.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-13 10:05:19 +10:00
Sage Weil
38aff741f3 mon/OSDMonitor: add mon_debug_no_require_luminous
Add debug option not to set the require_luminous osdmap flag on mkfs, for
debugging purposes.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-12 18:15:16 -04:00
Jason Dillaman
25eb40bd4b Merge pull request #14482 from trociny/wip-qa-diff
qa/workunits/rbd: diff.sh failed removing nonexistent file

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-04-12 16:43:21 -04:00
John Spray
7d6d542885 tools/cephfs: set dir_layout when injecting inodes
When we left this as zero, the MDS would interpret it was HASH_LINUX
rather than the default HASH_RJENKINS.  Potentially that
could cause problems if there perhaps were already dirfrags in
the metadata pool that were set up using rjenkins.  Mainly
it just seems more appropriate to explicitly set this field
rather than hit the fallback behaviour.

Related: http://tracker.ceph.com/issues/19406
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-12 16:30:51 -04:00
Jason Dillaman
7d12a7e184 Merge pull request #14463 from trociny/wip-19451
pybind/rbd: add image metadata methods

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-04-12 15:02:16 -04:00
Jason Dillaman
0d2f856464 Merge pull request #14471 from trociny/wip-19588
librbd: fix rbd_metadata_list and rbd_metadata_get

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-04-12 15:01:56 -04:00
Sage Weil
0b8644b0cd os/bluestore: fix narrow osr->flush() race
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-12 14:19:46 -04:00
Mykola Golub
bf21bae415 Merge pull request #14484 from dillaman/wip-librados-test-stub
test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-04-12 21:04:42 +03:00
Yehuda Sadeh
41f0dd1d27 Merge pull request #14375 from cbodley/wip-rgw-fetch-versioned-obj 2017-04-12 09:02:44 -07:00
Kefu Chai
ef7583d717 qa/suites: add mon-reweight-min-pgs-per-osd = 4
so we can run reweight commands in a small cluster with 3 OSD and 27
PGs.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-12 23:29:10 +08:00
Loic Dachary
f55b2a3fe1 Merge pull request #14476 from wjwithagen/wip-wjw-freebsd-ceph-disk-call-rc.d
ceph-disk/ceph_disk/main.py: fix calling of the bsdrc init scripts

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-04-12 17:04:35 +02:00
Jason Dillaman
9ffd464dac test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value
Fixes: http://tracker.ceph.com/issues/19597
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-04-12 10:49:05 -04:00
Sage Weil
92006f6d39 Merge pull request #14399 from ifed01/wip-bluestore-better-better-reuse
os/bluestore: refactor small write handling to reuse blob more effect…

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-12 09:31:45 -05:00
Sage Weil
048722605f Merge pull request #14422 from tchaikov/wip-19427
mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-12 09:28:38 -05:00
Sage Weil
4f210cd13d Merge pull request #14439 from adamemerson/wip-warning-sign
misc: Warning Elimination

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-12 09:28:10 -05:00
Mykola Golub
bf52a2a53b qa/workunits/rbd: diff.sh failed removing nonexistent file
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2017-04-12 16:22:43 +02:00
John Spray
015daa2686 Merge pull request #14347 from ukernel/wip-19501
mds: make C_MDSInternalNoop::complete() delete 'this'

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-12 14:28:11 +01:00
Ziye Yang
d70216b9fa bluestore, NVMEDevice: Add multiple thread support for SPDK I/O thread
Previously, we only have one thread to do SPDK I/O, and this patch
adds the multiple thread support.

In this first version, we use the tid of the thread to map the I/Os
of this thread to the corresponding SPDK I/O thread.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
2017-04-12 21:04:33 +08:00
Jason Dillaman
f682d26e6d rbd: import-diff should discard any zeroed extents
Sparse (zeroed) extents cannot be safely skipped. Instead, the
zeroed extent should be discarded from the image to ensure
the import remains consistent with the export.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-04-12 09:00:54 -04:00
Mykola Golub
9a80faa546 Merge pull request #14460 from dillaman/wip-qa-rbd-cli
qa/workunits: corrected issues with RBD cli test

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-04-12 15:44:25 +03:00
Henrik Korkuc
e0bbc70467 client/Client.cc: add feature to reconnect client after MDS reset
Client.cc marks session as stale instead of reconecting after received
reset from MDS. On MDS side session is closed so MDS is ignoring cap
renew. This adds option to reconnect stale client sessions instead of
just marking sessions stale.

Fixes: http://tracker.ceph.com/issues/18757

Signed-off-by: Henrik Korkuc <henrik@kirneh.eu>
2017-04-12 15:27:12 +03:00
Willem Jan Withagen
fc01473e55 ceph-disk/ceph_disk/main.py: fix calling of the bsdrc init scripts
- rc.d/ceph does not really like the extra stop/start as las cmd.
 - And make Start and Stop look similar.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-04-12 14:02:26 +02:00
Jos Collin
12df9a3a18 test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings
The following warning appears during make for several files in the test submodule:
warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]

Fixed the review comment too.

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-04-12 17:27:27 +05:30