Commit Graph

70847 Commits

Author SHA1 Message Date
Kefu Chai
472626b4ec README.md: use github heading syntax to mark the headings
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-17 23:49:32 +08:00
John Spray
4314e3d1c5 Merge pull request #14267 from yunfeiguan/master
cephfs: fix mount point break off problem after mds switch occured 

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-15 19:46:53 +01:00
John Spray
c65e4fbf8e Merge pull request #13587 from yanghonggang/master
cephfs: fix write_buf's _len overflow problem

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-15 19:45:54 +01:00
John Spray
177699981e Merge pull request #13522 from singler/wip-reconnect-client
client/Client.cc: after reset session from MDS - reconnect

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-15 19:44:53 +01:00
John Spray
0cb0c494d9 Merge pull request #13466 from jcsp/wip-cephfs-ecoverwrites
Add test for is_hacky_ecoverwrites in cephfs pool checks

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-15 19:44:00 +01:00
John Spray
f412345590 Merge pull request #14207 from jcsp/wip-19401
mds: ignore ENOENT on writing backtrace

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-15 19:40:59 +01:00
John Spray
39bd183349 Merge pull request #13579 from ukernel/wip-19022
client: drop cap snaps when auth mds session gets closed

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-15 12:15:04 +01:00
Sage Weil
d82ac99d70 Merge pull request #14490 from liewegas/wip-no-luminous
mon: add mon_debug_no_require_luminous

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-14 20:05:45 -05:00
Matt Benjamin
260de938f0 Merge pull request #14553 from guihecheng/rgw_file-revert
rgw_file: fix up potential race condition

verified f23
2017-04-14 14:30:38 -04:00
Kefu Chai
43a2f3ed33 Merge pull request #14546 from dachary/wip-19628-directory-boot
ceph-disk: enable directory backed OSD at boot time

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-15 00:06:50 +08:00
Sage Weil
8c6820a433 qa/suites/rados/thrash: do not thrash cluster full without luminous flag
The interface changes with the luminous flag; this will fail.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-14 12:04:47 -04:00
Sage Weil
4857f51e68 qa/suites/rados: expand other collections with no-require-luminous
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-14 11:45:05 -04:00
Kefu Chai
a29be2ad62 Merge pull request #14551 from tchaikov/wip-19584
mon/OSDMonitor: transit creating_pgs from pgmap when upgrading

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-14 22:42:34 +08:00
Sage Weil
2a4e431599 Merge pull request #14435 from liewegas/wip-bluestore-deferred
os/bluestore: tune deferred_batch_ops separately for hdd and ssd

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-04-14 08:35:50 -05:00
Sage Weil
08d3fa490f Merge pull request #14489 from liewegas/wip-bluestore-fix-flush
os/bluestore: fix narrow osr->flush() race
2017-04-14 08:35:35 -05:00
Sage Weil
16d3a524d8 mon/OSDMonitor: require_luminous_osds for pg-remap[-items]
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-14 09:21:17 -04:00
Loic Dachary
f425a127b7 ceph-disk: enable directory backed OSD at boot time
539385b143
introduced a regression preventing directory backed OSD from starting at
boot time.

For device backed OSD the boot sequence starts with ceph-disk@.service
and proceeds to

    systemctl enable --runtime ceph-osd@.service

where the --runtime ensure ceph-osd@12 is removed when the machine
reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at
boot time.

However directory backed OSD solely rely on the ceph-osd@.service unit
to start at boot time and will therefore fail to boot.

The --runtime flag is selectively set for device backed OSD only.

Fixes: http://tracker.ceph.com/issues/19628

Signed-off-by: Loic Dachary <loic@dachary.org>
2017-04-14 12:14:18 +02:00
Gui Hecheng
a32add15b8 rgw_file: add locks to fix potential race
Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
2017-04-14 17:53:04 +08:00
John Spray
d51c027311 Merge pull request #13859 from jcsp/wip-19204
mds: shut down finisher before objecter

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-14 10:50:14 +01:00
John Spray
f61c0ff395 Merge pull request #14505 from jcsp/wip-19566
mds: fix mgrc shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-04-14 10:49:20 +01:00
John Spray
517dbfdfa4 Merge pull request #14398 from jcsp/wip-19551
mon: emit cluster log messages on MDS health changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-04-14 10:48:25 +01:00
John Spray
131d1bd570 qa: add log whitelists for MDS health messages
Now that we send these to the cluster log, we must
whitelist them in the tests that exercise those
unhealthy states.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-14 05:47:43 -04:00
Kefu Chai
27dc8a1685 mon/OSDMonitor: transit creating_pgs from pgmap when upgrading
there could be some pg(s) still being created when we are upgrading to
luminous, and the pools holding them are not changed in the sense of
pg_pool_t::last_change after the upgrade and before we scan for
creating pgs. in that case, the existing update_pending_creatings()
will fail to collect the pgs being created before the upgrade.

with this change, the creating_pgs in pgmap are also used for updating
the OSDMonitor's creating_pgs if it's updated.

but we should stopupdating the pgmap once the upgrade completes. i.e.
stop dispatching MSG_PGSTATS messages to PGMonitor if the quorum and all
osds are luminous.

Fixes: http://tracker.ceph.com/issues/19584
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-14 16:45:23 +08:00
Gui Hecheng
accf8cfd22 Revert "rgw_file.h: fix potential data race condition"
This reverts commit 3f7b411e1e.
Causing hung up due to double lock.

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
2017-04-14 16:17:01 +08:00
Kefu Chai
125f27e3e8 Merge pull request #10851 from atheism/rgw-s3-doc-update
doc/radosgw/s3/cpp.rst: update usage of libs3 APIs to make the examples work

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-14 10:18:10 +08:00
Brad Hubbard
7aa5792d19 Merge pull request #14469 from badone/wip-return-error-on-missing-client-id
mds: Return error message instead of asserting

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-14 08:10:59 +10:00
Sage Weil
ef5e1d31e7 mon/OSDMonitor: slightly better debug output
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 17:11:19 -04:00
Sage Weil
7959760d91 mon/OSDMonitor: fix creating pg trimming during upgrade
The creating pgs are in the PGMap with the CREATING
state bit set.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 17:11:19 -04:00
Sage Weil
8fd450e986 qa/suites/rados: require_luminous_osds at end of run, then scrub
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 17:11:19 -04:00
Sage Weil
5ca72c1193 qa/tasks/exec_on_cleanup.py: add
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 17:11:19 -04:00
Loic Dachary
6e20d648e9 Merge pull request #14508 from smithfarm/wip-doc-dev
doc: dev guide: how to run s3-tests locally against vstart

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-04-13 20:54:46 +02:00
Sage Weil
9e88a71105 os/bluestore: tune deferred_batch_ops separately for hdd and ssd
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 14:09:14 -04:00
Sage Weil
9967fceb62 Merge pull request #14420 from optimistyzy/329
os/bluestore/NVMEDevice: Add multiple thread support for SPDK I/O thread

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-04-13 12:34:51 -05:00
Nathan Cutler
0dd2dfce1d doc: dev guide: how to run s3-tests locally against vstart
Add a bunch of verbiage to the Developer Guide

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-04-13 19:14:52 +02:00
Matt Benjamin
9a5cdf3b48 Merge pull request #14458 from linuxbox2/wip-rgw_file-readahead
rgw_file: support readdir cb type hints (plus fixes)
2017-04-13 13:12:27 -04:00
John Spray
177a97d5c5 mds: shut down finisher before objecter
Some of the finisher contexts would try to call into Objecter.
We mostly are protected from this by mds_lock+the stopping
flag, but at the Filer level there's no mds_lock, so in the
case of file size probing we have a problem.

Fixes: http://tracker.ceph.com/issues/19204
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 12:43:39 -04:00
John Spray
ccd6c81742 Merge pull request #13966 from jcsp/wip-client-logmsg
client: specify inode in get_caps log message

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-04-13 17:43:10 +01:00
John Spray
e1b4327d67 Merge pull request #14087 from jcsp/wip-doc-fscid
doc: how to specify filesystem for cephfs clients

 Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-13 17:42:13 +01:00
John Spray
3fccc2372f mds: ignore ENOENT on writing backtrace
We get ENOENT when a pool doesn't exist.  This can
happen because we don't prevent people deleting
former cephfs data pools whose files may not have
had their metadata flushed yet.

http://tracker.ceph.com/issues/19401
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 12:20:25 -04:00
Matt Benjamin
007b7451c2 rgw_file: don't expire directories being read
If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-13 11:30:31 -04:00
John Spray
01f79a748b mon: emit cluster log messages on MDS health changes
Previously, when we got a beacon that updated the health
metrics for an MDS, the user would just see mysterious-looking
cluster log messages indicating a rising fsmap epoch number.

It would be good to do this for health messages in general at
some point, but for now just do it for the MDS ones.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 11:06:13 -04:00
John Spray
fc28d29c0c mds: fix mgrc shutdown
Were previously only tearing MgrClient down when not
holding a rank, leading to it trying to continue
to run after monclient was shut down.

Fixes: http://tracker.ceph.com/issues/19566
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-13 11:00:46 -04:00
Matt Benjamin
e0191d74e3 rgw_file: chunked readdir
Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-13 10:58:22 -04:00
Matt Benjamin
2e66c7a7cc rgw_file: introduce rgw_lookup type hints
The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-13 10:35:37 -04:00
Kefu Chai
8442cbb521 Merge pull request #14468 from tchaikov/wip-show-me-fake-plugin
qa/workunits/ceph-helpers: display rejected string

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-13 22:24:03 +08:00
Sage Weil
20e5bff7fc mon/OSDMonitor: fix initial map when require_luminous_osds not set on mkfs
If we don't set the luminous flag, we should not set the new luninous
fields or else we'll get a crc mismatch.  (Funnily that happens in the
epoch where the flag is eventually set and the encoded map finally includes
the field we have set in memory.)

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 10:10:26 -04:00
Sage Weil
10ea8e1d7d mon/OSDMonitor: ignore health checks about require_luminous if debug flag is set
This makes it tedious for teuthology health checks to proceed when we
deliberately run luminous osds without this flag.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-13 10:10:26 -04:00
Yang Honggang
eb915d0eec cephfs: fix write_buf's _len overflow problem
After I have set about 400 64KB xattr kv pair to a file,
mds is crashed. Every time I try to start mds, it will crash again.
The root reason is write_buf._len overflowed when doing
Journaler::append_entry().

This patch try to fix this problem through the following changes:

 1. limit file/dir's xattr size
 2. throttle journal entry append operations

Fixes: http://tracker.ceph.com/issues/19033
Signed-off-by: Yang Honggang joseph.yang@xtaotech.com
2017-04-13 20:09:07 +08:00
John Spray
b81e4b70a4 Merge pull request #14234 from jcsp/wip-19406-dirhash
tools/cephfs: set dir_layout when injecting inodes

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-04-13 12:01:28 +01:00
Kefu Chai
051f9138df Merge pull request #14472 from joscollin/wip-warning-test
test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-13 16:28:12 +08:00