Commit Graph

81700 Commits

Author SHA1 Message Date
Sage Weil
6d7d8ba94b osd/PrimaryLogPG: sync_complete for C_OSD_AppliedRecoveryObject[Replica]
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:05:38 -06:00
Sage Weil
779e01085b osd/PG: sync_complete for C_UpdateLastRollbackInfoTrimmedToApplied
One of these is attached to every write!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:05:38 -06:00
Sage Weil
4df1645ee5 osd: allow BlessedContexts to finish synchronously
If bluestore chooses to it may try to call sync_finish() from the queueing
call chain (instead of finish() from a Finisher).  Allow it for several
Contexts in PG and PrimaryLogPG, including those used for the main IO
path.

We assume here that all Contexts that we bless can complete synchronously
by calling their normal finish() method.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:05:38 -06:00
Sage Weil
df1c7308c9 os/bluestore: simplify completion roll-ups
No need to wrap these in C_Contexts and do a new allocation; just slurp
up the list elements directly.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:05:38 -06:00
Sage Weil
7bca217ede os/bluestore: call onreadable from queue_transaction
Bluestore updates are immediately present in the cache and readable.  The
only exception are omap updates, but the read methods there block until
they commit, so we can still tell the OSD that they are readable.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:04:57 -06:00
Sage Weil
b877860e42 osd: flush before collection_list()
We would get this implicitly with FileStore if we waited for the onreadable
callbacks, but in some cases the OSD has already done that.  With BlueStore,
we need to explicitly flush().

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:04:57 -06:00
Sage Weil
eff2a4d24b include/Context: add sync_complete support to C_Contexts
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:04:57 -06:00
Sage Weil
dfd52c2bc3 include/Context: add sync_complete() hook
Sometimes we are able to complete a context synchronously, within the same
callchain of the caller who queued it.  In this case the locking rules are
usually a bit different for the caller.  Add a generic Context hook that
allows this.

If a sync-capable Context implements sync_complete, it can do whatever
it needs to do for this particular event.  If it is not implemented, the
generic implementation will return false, and the caller can use the
normal complete() as it normally would have (presumably by calling it
asynchronously, e.g., in a Finisher).

If sync_complete() is implemented, it *must* return true.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:04:57 -06:00
Sage Weil
f46efeb47c common/Finisher: simplify
The second _rval list was a dumb idea.  A vector of pairs is simpler
and more efficient.

Also, extend support to any container type.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-11 15:04:57 -06:00
Mykola Golub
3ea5799cde test/rbd_mirror: "use of uninitialised value" valgrind warning
The `on_call` context serves as a barrier and should be completed
after the `on_start_ctx` context is assigned.

The warning was observed sporadically e.g. by repeating
WaitingOnNonLeaderAcquireLeader test under valgrind.

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-12-11 22:56:40 +02:00
Patrick Donnelly
0a40f4fb46
mds: organize Filesystem class def
This follows the coding style guidelines:

https://google.github.io/styleguide/cppguide.html#Declaration_Order

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-11 11:51:06 -08:00
Gregory Farnum
da7d071654
Merge pull request #19191 from tchaikov/wip-mon-cleanup
mon: remove_is_write_ready()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-12-11 11:46:19 -08:00
Casey Bodley
5d4da4598d
Merge pull request #17438 from mikulely/fix-mstop
vstart.sh: fix mstop.sh can not stop rgw

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-12-11 14:11:06 -05:00
Dmitry Plyakin
56344f0e14 rgw: fix GET website response error code
Change NoSuchKey error code to NoSuchWebsiteConfiguration, when bucket doesn't have website configuration.

Fixes: http://tracker.ceph.com/issues/22272
Signed-off-by: Dmitry Plyakin <dplyakin@gmail.com>
2017-12-11 20:44:05 +03:00
Kefu Chai
e6695bb63a cmake: link executables against ceph-common
so they can have access to libstdc++ if it is linked statically.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-12 00:26:52 +08:00
Kefu Chai
4c2216de6a cmake: add WITH_STATIC_LIBSTDCXX option
to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-12 00:26:52 +08:00
Patrick Donnelly
6f3553351d
Merge PR #19432 into master
* refs/pull/19432/head:
	doc: correct the new fs allow opts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-11 08:18:11 -08:00
Nathan Cutler
aea9fa01ae doc: globally change CRUSH ruleset to CRUSH rule
Since kraken, Ceph enforces a 1:1 correspondence between CRUSH ruleset and
CRUSH rule, so effectively ruleset and rule are the same thing, although
the term "ruleset" still survives - notably in the CRUSH rule itself, where it
effectively denotes the number of the rule.

This commit updates the documentation to more faithfully reflect the current
state of the code.

Fixes: http://tracker.ceph.com/issues/20559
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-12-11 17:15:23 +01:00
Mykola Golub
65b92c40ac pybind/rbd: add deep_copy method
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-12-11 18:09:10 +02:00
Kefu Chai
8a0935988f install-deps.sh: use GCC-7 on xenial also
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-11 23:26:24 +08:00
Jason Dillaman
f10904b617
Merge pull request #19416 from trociny/wip-flatten-cleanup
librbd: remove unused member in FlattenRequest

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-12-11 10:01:23 -05:00
Wido den Hollander
3f07636bc2
mgr/influx: Print (in debug) time it takes to send data to Influx
On large clusters it might take a very long time to send data to Influx
due to the gathering and parsing of statistics.

By keeping a counter and printing it admins can adjust the interval if it
becomes to heavy for their cluster.

Signed-off-by: Wido den Hollander <wido@42on.com>
2017-12-11 15:44:05 +01:00
Wido den Hollander
429718af1c
mgr/influx: Revise configuration of module
It's a lot like the Zabbix module and allows for setting configuration
options on run-time and also fetch them from the module.

A few additional commands have been registered to make sure it is easy
to interact with the module.

Signed-off-by: Wido den Hollander <wido@42on.com>
2017-12-11 15:33:53 +01:00
Wido den Hollander
2b71a8919c
mgr/influx: Send fsid as a tag to Influx to identify cluster
This allows for multiple Ceph clusters to send their data to the
same Influx database.

Using the fsid values for different clusters can be queried from
Influx

Signed-off-by: Wido den Hollander <wido@42on.com>
2017-12-11 15:30:42 +01:00
Wido den Hollander
9440be3da2
mgr/influx: PEP-8 fixes to InfluxDB module
The code now scores:

  Your code has been rated at 8.94/10

Various indentation fixes, whitespaces and other PEP-8 related changes

Signed-off-by: Wido den Hollander <wido@42on.com>
2017-12-11 15:29:08 +01:00
Dan van der Ster
9d55d17c4e doc: correct the new fs allow opts
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2017-12-11 13:32:59 +01:00
Tianshan Qu
a0b51eef86 rgw: fix recursive lock
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
2017-12-11 19:48:00 +08:00
Kefu Chai
2d641bde20
Merge pull request #19426 from tchaikov/wip-cmake
cmake: disable FAIL_ON_WARNINGS for rocksdb
2017-12-11 16:34:41 +08:00
Kefu Chai
6559a85051 cmake: disable FAIL_ON_WARNINGS for rocksdb
otherwise -Werror=implicit-fallthrough will fail the build with GCC-7

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-11 14:09:45 +08:00
kungf
92910a9535 osd: reduce all_info map find to get primary
Signed-off-by: kungf <yang.wang@easystack.cn>
2017-12-11 11:00:18 +08:00
Yan, Zheng
7ffa87e6a2 osdc/Journaler: add 'stopping' check to various finish callbacks
These callbacks are executed by finisher. When they are being executed,
Journaler can be in stopping state.

Fixes: http://tracker.ceph.com/issues/22360
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-12-11 09:12:04 +08:00
Patrick Donnelly
d2ebe22a49
Merge PR #19373 into master
* refs/pull/19373/head:
	client: drop duplicate friend

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-09 16:34:24 -08:00
Kefu Chai
717591016d
Merge pull request #19417 from tchaikov/wip-install-deps-gcc
install-deps.sh: install new gcc as the default the right way

Reviewed-by: Sage Weil <sage@redhat.com>
2017-12-10 00:38:59 +08:00
Sage Weil
a5eb976cb3 qa/suites/rados: add missing openstack volumes
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-09 10:20:19 -06:00
Kefu Chai
3882deae8c install-deps.sh: install new gcc as the default the right way
* should install software-properties-common beforehand, otherwise
  the `add-apt-repository` command will not be available.
* the update-alternative commandline were copied from ceph-build,
  should remove the escape characters.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-09 20:00:28 +08:00
Mykola Golub
41bdd8b6d4 librbd: remove unused member in FlattenRequest
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-12-09 11:47:01 +02:00
Yan, Zheng
debb556076 mds: properly eval locks after importing inode
We should call Locker:eval() for all imported inodes who have non-zero
'wanted caps'. MDS does not properly handle following case.

- client open a inode for read, it send a cap message to MDS.a (the cap
  message updates 'wanted caps')
- MDS.a receive the cap message, the inode is non-auth and is ambiguous
  auth. MDS.a can not request 'wanted caps' from auth mds.
- MDS.a finishes importing the inode from. But no caps are imported and
  mds_caps_wanted map is empty.

The bug can cause read hang.

Fixes: http://tracker.ceph.com/issues/22357
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-12-09 15:05:11 +08:00
Kefu Chai
74a132d826
Merge pull request #19398 from tchaikov/wip-install-deps-use-dts
install-deps.sh: use DTS on centos if GCC is too old

Reviewed-by: Sage Weil <sage@redhat.com>
2017-12-09 11:51:33 +08:00
Shinobu Kinjo
fb50bde049 journal: Trivial cleanup
Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>
2017-12-09 12:42:32 +09:00
David Zafman
0f764890d9
Merge pull request #19368 from dzafman/wip-22086
ceph-objectstore-tool: Add option "dump-import" to examine an export

Reviewed-by: Sage Weil <sage@redhat.com>
2017-12-08 19:37:45 -08:00
Kefu Chai
b38621f302 ceph.spec: use dts-7 on rhel
devtoolset-7 is available for RHEL, but not for centos. only dts-6 is
available for centos.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-09 11:18:27 +08:00
David Zafman
c4602c9ac8 test: ceph_objectstore_tool.py: Perform dump-import
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-12-08 18:50:04 -08:00
root
dfabb21d9f mds: Improve the log info for read pointers
Signed-off-by: dongdong tao <tdd21151186@gmail.com>

fix
2017-12-09 09:30:22 +08:00
Sage Weil
a16ff1a7cd
Merge pull request #19382 from rishabh-d-dave/docfix-ceph-osd-ls
doc/man: add "ls" to "ceph osd" command's subcommands list
2017-12-08 18:31:35 -06:00
Jesse Williamson
a286fd35ce osd: migrate PGLOG_* macros to constexpr
Fixes: http://tracker.ceph.com/issues/20811
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2017-12-08 10:10:42 -08:00
Jan Fajerski
0457043d2f mon: reenable timer to send digest when paxos is temporarily inactive
Fixes: http://tracker.ceph.com/issues/22142

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-12-08 16:58:23 +01:00
songweibin
9eff61bf52 pybind/rbd: support open the image by image_id
We can initialize an image with image name or image id.

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2017-12-08 23:58:16 +08:00
Jason Dillaman
53be4ef96c
Merge pull request #19005 from shinobu-x/src_krbd
rbd: drop unnecessary using declaration, etc

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-12-08 10:18:22 -05:00
Yao Zongyou
4586cfbf99 rgw-admin: properly filtering bucket stats by user_id or bucket_name
If both --uid and --bucket args are provided, properly filtering
bucket stats by user_id and bucket_name.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
2017-12-08 21:26:09 +08:00
Kefu Chai
4cedada14e install-deps.sh: enable testing repo on centos on aarch64
the DTS-6 is only available in the testing repo

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-08 21:15:53 +08:00