Commit Graph

101387 Commits

Author SHA1 Message Date
Yingxin Cheng
b63c1a16d7 crimson/net: fix exception handling chain with shared_future<>
Fix the assertion failure when exception raised by sending ReconnectOk
or ServerIdent at REPLACING state.

Failed assertion:
ceph/src/seastar/include/seastar/core/future.hh:284: void
seastar::future_state_base::set_exception(std::__exception_ptr::exception_ptr&&):
Assertion `_u.st == state::future' failed.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 10:16:39 +08:00
Yingxin Cheng
5f347a8772 crimson/net: stop generating new client_cookie when reconnect
When handshake failed before client receive server_cookie, don't renew
client_cookie, or the session will be reset and pending messages will be
lost.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 10:16:39 +08:00
Yingxin Cheng
1f842bd06d crimson/net: move write exception handling to seastar::repeat()
* prevent exception leak in do_write_dispatch_sweep()
* no exception handling logic inside each repeat()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 10:16:39 +08:00
Yingxin Cheng
7b43b98a48 crimson/net: dispatch writes if messages are requeued
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 10:16:39 +08:00
Yingxin Cheng
c68fb13481 crimson/net: add missing return in Protocol::write_event()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 10:16:39 +08:00
Yingxin Cheng
58dcc60c73 crimson/net: add Connection::is_closed() for tests
Tests need to introspect whether the connection is closed correctly.
For normal users, ms_handle_reset() will notify them when a connection is
closed by messenger.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 10:16:39 +08:00
Yingxin Cheng
6631e3dc01 crimson/net: return bool from Connection::is_connected()
There's no need to return a futurized bool from is_connected() if we are
holding ConnectionRef from the same core. Also add the missing const
qualifier.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-28 09:17:57 +08:00
Patrick Donnelly
18a41277fb
Merge PR #28819 into master
* refs/pull/28819/head:
	doc/man/8/monmaptool: document the `--addv` argument.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-27 15:57:44 -07:00
Kefu Chai
a9469f4b78
Merge pull request #29470 from liu-chunmei/ceph_seastar_cyan_store
crimson: futurized CyanStore's member functions and Collection

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-28 00:47:58 +08:00
Kefu Chai
e0d62124fb crimson/os/cyan_store: remove stale comment
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-28 00:03:53 +08:00
chunmei Liu
0fbedd538c crimson: add FuturizedCollection
we will need to talk to BlueStore, and currently `ceph::os::Collection`
only works with `CyanStore`, so a wrapper around `ceph::os::Collection`
is added. we can subclass it to adapt to CyanStore and to BlueStore.

the underlying concrete `FuturizedStore` classes are responsible to
cast `FuturizedCollection` to their own `Collection` type if they
want to access the interface not exposed by `FuturizedCollection`.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
2019-08-28 00:03:53 +08:00
Kefu Chai
6188e31791 crimson/os: s/try_emplace()/insert()/
avoid specifying the type of `map<>::value_type`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-28 00:03:53 +08:00
Kefu Chai
da7aa4c5e2 crimson: always use Collection::get_cid()
prepare for the futurized Collection, which will be an abstract
interface allowing us to talk to BlueStore with one of its concrete
subclass.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-28 00:03:53 +08:00
Kefu Chai
cb1c9b81f5 os/ObjectStore: fix typo in comments
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 22:55:59 +08:00
Kefu Chai
952b88b009 crimson/osd: lift continuation in PGMeta::load()
after making `open_collection()` return a future, we don't need to chain
the body of `PGMeta::load()` after `omap_get_values()`. instead we can
lift it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 22:55:59 +08:00
chunmei Liu
36ef2b4dec crimson: add futurized interfaces to CyanStore
Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
2019-08-27 22:55:57 +08:00
Mykola Golub
ad4782075b
Merge pull request #29853 from dillaman/wip-41411
test/cli-integration/rbd: fixed missing image and snap ids

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-08-27 16:27:20 +03:00
Yuval Lifshitz
9945a9c084
Merge pull request #29877 from yuvalif/pubsub_regression_issue2
rgw/pubsub: service reordering issue
2019-08-27 16:18:33 +03:00
Nathan Cutler
808d394beb
Merge pull request #29797 from smithfarm/wip-backport-resolve-issue
script: add backport-resolve-issue

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-27 15:14:01 +02:00
Nathan Cutler
91d62cf8fa
Merge pull request #29904 from smithfarm/wip-bci-resolve-parent
script/backport-create-issue: add --resolve-parent feature

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-27 15:07:51 +02:00
Kefu Chai
e0f1ed20b9
Merge pull request #29912 from tchaikov/wip-wheels-only-for-make-check
install-deps.sh: only prepare wheels for "make check"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-27 20:49:01 +08:00
Nathan Cutler
70ebffca56 script: add backport-resolve-issue
This script processes GitHub backport PRs, checking for proper cross-linking
with a Redmine Backport tracker issue and, if a PR is merged and properly
cross-linked, it can optionally resolve the tracker issue and correctly
populate the "Target version" field.

The script takes a single positional argument, which is optional. If the
argument is an integer, it is assumed to be a GitHub backport PR ID (e.g. "28549").
In this mode ("single PR mode") the script processes a single GitHub backport
PR and terminates.

If the argument is not an integer, or is missing, it is assumed to be a
commit (SHA1 or tag) to start from. If no positional argument is given, it
defaults to the tag "BRI", which might have been added by the last run of the
script. This mode is called "scan merge commits mode".

In both modes, the script scans a local git repo, which is assumed to be
in the current working directory. In single PR mode, the script will work
only if the PR's merge commit is present in the current branch of the local
git repo. In scan merge commits mode, the script starts from the given SHA1
or tag, taking each merge commit in turn and attempting to obtain the GitHub
PR number for each.

For each GitHub PR, the script interactively displays all relevant information
(NOTE: this includes displaying the GitHub PR and Redmine backport issue in
web browser tabs!) and prompts the user for her preferred disposition.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-08-27 14:42:37 +02:00
Kefu Chai
b9bdd34d0a
Merge pull request #29790 from hjwsm1989/fix-unittest-osdmap
tests/osd: fix typo in unittest_osdmap

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-27 19:42:21 +08:00
Yuval Lifshitz
e91489f443 rgw/pubsub: service reordering issue
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-08-27 14:36:08 +03:00
Kefu Chai
ff631a19b4 install-deps.sh: only prepare wheels for "make check"
* move `for_make_check` to the beginning of script, as FreeBSD will also
  use this variable
* extract `preload_wheels_for_tox()` function out to improve readability
* call `preload_wheels_for_tox()` only if `for_make_check` is true

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 17:53:28 +08:00
Luca Castoro
046f5990c3 doc/man/8/monmaptool: document the --addv argument.
Fixes: http://tracker.ceph.com/issues/40568
Signed-off-by: Luca Castoro <luca.castoro@brightcomputing.com>
2019-08-27 17:47:13 +08:00
Kefu Chai
b0aec38341
Merge pull request #29090 from liewegas/wip-40792
mon/MonClient: ENXIO when sending command to down mon

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-27 17:34:13 +08:00
Kefu Chai
a08f4a30f9
Merge pull request #28177 from jecluis/wip-noreply-failures
mon: ensure prepare_failure() marks no_reply on op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-08-27 17:32:54 +08:00
Nathan Cutler
e4d6312fa2 script/backport-create-issue: add --resolve-parent feature
When --resolve-parent is provided on the command line, the script
will check the status of backport issues for those parent issues
all of whose backport issues already exist. If all the backport
issues are in status "Resolved", the parent issue's status is set
to "Resolved" as well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-08-27 11:22:46 +02:00
Kefu Chai
a342158bb7
Merge pull request #29862 from tchaikov/wip-41416
rgw,bluestore: fixes to address failures from check-generated.sh

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-27 17:16:41 +08:00
Xie Xingguo
6f74759cf4
Merge pull request #29893 from xiexingguo/wip-inc-recovery-5
osd/osd_types: inc-recovery - add special handler for lost_revert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-27 13:16:20 +08:00
Kefu Chai
bf0b161115 test/encoding/check-generated.sh: show diff if cmp fails
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 13:00:41 +08:00
Kefu Chai
2c5e59c1fb test/encoding/check-generated.sh: run loop in the same shell
set parent scope variables in the same shell.

foo | while read ....

in the above statement, `while read ...` is executed in a subshell. so
it cannot change the bash variables in its parent shell.

in this change, the output of `foo` is redirected to the stdin of `while
read` statement. so we can capture the test failures.

before this change, the test always succeed, even if there are failures.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 13:00:41 +08:00
Kefu Chai
c1b47b62e4 common/DecayCounter: do not decay when testing denc
check-generated.sh tries to verify the invariance of a type after
encoding and decoding. so we should keep the value of `DecayCounter` the
same when encoding/decoding/dumping.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 13:00:41 +08:00
Kefu Chai
123f42d0aa rgw: rgw_log_entry always initialize total_time
zero-initialize rgw_log_entry::total_time

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:58 +08:00
Kefu Chai
3e65551d0a mgr/ServiceMap: use plain gid
* partially revert 5c25a018. which is not backward compatible.
* change `ServiceMap::get_daemon()` so it returns a
  `pair<Daemon*,bool>`.

git is an `optional<uint64_t>`, so we cannot dump it without checking.

Fixes: https://tracker.ceph.com/issues/41424
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:58 +08:00
Kefu Chai
1726bf21ab osd: init pg_pool_t::pg_autoscale_mode in ctor
so its value is determined

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:58 +08:00
Kefu Chai
4c9e142b77 osd: use scoped enum for pg_pool_t::pg_autoscale_mode
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:57 +08:00
Kefu Chai
0c1836d3d3 osd: use in-class initialization in pg_pool_t
less repeat this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:57 +08:00
Kefu Chai
8bc4352725 os/bluestore: implement copy ctor and assignment for bluestore_blob_use_tracker_t
to address the `check-generated.sh` test failure.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:57 +08:00
Kefu Chai
6833234fd2 rgw: RGWObjManifest::obj_iterator::seek always update locations
we should update explicit loc and location even if
`ofs >= manifest->obj_size`. there is chance that we are updating an end
iterator whose ofs is equal to obj_size. before being updated, the end
iterator points to an implicit location, while after being updated, the
manifest could be using an explicit location, so we should update the
end iterator as well.

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:57 +08:00
Kefu Chai
690267208d rgw: RGWObjManifest::obj_iterator: init member variables in-class
* remove the init() member function which is solely used by the
  constructors.
* use constructor delegation to minimize the repeating

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:57 +08:00
Kefu Chai
1efc7b1d74 rgw: update_iterator() in RGWObjManifest::set_explicit()
we should update the end iterator when updating RGWObjManifest with its
parts

Fixes: https://tracker.ceph.com/issues/41416
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 12:41:57 +08:00
Kefu Chai
bad4ccc7bf
Merge pull request #29913 from tchaikov/wip-debian/control
debian/control: fix Build-Depends

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-08-27 10:28:00 +08:00
Kefu Chai
f6d7fd28af debian/control: fix Build-Depends
it's a regression introduced by 5d6d770e

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-27 10:05:17 +08:00
Patrick Donnelly
babd27c9e8
Merge PR #29867 into master
* refs/pull/29867/head:
	doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-26 14:09:39 -07:00
Neha Ojha
04b5c38ce4
Merge pull request #29822 from vumrao/wip-vumrao-41399
os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-08-26 12:57:58 -07:00
Vikhyat Umrao
96d48af325 os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
2019-08-26 12:40:36 -07:00
Kefu Chai
f3d3e37c7e
Merge pull request #29903 from tchaikov/wip-pip-download
install-deps.sh: download wheel using 'pip wheel'

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-27 00:44:28 +08:00
Kefu Chai
3562504afd install-deps.sh: download wheel using 'pip wheel'
otherwise we will fail to install the build dependencies of
`lazy-object-proxy` from the wheelhouse. as `lazy-object-proxy` does not
add `setuptools_scm` in its `setup.py`, instead it lists
`setuptools_scm` in `setup.cfg` and `pyproject.toml` as a `build-system`
requires. but unfortunately, `pip download` only downloads the
install/run-time dependencies at this moment. and `lazy-object-proxy`
does not offer binary package for at least python2.7.

ideally, `pip download` should collects its dependencies like

Collecting setuptools_scm>=3.3.1 (from lazy-object-proxy->astroid<3,>=2.2.0->pylint->-r requirements-lint.txt (line 1))

so we need to use `pip wheel` do download build-time dependencies

see also https://github.com/pypa/pip/issues/6222

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-26 23:06:39 +08:00