Commit Graph

116331 Commits

Author SHA1 Message Date
Kefu Chai
8da52eab4c cmake: link unittest_ipaddr against GTest::Main
otherwise its main() is not defined, and fails to link.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-20 14:09:49 +08:00
Milind Changire
50e680cb50 doc: document the mds_autoscaler mgr plugin
The MDS Autoscaler provides automation to maintain a required count of
MDSs in the system.
The MDS count in the system can be afected by:
* max_mds config option
* standby_count_wanted config option
* death of an active MDS Rank

Fixes: https://tracker.ceph.com/issues/46884
Signed-off-by: Milind Changire <mchangir@redhat.com>
2020-10-20 09:19:48 +05:30
chencan
768d7fc4e8 mon/MDSMonitor: do not ignore mds's down:dne request
Fixes: https://tracker.ceph.com/issues/47881
Signed-off-by: chencan <chen.can2@zte.com.cn>
2020-10-20 11:07:43 +08:00
Patrick Donnelly
4e2f47fc4e
Merge PR #37646 into master
* refs/pull/37646/head:
	mds: account for closing sessions in hit_session

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-19 19:28:40 -07:00
Patrick Donnelly
0fcff12086
Merge PR #37608 into master
* refs/pull/37608/head:
	mon/MDSMonitor: divide mds identifier and mds real name with dot

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-19 19:27:48 -07:00
Patrick Donnelly
008eaca6c1
Merge PR #37529 into master
* refs/pull/37529/head:
	qa: set rados op timeouts for mds/ceph-fuse
	qa: print debug info on mount cleanup
	qa: remove redundant rmr
	qa: use null mode to prevent undesired changes to mountpoint
	qa: unmount all clients before deleting the file system
	osdc: add timeout configs for mons/osds
	common: accept timespan for SaferCond.wait_for

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-10-19 19:26:52 -07:00
Patrick Donnelly
bc40523032
Merge PR #37294 into master
* refs/pull/37294/head:
	mds: assert when inode has no parent directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-19 19:25:29 -07:00
Samuel Just
bff80b3dc2 test/crimson/.../test_transaction_manager: reset ephemeral segment_manager state on replay
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:38:12 -07:00
Samuel Just
274e962bfe crimson/os/seastore/transaction: introduce weak transactions
This way, we can do a bulk scan of the store without building up
an unbounded amount of state in Transaction::read_set.  Note that
such transactions will not be snapshot isolated.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:38:09 -07:00
Samuel Just
c82610c070 crimson/.../lba_manager/btree: hold a reference to parent until added to cache
Currently, we need to rely on the Transaction::read_set to ensure cache
residence of a leaf node until TransactionManager adds the logical
extent to the cache.  The next patch, however, will introduce a lazy
flag for Transaction's to enable doing snapshot inconsistent scans
without populating the read_set, so we'll want this to work without
it.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:18:14 -07:00
Samuel Just
bed1b7bb30 crimson/os/seastore/transaction: rearrange to put private members at end
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:18:14 -07:00
Samuel Just
7be1e5bf4c crimson/os/seastore/lba_manager: add scan_mappings
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:18:14 -07:00
Samuel Just
fe4893ce6a crimson/os/seastore/segment_manager: expose segment_state_t
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:18:14 -07:00
Samuel Just
71a009ef09 crimson/os/seastore/segment_manager/ephemeral: fix segment bounds checks
Also adds some debugging.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-10-19 15:18:14 -07:00
Joshua Schmid
7d644cca9e
Merge pull request #37644 from mgfritch/orch-device-lsm_data
mgr/orchestrator: set `lsm_data` to default of Dict type
2020-10-19 16:21:56 +02:00
Or Friedmann
5d5f9a0d41 rgw: Add request timeout beast
Add request timeout beast

The beast frontend will use the same parameter as the civetweb one, request_timeout_ms and will be configured to 65 seconds by default

Fixes: https://tracker.ceph.com/issues/45431
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
2020-10-19 15:42:18 +03:00
Kiefer Chang
99e4210657 mgr/cephadm: do not configure Dashboard Ganesha settings
The Dashboard can get cluster information from the Orchestrator.
For settings that are set by previous revisions, the Dashboard will
check them and ask user to remove them.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-19 19:11:17 +08:00
Kiefer Chang
a5aaaa69cc doc/dashboard: add information for Orchestrator NFS-Ganesha clusters
Fixes: https://tracker.ceph.com/issues/46492
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-19 19:11:17 +08:00
Kiefer Chang
a9accaeccf mgr/dashboard: support Orchestrator and user-defined Ganesha clusters
This change make the Dashboard support two types of Ganesha clusters:

- Orchestrator clusters (Since Octopus)
  - Deployed by the Orchestrator.
  - The Dashboard gets the pool/namespace that stores Ganesha
    configuration objects from the Orchestrator.
  - The Dashboard gets the daemons in a cluster from the Orchestrator.

- User-defined clusters (Since Nautilus)
  - Clusters defined by using `ceph dashboard
    set-ganesha-clusters-rados-pool-namespace` command is treated as
    user-defined clusters.
  - Each daemon has its own RADOS configuration objects. The
    Dashboard uses these objects to deduce daemons.

Fixes: https://tracker.ceph.com/issues/46492
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-19 19:11:14 +08:00
Kiefer Chang
b88638873b mgr/dashboard: refator orchestrator service and daemon APIs
- Allow listing services by service_type.
- Allow listing daemons by daemon_type.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-19 19:09:28 +08:00
Kefu Chai
3ec03efc70
Merge pull request #37175 from changchengx/log_start
common: remove log_early configuration option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 18:44:29 +08:00
Lenz Grimmer
3ca068c0f9
Merge pull request #37617 from rhcs-dashboard/46869-sass-modules
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-10-19 11:56:55 +02:00
Kefu Chai
62a3498fc8
Merge pull request #37700 from tchaikov/wip-build-integration-branch
build-integration-branch: send token in the header

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-10-19 17:21:51 +08:00
Lenz Grimmer
86f6a89c1b
Merge pull request #37564 from s0nea/wip-dashboard-nfs-daemons-param
mgr/dashboard/qa: consider `daemons` parameter

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-19 10:42:22 +02:00
Kefu Chai
2e9db1c936
Merge pull request #37658 from neha-ojha/wip-crimson-suite
qa/crontab/teuthology-cronjobs: add crimson-rados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 14:56:00 +08:00
Changcheng Liu
dbdcb2535d common: remove log_early configuration option
After deciding to always enable tracking log in early phase, there's no
need to keep "log_early" option here and remove it directly.

Suggested-by: Kefu Chai <kefu@redhat.com>
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-19 14:30:28 +08:00
Changcheng Liu
71d4178ff4 common: remove useless log_early check condition here
The code logic shows that Log thread needs to be started even
log_early is false.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-19 14:23:51 +08:00
Kefu Chai
a9a4495693
Merge pull request #37315 from changchengx/paxos_extend
mon: define paxos services indexes using enum and cleanups

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.com>
2020-10-19 14:15:01 +08:00
Kefu Chai
337172a4b0
Merge pull request #36536 from matthewoliver/pick_address_warning_doc_update
common/pick_address: support IPv4 and IPv6 dual stack mode

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 14:12:52 +08:00
Kefu Chai
61c63b865b
Merge pull request #37670 from changchengx/crypto_plugin
crypto: remove useless code and unify file name variable

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 14:10:06 +08:00
Kefu Chai
a40337c2d7
Merge pull request #37635 from jecluis/wip-46816
mon: have 'mon stat' output json as well

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 14:08:09 +08:00
Kefu Chai
72c6daaf95
Merge pull request #33392 from alram/master
osdmaptool: add --mark-{up,in} and --adjust-crush-weight options

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 12:44:51 +08:00
Patrick Donnelly
21e63cd4ae
Merge PR #36603 into master
* refs/pull/36603/head:
	test: add test for setting ceph mirror virtual xattr
	tests/pybind/cephfs: cleanup xattrs before starting tests
	client: filter ceph.* xattrs from listing
	client: force an attr fetch for ceph xattrs
	client: changes for ceph.mirror.info xattr
	mds: restrict setting/removing certain xattrs in ceph namespace
	mds: introduce ceph.mirror.info virtual xattr
	mds: customize xattr handling using dispatch handlers
	mds: introduce is_ceph_vxattr() helper

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-18 18:44:51 -07:00
Mykola Golub
b94369f7c1
Merge pull request #37699 from dillaman/wip-47880
journal: possible race condition between flush and append callback

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-10-18 15:43:51 +03:00
Kefu Chai
891e4e80e8 build-integration-branch: change to be pep8 compliant
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-17 11:24:22 +08:00
Kefu Chai
4efb436e2b build-integration-branch: send token in the header
see
https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param/

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-17 11:09:53 +08:00
Kefu Chai
def1b6cd53
Merge pull request #37698 from tchaikov/wip-crimson-do-until
crimson/common: schedule action only if the future is not available

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-10-17 09:32:23 +08:00
Kefu Chai
b8068df1a3
Merge pull request #37695 from tchaikov/wip-dashboard-pep8
mgr/dashboard: do not use "l" for variable name

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-10-17 01:32:59 +08:00
Mykola Golub
fc483304ed
Merge pull request #37645 from dillaman/wip-librbd-aio-completion-race
librbd: update AioCompletion return value before evaluating pending count

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-10-16 18:45:32 +03:00
Mykola Golub
8af1e38365
Merge pull request #37643 from dillaman/wip-47840
librbd: ignore -ENOENT error when disabling object-map

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-10-16 18:45:04 +03:00
Mykola Golub
f90daf7495
Merge pull request #37444 from dillaman/wip-librbd-copyup-api
librbd: support the ability to process parent data prior to copyup

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Or Ozeri <oro@il.ibm.com>
2020-10-16 18:44:36 +03:00
Jason Dillaman
458ab997fe journal: possible race condition between flush and append callback
When notifying the journal recorder of an overflow or if the object
close request has completed due to no more in-flight IO, it was
possible for a race between a flush request and the processing of
an append completion to attempt to kick off duplicate notifications.
Since the overflowed and closed callbacks are properly protected from
duplicates, use a counter instead of a boolean to track possible
in-flight handler callbacks.

Fixes: https://tracker.ceph.com/issues/47880
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-10-16 11:29:54 -04:00
Kefu Chai
1c422d2f2a crimson/common: schedule action only if the future is not available
otherwise we could call do_until() recursively if we have other tasks
which need to prempt the reactor and current future's state is actually
always available.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-16 22:08:03 +08:00
Kefu Chai
710128607b mgr/dashboard: do not use "l" for variable name
see also https://www.flake8rules.com/rules/E741.html

also silences flake8 warnings like:

2: {tty:'./controllers/home.py:90:26: E741 ambiguous variable name 'l'':'./controllers/home.py:90:26: E741 ambiguous variable name 'l''}
2: {tty:'./controllers/logs.py:52:13: E741 ambiguous variable name 'l'':'./controllers/logs.py:52:13: E741 ambiguous variable name 'l''}
2: {tty:'./services/ganesha.py:36:40: E741 ambiguous variable name 'l'':'./services/ganesha.py:36:40: E741 ambiguous variable name 'l''}
2: 3     E741 ambiguous variable name 'l'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-16 18:56:09 +08:00
Kefu Chai
f5adeeb621
Merge pull request #37687 from tchaikov/wip-crimson-errorator
crimson/common: do not take from a future twice

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-10-16 16:50:10 +08:00
Kiefer Chang
d9e62340b1
Merge pull request #37602 from tspmelo/wip-flake8-ignore
mgr/dashboard: Remove some Flake8 ignore rules

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-10-16 15:27:35 +08:00
Kefu Chai
5d80c545a9 crimson/common: do not take from a future twice
before this change, in our specialization of seastar::do_until(),
we access `f` after calling `f.get()`, this is not correct. as `f.get()`
actually moves `f._state` away and detaches the associated promise if any.
so we cannot call `f._then()` anymore after calling `f.get()`. as
`f._then()` schedules `f` by detaching the future from promise and
attaching the scheduled task to the promise. but `future_base::detach_promise()`
does not check `_promise` before accessing it, hence the segfault.

after this change, the order of the checks is rearranged so that
`f.get()` is called at the end. and also use `f.get0()` to be more
explicit, as we are accessing the only element of the returned
value.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-16 14:36:58 +08:00
J. Eric Ivancich
ff342f2f02
Merge pull request #37684 from ivancich/rgw-yummy-orphan-testing
rgw: use yum rather than dnf for teuthology testing of rgw-orphan-list

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-15 16:14:07 -04:00
J. Eric Ivancich
11a08a5bb8 rgw: use yum rather than dnf for testing rgw-orphan-list
The teuthology testing for rgw-orphan-list needs to install
`s3cmd`. Switch from using dnf to yum to work on a wider variety of
platforms.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-10-15 15:08:45 -04:00
Adam C. Emerson
0deb4246dc rgw: Be less noisy when probing FIFO existence
Since when starting up the datalog we check whether Omap and FIFO
backed versions already exist, don't fill the error log with
complaints if they happen not to.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-15 13:57:10 -04:00