Commit Graph

110766 Commits

Author SHA1 Message Date
Kefu Chai
9486206f5a
Merge pull request #34429 from xxhdx1985126/wip-crimson-recovery-pr
crimson: pglog based recovery ---- part 1

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-28 17:22:06 +08:00
Lenz Grimmer
56a2a9ab9f
Merge pull request #34656 from rhcs-dashboard/44935-custom-cherryPy-header
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen <nia@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-28 10:52:53 +02:00
Yuri Weinstein
c272749ed4
Merge pull request #34748 from neha-ojha/wip-jewel-cleanup
qa/suites/rados: jewel cleanup

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-27 17:05:59 -07:00
Yuri Weinstein
975cb62796
Merge pull request #34602 from neha-ojha/wip-45075
qa/standalone/scrub/osd-scrub-repair.sh: fix race in TEST_auto_repair_bluestore_failed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-27 17:04:45 -07:00
Gregory Farnum
0a9612bcde
Merge pull request #34595 from rishabh-d-dave/fs-move-run_shell
qa/cephfs: move run_shell to mount.py and add methods for negative testing

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-04-27 16:35:20 -07:00
Laura Paduano
63d690fcc5
Merge pull request #34705 from bk201/wip-45149
mgr/dashboard: fixing RBD purge error in backend

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-04-27 10:09:55 +02:00
Kefu Chai
d8084806fb
Merge pull request #34752 from tchaikov/fix-test-perf-counters-mgr-get
mgr/dashboard: fix API tests to be py3 compatible

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2020-04-27 12:00:40 +08:00
Kefu Chai
d9ca5e5d7b
Merge pull request #34292 from cbodley/wip-bufferlist-comparison
bufferlist: use iterators for comparison operators

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-27 11:38:16 +08:00
Kefu Chai
f0a4fc14f0
Merge pull request #34090 from trociny/wip-ceph-erasure-code-tool
tools/erasure-code: new tool to encode/decode files

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-27 11:37:00 +08:00
Kefu Chai
fcf8bd7c4d
Merge pull request #34683 from ifed01/wip-ifed-fix-freelist-label
os/bluestore: fix freelist's meta usage when bdev label is unsupported

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
2020-04-27 11:35:37 +08:00
Kefu Chai
a8733598ed
Merge pull request #34588 from ifed01/wip-ifed-lower-spinner-mas
os/bluestore:  set bluestore_min_alloc_size to 4K for spinners

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-04-27 11:34:14 +08:00
Kefu Chai
b76f4d0687
Merge pull request #34006 from aclamk/wip-rocksdb-sharding
kv/RocksDBStore,os/blustore: shard rocksdb database using column families

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-27 11:31:53 +08:00
Kefu Chai
6ad027c63e
Merge pull request #33790 from bstillwell/add_conf_option_zstd_compression_level
compressor: Add a config option to specify Zstd compression level

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-27 11:28:05 +08:00
Kefu Chai
8ae1c578b5
Merge pull request #34635 from tchaikov/wip-mgr-py3
mgr: drop python2 support

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-04-27 11:27:19 +08:00
Xuehan Xu
6439714927 crimson: prevent PG::schedule_event_after from creating daggling future
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Xuehan Xu
ff08cb6b36 crimson/osdmap_gate: sort the blockers in ascending order of map epoch
When the osdmap_gate got a new map, it's supposed to release blocked ops on
all previous maps section of which is represented by [waiting_peering.begin(),
waiting_peering.upper_bound(epoch)]. So it's essentail that waiting_peering is
ordered in ascending order of the map epoch

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Xuehan Xu
8ff09f2da9 crimson: keep track of clean_regions/dirty_regions of modifications
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Xuehan Xu
b39d6e03b3 crimson: add osd interface for recovery messages
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Xuehan Xu
7b0462fce7 crimson: add recovery operations oriented request ---- "RecoverySubRequest"
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Xuehan Xu
566cd75748 crimson: add pg log based recovery machinery
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Xuehan Xu
7f26f70114 crimson/os: add readv method to futurized_store
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Samuel Just
60dd721fe3 crimson: add background recovery request and necessary interfaces for recover/backfill
Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Samuel Just
3ae73f0fb9 osd: add some clean ups
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-04-26 15:46:35 +08:00
Samuel Just
12a19749e3 crimson/osd/osd_operation: add support for AggregateBlocker
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-04-26 15:46:35 +08:00
Samuel Just
12d57c9cc0 crimson/osd: introduce scheduler implementations and operation throttler
Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-04-26 15:46:35 +08:00
Samuel Just
3fe103b3e8 crimson/osd: wire up local and remote async reservers
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-04-26 15:46:35 +08:00
Samuel Just
ce659e3fd7 common/AsyncReserver: template out the finisher type
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-04-26 15:46:35 +08:00
Samuel Just
0becb44ef4 osd/PeeringState: eliminate unused cct from PGPool
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-04-26 15:46:34 +08:00
Samuel Just
6723922b4e osd/PrimaryLogPG: move earliest_backfill to PeeringState
We'll want this in crimson as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-04-26 15:46:34 +08:00
Kefu Chai
28c7ba68ae
Merge pull request #34732 from lxbsz/fs_fix_new
qa: misc fixes to run for cephfs qa test

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-26 01:46:30 +08:00
Kefu Chai
dea86fd3a0
Merge pull request #31912 from zjcmszh/cmpext
pybind/rados: add Ioctx::cmpext()

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-04-25 22:29:57 +08:00
zhangjiao
5838730828 test/pybind/test_rados.py: test Ioctx::cmpext()
Signed-off-by: Zhang Jiao <zhangjiao@cmss.chinamobile.com>
2020-04-25 18:53:50 +05:30
zhangjiao
1787a3bd6d pybind/rados: add Ioctx:cmpext()
Signed-off-by: Zhang Jiao <zhangjiao@cmss.chinamobile.com>
2020-04-25 18:52:52 +05:30
Kefu Chai
d07960ef7a
Merge pull request #34578 from lxbsz/volume_client_py3
qa/volume_client: be python3 compatible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-25 18:58:43 +08:00
Xiubo Li
dd4fc808a8 qa/tasks/cephfs: switch to StringIO to fix byte-like errors
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:55 -04:00
Xiubo Li
8921c8e1dc qa/cephfs: switch to mount_wait() to avoid possible umounted assert
For the fuse mount, it sometime will need to wait to a moment to let
the client get ready.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:55 -04:00
Xiubo Li
211c3fbb4b qa/tasks/cephfs/fuse_mount: fix possible chmod 1777 error
INFO:teuthology.orchestra.run.smithi13a2:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2)
INFO:teuthology.orchestra.run.smithi132.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.2': Permission denied
DEBUG:teuthology.orchestra.run:got remote process result: 1

Here just wait and rety for 10 times.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:55 -04:00
Xiubo Li
bf10478294 qa/tasks/vstart_runner: remove the stdin str type check
For the "ceph-dencoder" it will use the byte type data.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:54 -04:00
Xiubo Li
7f57576123 qa/cephfs: fix AssertionError: CommandFailedError not raised under py3
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:54 -04:00
Mykola Golub
b0133c12ce
Merge pull request #34697 from dillaman/wip-44596
rbd-mirror: skip snapshot image-sync for newly created/cloned images

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-04-25 10:29:01 +03:00
Kefu Chai
16efdd62b5
Merge pull request #34751 from tchaikov/wip-doc-fix
doc/releases/mimic.rst: add title underline

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2020-04-25 10:58:40 +08:00
Kefu Chai
b152e498a0
Merge pull request #34741 from tchaikov/wip-teuthology-cronjobs-mimic
qa/tests: pass "-t 2" to teuthology-suite when testing mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2020-04-25 10:23:08 +08:00
Alfonso Martínez
0ac296430f mgr/dashboard: fix API tests
* test_perf_counters_mgr_get
* test_selftest_cluster_log

Fixes: https://tracker.ceph.com/issues/45170
Fixes: https://tracker.ceph.com/issues/45246
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-04-25 09:50:19 +08:00
Kefu Chai
2de78c3bd9 qa/tasks/vstart_runner.py: use StringIO for capturing text
to be consistent with 8bfe977854

Fixes: https://tracker.ceph.com/issues/45246
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-25 09:50:14 +08:00
Kefu Chai
2994fba70b doc/releases/general.rst: add mising link targets
they are referenced by 60c8254869

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-25 09:38:33 +08:00
Kefu Chai
598b39e949 doc/releases: silence sphinx warnings
* mimic.rst: add title underline

to address sphinx warning like:
```
WARNING: Title underline too short.
```

* octopus.rst: use consistent underline

to address following sphinx warning:
```
doc/releases/octopus.rst:49: WARNING: Title level inconsistent:
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-25 09:33:43 +08:00
Neha Ojha
c258ff2c15 qa/suites/rados/thrash-old-clients/1-install: remove hammer
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:45:19 +00:00
Neha Ojha
4a6361d93e qa/suites/rados/thrash/workloads: use default mon osd initial require min compat client
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:29:06 +00:00
Neha Ojha
5ac2f54701 qa/suites/rados/thrash-old-clients/1-install: remove jewel
Since we've changed mon_osd_initial_require_min_compat_client to
luminous in 986c271b75, we can remove jewel from
mix.

Fixes: https://tracker.ceph.com/issues/45242
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:29:06 +00:00
Sebastian Wagner
cc5b263941
Merge pull request #34698 from matthewoliver/cephadm_check_pool_exists
cephadm: Check for pool existence for iscsi And NFS

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-24 18:39:10 +02:00