Commit Graph

124184 Commits

Author SHA1 Message Date
Patrick Donnelly
8cb34b3849
Merge PR #41771 into master
* refs/pull/41771/head:
	qa: update scrub start code to use comma sep scrubopts

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-15 10:32:06 -07:00
myoungwon oh
2dedb397ef seastore/rbm: remove changes related to transaction/cache
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-16 00:43:34 +09:00
Kefu Chai
e3c7234771
Merge pull request #41864 from rzarzynski/wip-crimson-iointrcond-fix-debug
crimson/osd: fix debug in IOInterruptCondition::new_interval_created().

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-15 23:34:01 +08:00
Kefu Chai
31380f1f3b
Merge pull request #41865 from rzarzynski/wip-crimson-opseq-fix-dropped-abort
crimson/osd: fix missed OpSequencer::abort() on primary change.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-15 23:05:11 +08:00
Radoslaw Zarzynski
2b14df40ef crimson/osd: fix missed OpSequencer::abort() on primary change.
Commit b5efdc6f1c has unified
the interruption handling among `InternalClientRequest` and
`ClientRequest`. Unfortunately, a call to `abort()` of
`OpSequencer` has been overlooked and dropped. This was causing
crashes at Sepia after after fixing the ordering issues in
`ClientRequest`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-15 14:32:39 +00:00
Radoslaw Zarzynski
f40109f64b crimson/osd: fix debug in IOInterruptCondition::new_interval_created().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-15 14:29:06 +00:00
Samuel Just
c323e59930
Merge pull request #41801 from liu-chunmei/seastore-fix-get-retired-extent
crimson/seastore: fix cache::get_extent got retired extent

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-15 07:01:07 -07:00
Sebastian Wagner
0667a70c60
Merge pull request #41854 from sebastian-philipp/fix-mypy-yaml-representation
mgr/orch: fix mypy errors

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2021-06-15 13:59:54 +02:00
Sebastian Wagner
6251e217a9
Merge pull request #41846 from batrick/i51214
doc: fix typo in option

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
2021-06-15 11:39:37 +02:00
Sebastian Wagner
73e7698958 pyhton-common: fix mypy errors
Fixes:

```
py3 run-test: commands[2] | mypy --config-file=../mypy.ini -p ceph
ceph/deployment/service_spec.py: note: In member "yaml_representer" of class "ServiceSpec":
ceph/deployment/service_spec.py:659: error: Argument 1 to "represent_dict" of "SafeRepresenter" has incompatible type "_OrderedDictItemsView[str, Any]"; expected "Mapping[Any, Any]"
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-15 11:24:34 +02:00
Sebastian Wagner
90c9980e8f mgr/orch: fix mypy errors
Fixes:

```
orchestrator/__init__.py:6: note: In module imported here:
orchestrator/_interface.py: note: In member "yaml_representer" of class "DaemonDescription":
orchestrator/_interface.py:1039: error: Argument 1 to "represent_dict" of "SafeRepresenter" has incompatible type "ItemsView[Any, Any]"; expected "Mapping[Any, Any]"
orchestrator/_interface.py: note: In member "yaml_representer" of class "ServiceDescription":
orchestrator/_interface.py:1178: error: Argument 1 to "represent_dict" of "SafeRepresenter" has incompatible type "ItemsView[Any, Any]"; expected "Mapping[Any, Any]"
orchestrator/_interface.py: note: At top level:
orchestrator/_interface.py:1181: error: Argument 2 to "add_representer" has incompatible type "Callable[[SafeDumper, DaemonDescription], Any]"; expected "Callable[[SafeDumper, ServiceDescription], Node]"
Found 3 errors in 1 file (checked 29 source files)
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-15 10:19:40 +02:00
Kefu Chai
61976503f7
Merge pull request #41841 from ljflores/patch-1
doc: update testing.rst and build-integration test to say ./run-make-check.sh

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-15 14:26:05 +08:00
Jinyong Ha
8fc7c4783a seastore: add protected_read interface to nvmedevice
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
f865954761 seastore/rbm: use const
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
f7ac64dbaa seastore/rbm: return std::list in get_data_health()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
4073a89a9e seastore/rbm: remove unused bits in nvmedevice.h
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
4c90949681 seastore/rbm: replace blk_paddr_t with size_t in free_extent()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
d2cfc48676 seastore/rbm: relocate file layout like SegmentManager
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
dbde4ff9be seastore/rbm: fix wrong loop condition
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
a8001931b2 seastore/rbm: fix wrong statement
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
65f577a772 seastore/rbm: remove newline before function name
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
6cad63650c seastore/rbm: add check_bitmap_blocks when open()
This commits adds check_bitmap_blocks to check free_block_count
is correct whenever open() is called

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
f72ad486ac seastore/rbm: fix wrong statement
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
c8323d364b seastore/rbm: use move()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
88578e978d seastore/nvmedevice: use the necessary capture value and fix typo
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
46e72c1877 seastore: move implementation of TestMemory to nvmeblockdevice.cc
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
736adb212d seastore: utilize errorator as a return type of discard()
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
916faf07f2 seastore: utilize unique_ptr as a return value of create()
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
ac8ac07863 seastore: change class name NormalNBD to PosixNVMeDevice
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
6c5c841686 seastore: replace POSIX call to seastar::file in NormalNBD
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
4e2939687a seastore: add abort_allocation
when aborting uncommitted transaction, abort_allocation
should be called to cancel in-flight allocations in
in-memory allocator.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
eccf92ba05 test: add more tests
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
f61b3f4606 test: modify existing unit test to cover range based operations
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
302afffbe0 seastore: fix wrong operator
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
06e5cd5fcd seastore: make multiple bitmap block initialization to work correctly
when initializing bitmap area, this commit corrects multiple
block initialization.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
0b35d5a864 seastore: continous range based free_extent()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
e0d2f89ad3 seastore: continous range based complete_allocation()
To work with a range of allocated blocks as a input,
this commit modify complete_allocation().

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
0f287e0834 seastore: introduce rbm_sync_block_bitmap_by_range()
To set continous range of block id at once, this commit adds
rbm_sync_block_bitmap_by_range().

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
ccd76f477f seastore: make alloc_extent to allocate continuous blocks
If there is no continous blocks, alloc_extent() returns nospc

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
835ddb7303 seastore: use deltas to deliver allocated info
alloc_extent() will allocates continous blocks (e.g., block 1 ~ 4).
To deliver such informations to journal efficiently,
this commit adds a member variable regarding only modified range in the deltas

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
70bced5b17 seastore: add NormalNBD
NormalNBD - NVMe Block device with seastar file
 1. Support preffered write granularity/alignment which is reported
 from NVMe SSD. User should follow this write guidance for enhancing write
 performance.
 2. Support multi-stream IO in NVMe SSD.

Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
Myoungwon Oh
0e04bcfa97 seastore: randomblockmanager inital commit
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
chunmei-liu
3fdd94c43f crimson/seastore: fix cache::get_extent got retired extent
one transaction got an extent whose state is MUTATION_PENDINGat that time.
but another transaction do split and set the extent state to RETIRED.
when the first transaction resume and do continuation, the state of the extent
has been changed to RETIRED. So need eagain to try again.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-06-14 15:29:18 -07:00
Patrick Donnelly
3511c83c6b
doc: fix typo in option
Fixes: https://tracker.ceph.com/issues/51214
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-14 13:37:29 -07:00
Sage Weil
ebb5a3f0bc qa/suites/orch/cephadm/dashboard: remove remaining bits
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-14 13:00:45 -05:00
Patrick Donnelly
a402b23c84
qa: update cli syntax to conventional
This was using an obscure syntax that worked at one time and wasn't
documented (AFAIK).

Fixes: https://tracker.ceph.com/issues/51182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-14 10:21:43 -07:00
Laura Flores
36de216d25
script/build-integration-branch: update print statement based on update to documentation
The print statement originally suggested to the user to run "make && ctest -j12", but that is outdated. The user should execute "./run-make-check.sh" instead.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-14 12:03:01 -05:00
Laura Flores
13695c1a92
doc/dev: should be ./run-make-check.sh
Updating the initial pull request. The smoke test line should read "./run-make-check.sh" instead of "make && ctest -j12".

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-14 11:23:54 -05:00
Laura Flores
5bbed1f6bd
doc/dev: update testing.rst to say ninja
The current version of Ceph uses "ninja" rather than "make," so the testing documentation needs to be updated.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-14 11:01:15 -05:00
Sage Weil
6824b29ca1 Merge PR #41799 into master
* refs/pull/41799/head:
	cephadm: validate `--fsid` during bootstrap

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-06-14 11:02:31 -04:00