Commit Graph

124216 Commits

Author SHA1 Message Date
Jinyong Ha
6c5c841686 seastore: replace POSIX call to seastar::file in NormalNBD
Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
4e2939687a seastore: add abort_allocation
when aborting uncommitted transaction, abort_allocation
should be called to cancel in-flight allocations in
in-memory allocator.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
eccf92ba05 test: add more tests
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
f61b3f4606 test: modify existing unit test to cover range based operations
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
302afffbe0 seastore: fix wrong operator
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
06e5cd5fcd seastore: make multiple bitmap block initialization to work correctly
when initializing bitmap area, this commit corrects multiple
block initialization.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
0b35d5a864 seastore: continous range based free_extent()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
e0d2f89ad3 seastore: continous range based complete_allocation()
To work with a range of allocated blocks as a input,
this commit modify complete_allocation().

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
0f287e0834 seastore: introduce rbm_sync_block_bitmap_by_range()
To set continous range of block id at once, this commit adds
rbm_sync_block_bitmap_by_range().

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
ccd76f477f seastore: make alloc_extent to allocate continuous blocks
If there is no continous blocks, alloc_extent() returns nospc

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
myoungwon oh
835ddb7303 seastore: use deltas to deliver allocated info
alloc_extent() will allocates continous blocks (e.g., block 1 ~ 4).
To deliver such informations to journal efficiently,
this commit adds a member variable regarding only modified range in the deltas

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
Jinyong Ha
70bced5b17 seastore: add NormalNBD
NormalNBD - NVMe Block device with seastar file
 1. Support preffered write granularity/alignment which is reported
 from NVMe SSD. User should follow this write guidance for enhancing write
 performance.
 2. Support multi-stream IO in NVMe SSD.

Signed-off-by: Jinyong Ha <jy200.ha@samsung.com>
2021-06-15 11:09:06 +09:00
Myoungwon Oh
0e04bcfa97 seastore: randomblockmanager inital commit
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-15 11:09:06 +09:00
chunmei-liu
3fdd94c43f crimson/seastore: fix cache::get_extent got retired extent
one transaction got an extent whose state is MUTATION_PENDINGat that time.
but another transaction do split and set the extent state to RETIRED.
when the first transaction resume and do continuation, the state of the extent
has been changed to RETIRED. So need eagain to try again.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-06-14 15:29:18 -07:00
Patrick Donnelly
3511c83c6b
doc: fix typo in option
Fixes: https://tracker.ceph.com/issues/51214
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-14 13:37:29 -07:00
Sage Weil
ebb5a3f0bc qa/suites/orch/cephadm/dashboard: remove remaining bits
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-14 13:00:45 -05:00
Patrick Donnelly
a402b23c84
qa: update cli syntax to conventional
This was using an obscure syntax that worked at one time and wasn't
documented (AFAIK).

Fixes: https://tracker.ceph.com/issues/51182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-14 10:21:43 -07:00
Laura Flores
36de216d25
script/build-integration-branch: update print statement based on update to documentation
The print statement originally suggested to the user to run "make && ctest -j12", but that is outdated. The user should execute "./run-make-check.sh" instead.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-14 12:03:01 -05:00
Laura Flores
13695c1a92
doc/dev: should be ./run-make-check.sh
Updating the initial pull request. The smoke test line should read "./run-make-check.sh" instead of "make && ctest -j12".

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-14 11:23:54 -05:00
Laura Flores
5bbed1f6bd
doc/dev: update testing.rst to say ninja
The current version of Ceph uses "ninja" rather than "make," so the testing documentation needs to be updated.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-14 11:01:15 -05:00
Sage Weil
6824b29ca1 Merge PR #41799 into master
* refs/pull/41799/head:
	cephadm: validate `--fsid` during bootstrap

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-06-14 11:02:31 -04:00
Sage Weil
3ded6803c2 Merge PR #41805 into master
* refs/pull/41805/head:
	cephadm: set TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-06-14 11:02:16 -04:00
Kefu Chai
75b91d49b8
Merge pull request #39624 from sebastian-philipp/mypy-812
src,qa: Upgrade to mypy 0.901

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-14 22:53:02 +08:00
Sebastian Wagner
caf793662e
Merge pull request #41632 from jmolmo/fix_osd_rm_help_msg
mgr/cephadm: fix errors in command <ceph orch osd> help and documentation

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-14 16:09:07 +02:00
Sebastian Wagner
07189c8aec
Merge pull request #41556 from adk3798/daemon-count
mgr/cephadm: fix --service-type flag in orch ls when service has id

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-14 16:08:46 +02:00
Sage Weil
9074e87611 Merge PR #41827 into master
* refs/pull/41827/head:
	qa: move dashboard e2e from cephadm -> rados suite

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-06-14 09:11:04 -04:00
Kefu Chai
5cf3f8491b
Merge pull request #41796 from rzarzynski/wip-crimson-cltreq-dbgs
crimson/osd: improve debugs in ClientRequest

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-14 21:00:04 +08:00
Radoslaw Zarzynski
c451bb8519 crimson/osd: improve debugs in OpSequencer.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-14 12:26:22 +00:00
Radoslaw Zarzynski
36b14288bd crimson/osd: should_abort_request() dumps Operation's details.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-14 12:26:22 +00:00
Radoslaw Zarzynski
6463cd9c12 crimson/osd: print prev_op_id in ClientRequest's debugs.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-14 12:26:22 +00:00
Kefu Chai
d7a70ed770
Merge pull request #41832 from inspur-wyq/patch-4
os/ObjectStore: fix typo in comments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-14 18:13:21 +08:00
Kalpesh Pandya
809571fe3d qa/tasks: Keycloak failure checking
This PR is solving an error: Directory not empty.
This was encountered during removal of the downloaded keycloak package from keycloak.py

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-06-14 13:27:18 +05:30
Venky Shankar
98a8980e86 cephfs-mirror: silence warnings when connecting via mon host
When connecting to remote cluster via mon address, the cluster
confuration file is not required. However, the mirror daemon
fills up the ceph context with the cluster name resulting in
the following warning in (systemctl) logs:

    auth: unable to find a keyring on ...

These warning messages need to be silence as they can mislead
the user.

Fixes: http://tracker.ceph.com/issues/51204
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-06-14 03:09:37 -04:00
Patrick Donnelly
b01591b790
Merge PR #41786 into master
* refs/pull/41786/head:
	mon/MonCommands: remove obsolete mds commands
	mon/MonCommands: remove obsolete mon commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:57:37 -07:00
Patrick Donnelly
31a771139d
Merge PR #41580 into master
* refs/pull/41580/head:
	cephfs: make ceph_fallocate() platform independent

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-13 19:57:05 -07:00
Patrick Donnelly
7e320919f7
Merge PR #41482 into master
* refs/pull/41482/head:
	qa: remove obsolete deactivate routines

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-06-13 19:56:34 -07:00
Patrick Donnelly
6a173d1a39
Merge PR #41481 into master
* refs/pull/41481/head:
	client: Add ceph.caps vxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-06-13 19:56:04 -07:00
Patrick Donnelly
dfce83ef12
Merge PR #41431 into master
* refs/pull/41431/head:
	mds: progress the recover queue immediately after the inode is enqueued
	mds: switch to mdr->get_client()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:54:45 -07:00
Patrick Donnelly
6a095654f4
Merge PR #41422 into master
* refs/pull/41422/head:
	qa/tasks/cephfs/test_sessionmap: reap connections immediately
	msg/async: configurable threshold for reaping dead connections

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:52:48 -07:00
Patrick Donnelly
0441b3d60f
Merge PR #41403 into master
* refs/pull/41403/head:
	mgr/volumes: Add config to insert delay at the beginning of the clone

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:51:52 -07:00
wangyunqing
7e0d79b9d1 os/ObjectStore: fix typo in comments
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2021-06-14 10:42:14 +08:00
Kefu Chai
9b6be59d6f test/pybind/test_ceph_argparse: add a test where args contains comma
to ensure the support for the new syntax of "prefix --name bon,jour"
does not break existing behavior of "prefix hello cruel,world" where value2
contains reads "hello,world", and the parsed result should be

prefix="prefix"
value=["hello", "cruel,world"]

instead of something like

prefix="prefix"
value=["hello", "cruel", "world"]

or

prefix="prefix"
value=["cruel", "world"]

the above test only applies to the test where "value" is a CephString.
if "value" is a CephChoices, the parsed argument should be

prefix="prefix"
value=["hello", "cruel", "world"]

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
49be76ce11 cmake: run nose test using "python3 -m nose"
otherwise the tests are not performed at all.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
cf4570d663 test/pybind/test_ceph_argparse: check for expected argument
instead of comparing the validated argument with empty dict, check for
its content for better coverage of testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
3d7516f8da test/pybind/test_ceph_argparse: decorate helper method with @nottest
otherwise nose tries to run it as a test as its name contains "test", like

E.
======================================================================
ERROR: Runs validate in different arg/kargs ways.
----------------------------------------------------------------------
TypeError: arg_kwarg_test() missing 3 required positional arguments: 'prefix', 'args', and 'sig'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
77cb1fed0c test/pybind: do not test obsoleted command
"scrub" command was marked obsoleted in
e9a5ce0897, so the test_ceph_argparse
cannot retrieve its command description using "get_command_descriptions"
cli tool anymore, let's drop the related test accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
b8389557ad test/pybind: drop forward and readforward from test
these two write modes were dropped in
a0a3ed324a, so update the test
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Kefu Chai
952285dcde test/pybind: do not use tab for indent
the python source file is idented using spaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Kefu Chai
0948a78ee6 pybind/ceph_argparse: validate csv if desc.N
if desc.N is not None, we should take the argument as a comma separated
values, and validate the values individually.

restructure the validate() function and its helpers to pass the
validated args if desc.N explicitly, as desc.instance.val should only
hold a single value of desc.instance type, otherwise we need to reset
it after collecting all the argument in a CSV string value is parsed.

this change would be less consistent and hackish.

and update a test to verify this behavior.

Fixes: https://tracker.ceph.com/issues/51145
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Kefu Chai
bc182888a3 test/pybind: pep8 cleanups
* do not import unused symbols
* add two empty lines before method
* aligned indents

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00