Commit Graph

76703 Commits

Author SHA1 Message Date
Alfredo Deza
fc5c07d4d1 ceph-volume: systemd: create systemd entry point script
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
7b8cb78e05 ceph-volume: process: module for running system commands
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
0e3ea6f09c ceph-volume: main: initial take on main
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
4fce55b381 ceph-volume: terminal: easier terminal reporting utils
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
970e3978f3 ceph-volume: decorators: catch exceptions, disable on debug
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
f291b321c7 ceph-volume: log: be more robust, report back to config
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
3c576f91a7 ceph-volume: log: create a utility for setting up logging
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
1f9807b024 ceph-volume: config: default to 'info' verbosity, set the config dict
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
0a71da843c ceph-volume: rename: initial take on renaming to ceph-volume
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Dan Mick
de06f8e504 cmake/Distutils.cmake: make pkg files optional, add bin
Skip nonexistent 'expected' Python packaging files, and add bin/ as
a possible file for symlinking during build (for Python projects with
slightly different forms, currently ceph-volume)

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
7e0ee3d6b5 build: include the ceph-volume-systemd command in the spec file
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
171bee9ce1 build: add ceph-volume to CMakeLists in src
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
b20fdb8e84 systemd: include the ceph-volume service
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
49af65951e build: create a CMakeLists file for ceph-volume
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
d6954111dc systemd: create a service file for ceph-volume
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
31524efd1d build: add deb install rule for ceph-volume executable
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
7ca7b04521 build: add deb rules for ceph-volume systemd service file
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
004e907e6b build/ops: add ceph-volume scripts/files to spec file
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
59052b3a04 ceph-volume: create the executables
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
1fff4ffa9a ceph-volume: packaging: create a setup.py
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
dee11ceb72 ceph-volume: tests: create a tox.ini file
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
abd21c82fa ceph-volume: add a manifest.in file
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
Alfredo Deza
e25b93ffd4 ceph-volume: create a .gitignore specific to ceph-volume
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-04 10:25:57 -04:00
amitkuma
b386dfdf2a messages: Intilizating uninitized members MMonGetVersionReply
Fixes coverity Issue:

** 717297 Uninitialized scalar field
2. uninit_member: Non-static class member handle is not initialized in this constructor nor in any functions that it calls.
  	4. uninit_member: Non-static class member version is not initialized in this constructor nor in any functions that it calls.

CID 717297 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
6. uninit_member: Non-static class member oldest_version is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
2017-08-04 19:48:59 +05:30
amitkuma
cb13c4822c messages: Intilizating uninitized members MMonGetVersion
Fixes coverity Issue:

** 717296 Uninitialized scalar field
CID 717296 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member handle is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
2017-08-04 19:42:52 +05:30
amitkuma
c5828f9cd3 messages: Intilizating uninitized members MMDSTableRequest
Fixes coverity Issue:

** 717293 Uninitialized scalar field
2. uninit_member: Non-static class member table is not initialized in this constructor nor in any functions that it calls.
  	4. uninit_member: Non-static class member op is not initialized in this constructor nor in any functions that it calls.

CID 717293 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
6. uninit_member: Non-static class member reqid is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
2017-08-04 19:35:10 +05:30
Sage Weil
6dfae21eda Merge pull request #16805 from liewegas/wip-crush-classes
crush: misc changes/fixes for device classes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-04 08:48:53 -05:00
Sage Weil
c8af364699 Merge pull request #16739 from liewegas/wip-multi-backfill-reject
qa/suites/rados/singleton-nomsgr/all/multi-backfill-reject: sleep longer
2017-08-04 08:41:06 -05:00
Sage Weil
f681d6e794 Merge pull request #16794 from liewegas/wip-rbd-app
qa/tasks/ceph: enable rbd on rbd pool
2017-08-04 08:40:45 -05:00
Alfredo Deza
0d70d36e2f Merge pull request #16776 from dachary/wip-20893-ceph-fsid
ceph-disk: dmcrypt cluster must default to ceph

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-04 09:40:37 -04:00
Sage Weil
7ed9d1e8aa qa/tasks/ceph: enable rbd on rbd pool
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-04 09:40:26 -04:00
Sage Weil
b87cb025ba Merge pull request #16798 from jdurgin/wip-complete-to-loop
osd/PGLog: avoid infinite loop if missing version is corrupted

Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-04 08:38:31 -05:00
Sage Weil
1ae9ff173b qa/suites/rados/upgrade: ignore FS_DEGRADED from mds restart
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-04 09:34:31 -04:00
Sage Weil
27a685f626 qa/suites/rados/monthrash: ignore MGR_DOWN
Heavily thrashing mons + mgr reconnect backoff may make us fail
to process the beacon.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-04 09:34:15 -04:00
xie xingguo
244db7405e osd/OSD: tolerate any 'set-device-class' error on OSD startup
We might use 'ceph-disk prepare --crush-device-class' to specify a class
for an OSD and then manually change its class to quite another.

This will prevent OSD from booting on next startup:

mon_cmd_maybe_osd_create fail: 'osd.2 has already bound to class 'class2', can not reset class to 'class1'; use 'ceph osd crush rm-device-cl
ass <osd>' to remove old class first': (16) Device or resource busy
2017-08-04 15:52:41.997158 7f7c55aa4d00 -1 osd.2 231 init unable to update_crush_device_class: (16) Device or resource busy

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-04 17:36:58 +08:00
Pan Liu
9964cf177b Merge pull request #16804 from xiexingguo/wip-some-cleanup
os/bluestore: space between func and contents
2017-08-04 13:12:17 +08:00
Luo Kexue
e33ec7c1fa test/librbd/fsx: Add break in case OP_WRITESAME and OP_COMPARE_AND_WRITE
Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>
2017-08-04 11:34:33 +08:00
Josh Durgin
b9439b59b4 osd/PG: fix lost unfound + delete when there are no missing objects
The default constructed pg_missing_t for a peer with no missing
objects resets may_include_deletes to false, so when missing items for
lost deletes should be added later, they are not, and the old
version of the lost object is left on the replica.

Fix this by always setting may_include_deletes upon entering the
GetMissing state, and using clear() rather than the default
constructor to create an empty missing set.

This only affects lost_unfound delete since it is the only way for a
peer to gain missing delete entries. If the peer starts out missing
some objects, its missing set sent over the wire is used instead, with
the correct setting for may_include_deletes.

Fixes: http://tracker.ceph.com/issues/20904
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-08-03 23:02:43 -04:00
Sage Weil
3c6a6c46d5 os/bluestore: set deferred_aggressive if initial throttle get fails
This ensures that in-progress transactions with deferred writes queue their
IO immediately.  Otherwise, we may end up waiting indefinitely.

This is a biggish hammer.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:54:18 -04:00
Sage Weil
ca32d575eb os/bluestore: allow multiple DeferredBatches in flight at once
The current code only allows two DeferredBatches per osr: one that is
accumulating new writes and one that is currently in flight to disk.  If
the previous batch is in flight ot disk, then the currently accumulating
one can't also be queued for disk.

This can cause problems, notably that described in
http://tracker.ceph.com/issues/20295, where one transaction is trying to
grab deferred_throttle but cannot due to other in-progress txcs that
include deferred IO.  The short version is that it cannot queue all of the
IO needed, and that later when the IO does complete it is awkward to
determine whether other IO queued behind it also needs to be queued
immediately.  And since it's not, this leads to a deadlock/stall.

Simply allowing multiple batches of IO to be in flight at once is a simple
fix.  Specifically, in queue_transactions(), if throttle_deferred_bytes
get_or_fail() fails, we can now deferred_submit_all() and be sure that
other IO will be submitted and thus complete and release the throttle that
we need to continue.

It is possible that the deferred_aggressive behavior could be simplified
now that the old restriction is dropped, but that needs a closer review
of the code.

Fixes: http://tracker.ceph.com/issues/20295
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:54:18 -04:00
Nathan Cutler
d919987caa tests: rbd: reproducer for rbd-on-EC issue
This introduces a new "rbd/singleton-bluestore" suite because creating an rbd
on an EC-backed datapool will fail on filestore.

References: http://tracker.ceph.com/issues/20295
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-08-03 22:54:17 -04:00
Sage Weil
47480d8a06 qa/suites/rados/thrash-erasure-code-big: add k=4 m=2
Get better coverage for larger codes.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:50:16 -04:00
xie xingguo
af0d3661db os/bluestore: space between func and contents
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-04 10:47:25 +08:00
Sage Weil
c121501ad3 crush: remove cleanup_classes()
I can't for the life of me figure out what this is supposed to do.

- why remove (some) classes right after we populated them?
- why remove them after we decode the crush map?

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:39:05 -04:00
Sage Weil
226d210ab6 crush/CrushWrapper: rebuild_roots_with_classes on bucket removal
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:32:47 -04:00
Sage Weil
cb807b0006 crush: fix preservation of shadow bucket ids
1- a decompiled and recompiled was parsing the class bucket ids but it
wasn't actually using them.
2- rebuild_roots_with_classes() was throwing out the old ids and assigning
new ids when the tree was rebuilt.

Fix by passing in a (potentially partial) class_bucket map into
populate_classes().  Take care to allocate new bucket ids that don't
collide with previously used ids.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:32:47 -04:00
Dan Mick
89f85d22c1 common/buffer: off-by-one error in max iov length blocking
The loop in write_fd stops at IOV_MAX-1; it could go to IOV_MAX
and write in more-natural sizes without extra tail writes.

Fixes: http://tracker.ceph.com/issues/20907
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-03 18:25:00 -07:00
Patrick Donnelly
d89af4a3e8
Merge PR #16802 into master
* refs/remotes/upstream/pull/16802/head:
	qa: update wait_for_health for new health json syntax

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 16:20:20 -07:00
John Spray
0613d411aa qa: update wait_for_health for new health json syntax
Fixes: http://tracker.ceph.com/issues/20890
Signed-off-by: John Spray <john.spray@redhat.com>
2017-08-03 23:46:41 +01:00
Patrick Donnelly
051dd86898
Merge PR #16766 into master
* refs/remotes/upstream/pull/16766/head:
	cephfs: Clarify health warning
	cephfs: Always print filesystem name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 15:05:53 -07:00