Commit Graph

110772 Commits

Author SHA1 Message Date
Igor Fedotov
2f86e19d7f os/bluestore: set bluestore_min_alloc_size to 4K for spinners.
Adjusting dependant params as well.
Reolving space amplification caused by small objects and/or EC
overwrites.

Relates to: https://tracker.ceph.com/issues/44213

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2020-04-16 14:23:44 +03:00
Kiefer Chang
223c8d816f
mgr/test_orchestrator: fix error when listing services
- Adapting attribute change of ServiceSpec.
- Update dummy data.

Fixes: https://tracker.ceph.com/issues/45108
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-04-16 17:55:40 +08:00
Kefu Chai
08bec918f7 qa/tasks/mgr/dashboard: set last_opt_revision when setting enabled
telemetry requires user to re opt-in when opt_revision is bumped up

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-16 16:14:47 +08:00
Kiefer Chang
45fd5ee275
mgr/orch: Fixes some deserialization errors
Fixes errors when calling `from_json` of these classes:
- InventoryHost: parsing labels
- ServiceDescription: `last_refresh` and `created` fields should be parsed
  to datetime type.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-04-16 15:30:24 +08:00
Tim Serong
bed711d3d1 cephadm: bail out early during adopt if data dir not present
In case someone tries to run this again on an already adopted daemon...

Signed-off-by: Tim Serong <tserong@suse.com>
2020-04-16 15:42:12 +10:00
Tim Serong
225b255129 cephadm: add --force-start option to adopt
Current behaviour is to only start a newly adopted ceph daemon if it was
already running before the adopt.  Adding a --force-start option allows
the adopt command to start newly adopted daemons that weren't originally
running, to save the user having to manually invoke `systemctl start
ceph-$FSID@$DAEMMON.$ID`.

Signed-off-by: Tim Serong <tserong@suse.com>
2020-04-16 15:32:06 +10:00
Tim Serong
6834ad9837 cephadm: don't throw when disabling ceph-volume simple unit
When adopting OSDs, if a ceph-volume simple service is already disabled
(or otherwise missing) the previous implementation would raise an error,
thus killing the adopt.

Signed-off-by: Tim Serong <tserong@suse.com>
2020-04-16 15:09:13 +10:00
Tim Serong
9169119bd3 cephadm: handle adopting offline OSDSs
The current adopt behavior expects OSDs to be online, in order to read
/var/lib/ceph/osd/ceph-$ID/fsid.  To handle the case where OSDs
are offline, this change first checks to see if that file is present,
and if not, falls back to calling `ceph-volume lvm list` to see if
there's a matching OSD there, and if that doesn't work, it checks
/etc/ceph/osd/*.json to see if there's a matching old-style simple
OSD present.

For LVM OSDs, the only thing we need is the ODS's fsid; the remainer
of the adopt procedure "just works", as the various other files
in /var/lib/ceph/$FSID/osd.$ID are created by magic anyway when the
OSD is activated, so it doesn't matter if they're not present at
adoption time.

For simple (ceph-disk created) OSDs, we actually need all the files under
/var/lib/ceph/osd/ceph-$ID/ to be moved to /var/lib/ceph/$FSID/osd.$ID
so if a simple OSD is found, it's mounted first, so the existing
move_files() a bit further down around line 3200 continues to work.

Fixes: https://tracker.ceph.com/issues/45095
Signed-off-by: Tim Serong <tserong@suse.com>
2020-04-16 14:55:58 +10:00
Josh Durgin
fb3c6bd0cd doc: add object corpus to release checklist
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-04-16 00:53:17 -04:00
Josh Durgin
09d928ff90 ceph-object-corpus: pick up octopus 15.2.1-40-ga838bb1aae
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-04-16 00:53:14 -04:00
Kefu Chai
d65516a020
Merge pull request #34577 from tchaikov/wip-doc-dev-corpus
doc/dev/corpus.rst: tweak formatting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-16 12:37:25 +08:00
Yan, Zheng
1d676851e3 client: reset requested_max_size if file write is not wanted
write can stuck at waiting for larger max_size in following sequence of
events:

- client opens a file and writes to position 'A' (larger than unit of
  max size increment)
- client closes the file handle and updates wanted caps (not wanting
  file write caps)
- client opens and truncates the file, writes to position 'A' again.

At the 1st event, client set inode's requested_max_size to 'A'. At the
2nd event, mds removes client's writable range, but client does not reset
requested_max_size. At the 3rd event, client does not request max size
because requested_max_size is already larger than 'A'.

Fixes: https://tracker.ceph.com/issues/44801
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-04-16 11:22:27 +08:00
Kefu Chai
a1f42101cf doc/dev/corpus.rst: tweak formatting
* use `shell` lexer, otherwise the Python one is used, and the rendered
  result does not look right
* be consistent when indenting -- either use tab or spaces, otherwise
  the indent in code block would be wrong.
* double quote the variables in text

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-16 11:12:25 +08:00
Matthew Oliver
8204b1d524 cephadm: add RGW SSL support
This patch adds SSL support to RGW when using cephadm.
If an SSL certificate is provided, inside the json supplied with:

  cpeh orchestrator rgw create -i rgw.json

Then the SSL cert and/or key will be added to pushed into the mon config-key database
using the key `rgw/cert/<rgw_realm>/<rgw_zone>.[crt|key]`.
Which will then be referenced in the config:

  rgw_frontends = beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/<rgw_realm>/<rgw_zone>.crt

And if an ssl key is also supplied this becomes something like:

  rgw_frontends = beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/<rgw_realm>/<rgw_zone>.crt ssl_key=config://rgw/cert/<rgw_realm>/<rgw_zone>.key

Of course you could also just upload the cert and key yourself to
config-key location, and ssl will be enabled as well. But this patch
let's you either supply them via `-i` or as a manual upload step.

Co-Authored-By: Michael Fritch <mfritch@suse.com>
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-04-16 02:31:02 +00:00
Josh Durgin
06e862bc68 include/denc: replace bufferlist::copy with iterator version
This version was only compiled as part of ceph-object-corpus
generation, when ENCODE_DUMP_PATH is defined, so it was missed
when bufferlist::copy() was removed.

Fixes: https://tracker.ceph.com/issues/45023
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-04-15 21:26:52 -04:00
Neha Ojha
61ad12e6ad
Merge pull request #34541 from neha-ojha/wip-balancer-on
mgr: turn on balancer in upmap mode by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-15 15:03:28 -07:00
Jason Dillaman
fb4311f597 qa/workunits/rbd: retry the addition of a mirror pool peer
We might race with the remote rbd-mirror daemon creating a
tx-only peer when adding a new peer. Therefore, delete the
tx-only peer and attempt to re-create it.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-04-15 16:27:07 -04:00
Jason Dillaman
ef68f686be librbd: mirror peer add error message should include site name
It previously included the pointer to string holding the generated
uuid (neither of which would mean much to an end user).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-04-15 16:26:09 -04:00
Jason Dillaman
74cded6b0d rbd: improved error message when adding a duplicate mirror pool peer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-04-15 16:25:48 -04:00
Lenz Grimmer
1576cc309c
Merge pull request #32747 from Exotelis/wip-40702-collapse
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-04-15 17:33:50 +02:00
Igor Fedotov
81e46e708a test/store_test: refator spillover test.
Fixes: https://tracker.ceph.com/issues/44880
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2020-04-15 18:17:26 +03:00
Casey Bodley
b62e0c262b
Merge pull request #34094 from cbodley/wip-rgw-data-sync-cache
rgw multisite: data sync optimizations

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2020-04-15 11:00:29 -04:00
Casey Bodley
463a1dd3c0 rgw: remove RGWBucketInfo::BIShardsHashType enum
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-15 10:09:52 -04:00
Casey Bodley
5e2514dc7f rgw: fix layout init in create_bucket
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-15 10:04:27 -04:00
Joshua Schmid
e670b1ea83 mgr/cephadm: add replacement flag to _create_osd
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-04-15 15:56:45 +02:00
Kotresh HR
138117f2f4 mgr/status: Fix "ceph fs status" json format writing to stderr
"ceph fs status" json format outputs to stderr instead of
stdout. This patch fixes the same.

Fixes: https://tracker.ceph.com/issues/44962
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2020-04-15 19:15:11 +05:30
Sebastian Wagner
62ed783dc2 mgr/orch: cleanup python imports
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-15 15:24:36 +02:00
Casey Bodley
8d7077ccad rgw: fix json decode of bucket layout
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-15 09:21:15 -04:00
Mykola Golub
256ac4b46d
Merge pull request #34549 from dillaman/wip-qa-rbd-read-balance
qa/suites/rbd: tweak 'require_osd_release' to 'pacific'

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-04-15 16:08:08 +03:00
Mykola Golub
34986728bd
Merge pull request #34496 from dillaman/wip-44105
rbd-mirror: clean-up unnecessary non-primary snapshots

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-04-15 15:30:48 +03:00
Sebastian Krah
d2d0efdc05
mgr/dashboard: Add expand/collapse datatable feature
Adds expand/collapse feature to every datatable with details.

Fixes: https://tracker.ceph.com/issues/40702
Signed-off-by: Sebastian Krah <skrah@suse.com>
2020-04-15 14:02:27 +02:00
Ramana Raja
2f45558eb8 mon/FSCommands: Fix 'fs new' command
After creating a filesystem using the 'fs new' command, the value
of the 'data' and 'metadata' key of the datapool and metadatapool's
application tag 'cephfs' should be the filesystem's name. This
didn't happen when the data or metadata pool's application metadata
'cephfs' was enabled before the pool was used in the 'fs new' command.
Fix this during the handling of the 'fs new' command by setting the
value of the key of the pool's application metadata 'cephfs' to the
filesystem's name even when the application metadata 'cephfs' is
already enabled or set.

Fixes: https://tracker.ceph.com/issues/43761
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-04-15 17:17:23 +05:30
Joshua Schmid
fa88483f66 mgr/cephadm: use --osd-ids instead of --osd-id
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-04-15 13:12:03 +02:00
Joshua Schmid
5b32c3e51b mgr/cephadm: add support for osd_id_claims
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-04-15 13:12:02 +02:00
Sebastian Wagner
a07f2d6ebe
Merge pull request #34529 from maran/master
cephadm: warn when no valid version arguments are supplied to add-repo

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-15 12:56:03 +02:00
Sebastian Wagner
91d0b5a318
Merge pull request #34490 from sebastian-philipp/cephadm-command_add_repo-no-podman
cephadm: add-repo: Don't test for podman

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-04-15 12:54:41 +02:00
Sebastian Wagner
0cc05d0f22
Merge pull request #34443 from sebastian-philipp/cephadm-revert-33974
cephadm: Update images used

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-04-15 12:54:17 +02:00
Ramana Raja
3c49092322 mon/FSCommands: Fix 'add_data_pool' command
After making a RADOS pool a filesystem's data pool using the
'add_data_pool' command, the value of the 'data' key of the pool's
application metadata 'cephfs' should be the filesystem's name. This
didn't happen when the pool's application metadata 'cephfs' was
enabled before the pool was made the data pool. Fix this during the
handling of the 'add_data_pool' command by setting the value of
the 'data' key of the pool's application metadata 'cephfs' to the
filesystem's name even when the application metadata 'cephfs' is
already enabled or set.

Fixes: https://tracker.ceph.com/issues/43061
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-04-15 15:56:56 +05:30
Sebastian Wagner
4313f863a0
Merge pull request #34556 from ahaase-c4u/patch-1
mgr/cephadm: corrected documentation for starting an upgrade

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-15 11:44:31 +02:00
Sebastian Wagner
dce6285cad
Merge pull request #34555 from ahaase-c4u/45065
mgr/cephadm: Fixed warn_on_stray_hosts setting

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-15 11:41:26 +02:00
Sebastian Wagner
1e1d5776f2
Merge pull request #34415 from sebastian-philipp/orch-fix-rgw-daemon-description-id
mgr/orch: Fix DaemonDescription().daemon_id() for RGWs

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-04-15 11:34:50 +02:00
Sebastian Wagner
8f605c7c86
Merge pull request #34216 from jschmid1/preview_drivegroups
mgr/cephadm: preview osd creation for osdspecs

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-15 11:33:21 +02:00
Sebastian Wagner
de095e8a5a
Merge pull request #34558 from mgfritch/cephadm-enter-container
cephadm: use CephContainer during command `enter`

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-04-15 11:29:36 +02:00
Joshua Schmid
462a68ab87 mgr/cephadm: enable osd previews
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-04-15 09:48:18 +02:00
ahaase-c4u
bf3d06d180 mgr/cephadm: corrected documentation for starting an upgrade
Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
2020-04-15 09:47:19 +02:00
Neha Ojha
0ec13bace9 doc/rados/operations: update docs to reflect balancer being on by default
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-15 03:32:42 +00:00
Kefu Chai
32c240fd7d
Merge pull request #34562 from tchaikov/wip-jenkins-trigger
github: add "jenkins test classic perf" to jenkins commands list

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-15 10:36:34 +08:00
Kefu Chai
eb586f51e9 github: add "jenkins test classic perf" to jenkins commands list
it's used for triggering perf test with classic OSD

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-15 09:57:57 +08:00
Neha Ojha
8f90e41d34 PendingReleaseNotes: add note about balancer being on by default
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-15 00:53:01 +00:00
Kefu Chai
bad52d108c mon/OSDMonitor: refactor loops for better readability
implement the suggestion proposed in #19076

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-15 08:07:31 +08:00