Commit Graph

127166 Commits

Author SHA1 Message Date
Ilya Dryomov
0dcea098cf librbd: honor FUA op flag for write_same() in write-around cache
WriteAroundObjectDispatch::write_same() should pass op_flags through
to dispatch_io() so that it can bypass the cache if needed.

Fixes: https://tracker.ceph.com/issues/52956
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-15 18:24:58 +02:00
Sebastian Wagner
c19250ce3c
Merge pull request #42997 from josephsawaya/wip-mgr-rook-orch-ls
mgr/rook: orch ls displays info about OSDs

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2021-10-14 17:33:26 +02:00
Sebastian Wagner
4e28b163a2
Merge pull request #43332 from sebastian-philipp/sphinx-rgw-spec
doc/cephadm: use sphinx autoclass to document RGWSpec

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-14 16:54:18 +02:00
Kefu Chai
70b049ffdb
Merge pull request #43239 from trociny/wip-48959
osd: handle inconsistent hash info during backfill and deep scrub gracefully

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-14 22:43:16 +08:00
Kefu Chai
7aefc0ff77
Merge pull request #43498 from dheart-joe/bluestore_fix
os/bluestore: _do_write_small fix head_pad

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-10-14 22:41:52 +08:00
Kefu Chai
44e89c613c
Merge pull request #42099 from aclamk/wip-bluefs-fine-grain-locking-2
os/bluestore: BlueFS fine grain locking

Reviewed-by: Sage Weil <sage@newdream.net>
2021-10-14 22:40:20 +08:00
Kefu Chai
552ac66577
Merge pull request #43374 from ljflores/wip-remove-pool-name
mgr/telemetry: remove pool name from telemetry perf report

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2021-10-14 22:39:14 +08:00
Kefu Chai
b5d2548ceb
Merge pull request #43463 from Zhiwei-Dai/wip-enhance-qa-python3-compatibility
qa/tasks: replace iterkeys() with keys() for Python 3

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:38:39 +08:00
Kefu Chai
94a9757ff8
Merge pull request #43405 from ljflores/wip-perfcounter-priorities
os/bluestore: update perf counter priorities

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-14 22:37:36 +08:00
Kefu Chai
cb40e8fc65
Merge pull request #43422 from ronen-fr/wip-rf-exact-time-sec
common/ceph_time: fix wrong seconds output in exact_timespan_str()

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:36:30 +08:00
Kefu Chai
84adba7588
Merge pull request #43515 from fengchunsong/dpdk-dev
msgs/async/dpdk: fix dpdk build error

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:34:56 +08:00
Kefu Chai
9fb3ded7b5
Merge pull request #43531 from tchaikov/wip-pick-addr
common/pick_address: refactor pick_addresses()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-14 22:33:39 +08:00
Sebastian Wagner
a3fc496553
Merge pull request #43073 from strenuous-life/wip-cephadm-osd-rm-status
mgr/cephadm: the display of 'orch osd rm status' is incorrect

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:29:05 +02:00
Sebastian Wagner
d4783f5a65
Merge pull request #43214 from batrick/i52654
pybind/mgr/cephadm: set allow_standby_replay during CephFS upgrade

Reviewed-by: Sage Weil <sage@newdream.net>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:28:30 +02:00
Sebastian Wagner
64c6fa2d84
Merge pull request #43253 from guits/fix_orch_ls_osd
cephadm: fix osd size reported in 'orch ls'

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:28:03 +02:00
Sebastian Wagner
9e68e27ce6
Merge pull request #43339 from adk3798/volume-chunk
mgr/cephadm: use json-pretty for ceph-volume calls

Reviewed-by: Sage Weil <sage@newdream.net>
2021-10-14 16:27:10 +02:00
Patrick Donnelly
6e589e5080
Merge PR #43517 into master
* refs/pull/43517/head:
	doc / cephfs: health message codes should be permalinks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-14 10:08:18 -04:00
Kefu Chai
a948322275
Merge pull request #43538 from rzarzynski/wip-crimson-net-actual-bind-failure
crimson/osd: print the actual reason on bind failure.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:00:27 +08:00
Ernesto Puerta
90bbcab09f
Merge pull request #42557 from ceph/feature-50336-cluster-creation-wizard
mgr/dashboard: Cluster Creation/Expansion Wizard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Volker Theile <vtheile@suse.com>
2021-10-14 15:12:42 +02:00
Ernesto Puerta
9f2fd685fa
Merge pull request #43453 from rhcs-dashboard/e2e-host-fix-teuth
mgr/dashboard: Fix orchestrator/01-hosts.e2e-spec.ts failure

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-14 13:26:44 +02:00
Chunsong Feng
f696c365a2 src/msg/dpdk: reserve funcs capacity to avoid reallocation
When a new vector is added larger than then current vector capacity,
it reallocates space. lamda function accesses the previous adress will
cause a segment fault. Therefore, reserve sufficient funcs space to
avoid reallocation.

Fixes: https://tracker.ceph.com/issues/52907
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2021-10-14 09:55:05 +00:00
Chunsong Feng
86ec66a9b5 msgs/async/dpdk: fix dpdk build error
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2021-10-14 09:55:00 +00:00
Radoslaw Zarzynski
bba537988b crimson/osd: print the actual reason on bind failure.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-14 09:53:48 +00:00
Yuval Lifshitz
d009895a6b
Merge pull request #43308 from TRYTOBE8TME/wip-rgw-post-object-upload
src/rgw: HTTP POST object upload
2021-10-14 10:41:15 +03:00
Kefu Chai
45d9eff5e4 common/pick_address: refactor pick_addresses()
* consolidate the logic handling CEPH_PICK_ADDRESS_PREFER_IPV4 using
  std::sort(). this might be overkill. but it helps to explain
  what CEPH_PICK_ADDRESS_PREFER_IPV4 is for, and helps to dedup
  the code to order the addresses.
* let fill_in_one_address() return an optional<entity_addrvec_t>.
  more readable this way
* early return if the required address is not found, instead of
  checking variables like ipv4_r
* rename fill_in_one_address() to get_one_address() to reflect
  the change of the function's return value's type

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 05:38:27 +00:00
Venky Shankar
3d97d6d98f doc / cephfs: health message codes should be permalinks
... so that such links can be included in alert warnings.

Additionally, document some other health warnings. Credit to @pcuzner
to point out that not all health warnings have been documented.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-10-14 10:21:07 +05:30
Kefu Chai
8661ba1807 common/pick_address: use string_view when appropriate
we don't need a string when string_view is good enough.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 01:23:00 +00:00
adk3798
c7419c8c43
Merge pull request #43526 from adk3798/disable-agent-default
mgr/cephadm: disable agent by default

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-13 17:45:42 -04:00
Sage Weil
e94ba142f0 Merge PR #43049 into master
* refs/pull/43049/head:
	mgr/rook: apply mds using placement spec and osd_pool_default_size
	mgr/rook: factor out replica/failureDomain calc

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-10-13 17:08:20 -04:00
Adam King
7120d5dc6f mgr/cephadm: disable agent by default
to avoid issues while it is stabilized

Signed-off-by: Adam King <adking@redhat.com>
2021-10-13 12:33:47 -04:00
Sebastian Wagner
dc855b4431
Merge pull request #43490 from Daniel-Pivonka/cephadm-doc-adopt-sshuser
doc/cephadm: document setting ssh-user during adopt

Reviewed-by: Adam King <adking@redhat.com>
2021-10-13 18:11:36 +02:00
Daniel Pivonka
bafbeb6fca doc/cephadm: document setting ssh-user during adopt
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
2021-10-13 11:38:01 -04:00
Laura Flores
8790f04c54 os/bluestore: update priorities and nicks of bluestore perf counters
These perf counters do not show up in telemetry unless they are set to a "useful" priority or higher. Fetching these counters in telemetry may help to diagnose problems with RocksDB / BlueFS prefetching / insufficient cache sizes.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-10-13 14:18:48 +00:00
Ernesto Puerta
9b40c9df26
Merge pull request #43377 from rhcs-dashboard/fix-clients-connection-query
mgr/dashboard: replace "Ceph-cluster" Client connections with active-standby MGRs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-13 13:37:51 +02:00
Nizamudeen A
3276147049 mgr/dashboard: Cluster expansion e2e cleanups
Fixes: https://tracker.ceph.com/issues/52645
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-13 16:02:51 +05:30
Avan Thakkar
6644a00a2c mgr/dashboard: introduce gather facts in host list
Fixes: https://tracker.ceph.com/issues/52017
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2021-10-13 16:02:51 +05:30
Aashish Sharma
b914f59ff1 mgr/dashboard: Cluster Creation Add Services Section
Add Services section in cluster creation wizard
Create Cluster OSD Section Followups

1. The device preview disappearing when going to next step and coming back to the previous step
2. Even when clearing the device preview, the Storage Capacity count and the drive group spec doesn't get cleared.
3. Expanding the cluster without selecting any devices gives a 400
   error.
4. Renamed "Delete Host" to "Remove Host"
5. Generalizing most of the sub component code

Fixes: https://tracker.ceph.com/issues/52499
Fixes: https://tracker.ceph.com/issues/51991
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-13 16:01:21 +05:30
Nizamudeen A
ad1535eea4 mgr/dashboard: Cluster Creation Create OSDs Section
Create OSDs section in cluster creation wizard

Fixes: https://tracker.ceph.com/issues/51991
Fixes: https://tracker.ceph.com/issues/52298
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-13 15:56:09 +05:30
Avan Thakkar
2e31ba5427 mgr/dashboard: Review Section for the Create Cluster Workflow
Fixes: https://tracker.ceph.com/issues/50566
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2021-10-13 15:56:09 +05:30
Nizamudeen A
59cbf97e6c mgr/dashboard: Cluster Creation Add Host Section and e2es
Add host section of the cluster creation workflow.

1. Fix bug in the modal where going forward one step on the wizard and coming back opens up the add host modal.
2. Rename Create Cluster to Expand Cluster as per the discussions
3. A skip confirmation modal to warn the user when he tries to skip the
   cluster creation
4. Adapted all the tests
5. Did some UI improvements like fixing and aligning the styles,
   colors..
- Used routed modal for host Additon form
- Renamed the Create to Add in Host Form

Fixes: https://tracker.ceph.com/issues/51517
Fixes: https://tracker.ceph.com/issues/51640
Fixes: https://tracker.ceph.com/issues/50336
Fixes: https://tracker.ceph.com/issues/50565
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-13 15:55:23 +05:30
Avan Thakkar
b9f38cadc4 mgr/dashboard: Create Cluster Workflow welcome screen and e2e tests
A module option called CLUSTER_STATUS has two option. INSTALLED
AND POST_INSTALLED. When CLUSTER_STATUS is INSTALLED it will allow to show the
create-cluster-wizard after login the initial time.  After the cluster
creation is succesfull this option is set to POST_INSTALLED
Also has the e2e codes for the Review Section

Fixes: https://tracker.ceph.com/issues/50336
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-13 15:52:14 +05:30
Kalpesh Pandya
ffe7de8853 src/test: HTTP POST object upload
This solves the tracker: https://tracker.ceph.com/issues/51724
Basically it is using 'generate_presigned_post()' boto3 API.
This is verified under AMQP endpoint.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-10-13 11:09:51 +05:30
Samuel Just
7d74578ab0
Merge pull request #43500 from rzarzynski/wip-crimson-interruptible-condalias
crimson: osd operations respect interruptor's InterruptCondition.

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-12 14:39:38 -07:00
Samuel Just
5a99bb8fc7
Merge pull request #43501 from rzarzynski/wip-crimson-parse-env
crimson: respect the CEPH_ARGS environment variable.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-12 14:38:47 -07:00
Sebastian Wagner
53382d70eb
Merge pull request #43274 from pcuzner/add-mib
monitoring:Adding the Ceph MIB

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-12 22:29:06 +02:00
Gregory Farnum
4dd8ed2578
Merge pull request #43509 from ceph/revert-43462-wip-test-erasure-code-for-aarch64
Revert "qa: support isal ec test for aarch64"

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2021-10-12 12:00:56 -07:00
Zack Cerza
b57539dc94 Revert "qa: support isal ec test for aarch64"
This commit has been causing scheduled jobs to request e.g. aarch64
smithi machines, which don't exist. The dispatcher then tries to find them forever, requiring the dispatcher to be killed and restarted. The queue
will sit idle until someone notices the problem.

Signed-off-by: Zack Cerza <zack@redhat.com>
2021-10-12 12:53:58 -06:00
Samuel Just
79a5deda62
Merge pull request #43492 from xxhdx1985126/wip-52894
crimson/os/seastore: set ExtentPlacementManager::allocated_to before rolling segments

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-12 08:09:58 -07:00
Nizamudeen A
db5cfb15e5 mgr/dashboard: Fix orchestrator/01-hosts.e2e-spec.ts failure
The test is failing on deleting a host because the agent daemon is
present in that host. Its not possible to simply delete a host. We need
to drain it first and then delete it.

Fixes: https://tracker.ceph.com/issues/52764
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-12 20:30:12 +05:30
Radoslaw Zarzynski
7b75040d25 crimson: respect the CEPH_ARGS environment variable.
Rook uses `CEPH_ARGS` to convey the `mon_host` configurable. Lack of
support for it was the root cause of inability to reach out to monitors.

See: https://gist.github.com/rzarzynski/95746aa73a48e811749a3b0aaeb31680#gistcomment-3924233.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-12 13:47:38 +00:00