Commit Graph

140625 Commits

Author SHA1 Message Date
Adam King
90322fb617 cephadm: move logging from registry_login to command_registry_login
So that registry_login can be moved to container_engines.py
without creating a dependency on logging there

Signed-off-by: Adam King <adking@redhat.com>
2023-09-27 15:23:42 -04:00
Casey Bodley
de55616429
Merge pull request #53607 from cfsnyder/wip-cfsnyder-bucket-check-bug
rgw: fix radosgw-admin bucket check stat calculation bug

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-27 18:38:42 +01:00
Yuval Lifshitz
935c513a14
Merge pull request #52393 from yuvalif/wip-yuval-enable-trace-lua
common/tracer: remove is_enabled check in add_span methods

reviewed-by: cbodley, ljflores
2023-09-27 18:07:28 +03:00
Nizamudeen A
219c62bea6 mgr/dashboard: allow tls 1.2 with a config option
Provide the option to allow tls1.2

`ceph dashboard set-enable-unsafe-tls-v1-2 True` followed with a mgr
restart will enable tls 1.2.

With tls1.2 enabled
```
╰─$ nmap -sV --script ssl-enum-ciphers -p 11000 127.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-09-27 16:56 IST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00018s latency).

PORT      STATE SERVICE  VERSION
11000/tcp open  ssl/http CherryPy wsgiserver
|_http-server-header: Ceph-Dashboard
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 16.55 seconds
```

Without tls1.2 enabled (which defaults to tls 1.3)
```
╰─$ nmap -sV --script ssl-enum-ciphers -p 11000 127.0.0.1
Starting Nmap 7.93 ( https://nmap.org ) at 2023-09-27 16:54 IST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000075s latency).

PORT      STATE SERVICE  VERSION
11000/tcp open  ssl/http CherryPy wsgiserver
| ssl-enum-ciphers:
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A
|_http-server-header: Ceph-Dashboard
```

Fixes: https://tracker.ceph.com/issues/62940
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-09-27 17:25:30 +05:30
lichaochao
acd4348903 rgw: fix unwatch crash at radosgw startup
During radosgw initialization, if there is an exception in init_watch that causes the watcher registration to fail,
When finalize_watch is executed, a crash occurs due to unregister an unregistered watch.

Fixes: https://tracker.ceph.com/issues/60094

Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
2023-09-27 12:06:41 +02:00
Nizamudeen A
fe8fa180d3 mgr/dashboard: fix cephfs form validator
Number is not allowed as the starting character of the mds service

Fixes: https://tracker.ceph.com/issues/63005
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-09-27 15:13:34 +05:30
Ilya Dryomov
73dd77f8e7
Merge pull request #53523 from rhcs-dashboard/add-ceph-daemon-label-labeled-counters
exporter: add ceph_daemon labels to labeled counters as well

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-27 09:27:04 +02:00
Nizamudeen A
3df6ce3071
Merge pull request #52828 from rhcs-dashboard/angular-15
mgr/dashboard: upgrade to angular v15

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-09-27 11:11:35 +05:30
Nizamudeen A
95d8f59938
Merge pull request #53596 from rhcs-dashboard/fix-validations-in-cephfs-forms
mgr/dashboard: fix cephfs forms validations

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2023-09-27 10:43:14 +05:30
Venky Shankar
c01beb38b7 Merge PR #53529 into main
* refs/pull/53529/head:
	doc: update colorama, packaging
	cephfs-shell: drop LooseVersion for version.parse

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-27 09:57:25 +05:30
zdover23
4dee8b7928
Merge pull request #53685 from zdover23/wip-doc-2023-09-27-architecture-11-of-x
doc/architecture: edit "OSDs service clients directly"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-09-27 06:54:06 +10:00
Zac Dover
6f9dfd75b5 doc/architecture: edit "OSDs service clients directly"
Edit "OSDs service clients directly" in the list in
"Smart Daemons Enable Hyperscale" in doc/architecure.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-27 06:00:34 +10:00
Casey Bodley
0c07d6c63a rgw/async: use optional_yield for keystone and kms requests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-26 13:46:20 -04:00
John Mulligan
a8be9caefc cephadm: remove (doc)string
Remove a, now irrelevant (IMO), docstring that might have been
associated with the recently moved `cached_stdin` global. It's not
really clear how helpful it is in light of the new "compiled"
cephadm, so I am opting to remove it rather than move it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-26 13:45:35 -04:00
John Mulligan
f680b7d381 cephadm: black format context_getters.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-26 13:31:26 -04:00
John Mulligan
6ec6384c07 cephadm: rename fetch_tcp_ports to fetch_endpoints
Rename fetch_tcp_ports to fetch_endpoints to more closely match what
the function is doing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-26 13:31:26 -04:00
John Mulligan
fa4846a04a cephadm: move context based getters to context_getters.py
Move functions that exist mainly to pull information out of the
CephadmContext in various ways to a new context_getters.py module.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-26 13:31:26 -04:00
John Mulligan
f9e9f30b27 cephadm: move a logging line closer to where the data is used
Move a logging line closer to where the data being logging is
used. This avoids having a dependency on logging in a fairly
simple function and should make moving the function in a future
commit easier.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-26 13:31:26 -04:00
John Mulligan
3013780ea1 cephadm: move pathify & get_file_timestamp to file_utils
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-26 12:56:35 -04:00
Casey Bodley
71b2070b74 rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH
ERR_SIGNATURE_NO_MATCH means that we found the given access key in
keystone, so we should use reject() instead of deny() to prevent
other engines like LocalEngine from looking up the access key again

this change causes us to return the SignatureDoesNotMatch error expected
by s3test case test_list_buckets_bad_auth()

Fixes: https://tracker.ceph.com/issues/62989

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-26 12:16:54 -04:00
Shilpa Jagannath
595aeb542c rgw/multisite: call drain before flushing markers in incremental sync
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-09-26 11:44:23 -04:00
Vedansh Bhartia
1e2b08c8cc rgw: Add coverity annotations for missing mutex locks
Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-09-26 20:07:32 +05:30
Ronen Friedman
05c5f97c1d
Merge pull request #53615 from ronen-fr/wip-rf-great-move
tools, rgw/test: add std:: qualifiers to 'move'

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-09-26 17:34:41 +03:00
Laura Flores
1f378332c3
Merge pull request #53048 from YiteGu/add-slow-count-committed-kv
os/bluestore: add some slow count for bluestore
2023-09-26 10:22:44 -04:00
Laura Flores
4fbb4acab4
Merge pull request #52930 from leonid-s-usov/msg_wait
msg/AsyncMessenger: re-evaluate the stop condition when woken up in 'wait()'
2023-09-26 10:22:23 -04:00
Laura Flores
35506db4a4
Merge pull request #52906 from sseshasa/wip-fix-config-dump-pretty-print
mon/ConfigMonitor: Show localized name in "config dump --format json" output
2023-09-26 10:22:08 -04:00
Laura Flores
8dd1c9db87
Merge pull request #52200 from jinmyeonglee/61819
client: append to buffer list to save all result from wildcard command
2023-09-26 10:21:51 -04:00
Laura Flores
611c7b4cd1
Merge pull request #50585 from ssdohammer-sl/fix-metadata-obj-promote
osd: chunked type manifest object not to be promoted
2023-09-26 10:21:14 -04:00
Nizamudeen A
630ba3ab37 mgr/dashboard: upgrade to cypress 12
Looks like chrome 117 will need cypress >=12.15.0
https://github.com/cypress-io/cypress-documentation/issues/5479

Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-09-26 19:42:13 +05:30
Nizamudeen A
e2626fb63c mgr/dashboard: adapt and refactor jest test files
Use the `configureTestBed` as the placeholder for adding the
declarations, imports... that is required for the unit tests to run

Fixes: https://tracker.ceph.com/issues/62844
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-09-26 19:41:59 +05:30
Kamoltat (Junior) Sirivadhna
86047f0fe1
Merge pull request #51923 from kamoltat/wip-ksirivad-fix-61570
pybind/mgr/pg_autoscaler: fix warn when not too few pgs
Reviewed-by: Laura Flores <lflores@redhat.com>
2023-09-26 09:45:18 -04:00
Adam Kupczyk
7e2c94fd48
Merge pull request #53324 from aclamk/wip-aclamk-osd-op-scraper-fixes
Set of scraper improvements
2023-09-26 15:20:36 +02:00
avanthakkar
f061955f0b exporter: add ceph_daemon labels to labeled counters as well
Exporter missed adding the `ceph_daemon` or `instance_id`
labels(in case if rgw metrics) to the new labeled performance counters.

Fixes: https://tracker.ceph.com/issues/62874
Signed-off-by: avanthakkar <avanjohn@gmail.com>
2023-09-26 16:45:54 +05:30
Yingxin
0f539cd011
Merge pull request #53284 from xxhdx1985126/wip-seastore-remap-intermediate_key
crimson/os/seastore/transaction_manager: move intermediate_key by "remap_offset" when remapping the "back" half of the original pin

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2023-09-26 16:36:43 +08:00
Yingxin
0102c2afa3
Merge pull request #53455 from xxhdx1985126/wip-seastore-no-touch-pending
crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-09-26 16:35:16 +08:00
Xiubo Li
283b91a891 mds: fix stray CInodes' use-after-free bug when submit ELid entry
When submitting a journal log entry it could start a new segment
and it could advance the stray CInodes, which has been released
just before it. Just skip advancing the stray dentries when MDS is
shutting down.

Reported-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: commit 5a537476544("mds: introduce ELid event to create/close log")
Fixes: https://tracker.ceph.com/issues/62861
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-26 15:32:22 +08:00
zdover23
3ed92db078
Merge pull request #53657 from zdover23/wip-doc-2023-09-26-architecture-10-of-x
doc/architecture: edit "SDEH"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-09-26 14:38:38 +10:00
Zac Dover
5054f6c753 doc/architecture: edit "SDEH"
Edit the front matter of the "Smart Daemons Enable Hyperscale" section
of doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-26 13:45:26 +10:00
Xuehan Xu
f60f318450 crimson/os/seastore/cache: replace is_clean by is_stable_clean wherever
possible

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-09-26 11:30:54 +08:00
Xuehan Xu
dc6c091fd3 crimson/os/seastore/transaction_manager: move intermediate_key by
"remap_offset" when remapping the "back" half of the original pin

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-09-26 11:17:07 +08:00
Kamoltat (Junior) Sirivadhna
dc5d6ffa42
Merge pull request #52442 from kamoltat/wip-ksirivad-fix-tracker-61922
pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2023-09-25 17:57:38 -04:00
Adam King
47c8ad4524 mgr/cephadm: add unit test for _process_ls_output
This is a weird function to make a unit test for
since it's essentially just moving data from a
list of dicts into a list of DaemonDescriptions,
but wanted to have some coverage to lower the
chance of breaking something again.

Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 15:03:50 -04:00
Adam King
06cf5bb952 mgr/cephadm: fix REFRESHED column of orch ps being unpopulated
The way the daemon ls data was processed was changed in
1fd4132c7c
and it seems that commit removed a line that set the
last_refresh field. This commit just adds it back
in the new location after the change.

Without this in "ceph orch ps" the REFRESHED column
for every daemon just reports "-"

Fixes: https://tracker.ceph.com/issues/62954

Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 15:02:47 -04:00
Adam King
9b084a8f13 cephadm: re-format black net_utils.py
There was a conflict here between what black
and flake8 were okay with. After running
format-black flake8 would report

cephadmlib/net_utils.py:211:29: E203 whitespace before ':'
cephadmlib/net_utils.py:259:25: E203 whitespace before ':'
cephadmlib/net_utils.py:272:27: E203 whitespace before ':'

but removing the whitespace before the ":" would
cause black to complain. For parse_mon_ip and
parse_mon_addrv, it was doing array slicing with
a start of "0" so I believe we can just remove the
start point without affecting anything (since "0" is
just the beginning of the string anyway). For
get_ipv6_address it had to actually be altered in
a way that had the potential to be done incorrectly,
so I added a unit test for it in a previous commit
in order to make sure we maintain the behavior.

Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 15:00:24 -04:00
Adam King
4a826da317 cephadm: add unit test for get_ipv6_address
I wanted to modify this function slightly
to try to make both black and flake8 happy
with it, so adding a unit test to make sure
I don't break it.

Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 15:00:20 -04:00
Adam King
6222c395fc cephadm: move more funcs into net_utils.py
Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 14:57:55 -04:00
Adam King
ad54514806
Merge pull request #51926 from adk3798/host-drain-zap
mgr/cephadm: add ability to zap OSDs' devices while draining host

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-09-25 13:39:54 -04:00
Adam King
57d18f16e0
Merge pull request #53589 from adk3798/cephadm-refactor-host-facts
cephadm: create host_facts.py in cephadmlib

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-09-25 13:38:07 -04:00
Adam King
dec3717b45 cephadm: format black host_facts.py
Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 10:26:53 -04:00
Adam King
987452dbd6 cephadm: create host_facts.py in cephadmlib
For storing classes/functions related to gathering
information about the hosts such as disk enclosures
and networks

Signed-off-by: Adam King <adking@redhat.com>
2023-09-25 10:26:49 -04:00