Commit Graph

130997 Commits

Author SHA1 Message Date
Sarthak0702
7f05a9089a githubmap: Email and Id upadted
Signed-off-by: Sarthak0702 <sarthak.dev.0702@gmail.com>
2022-05-11 23:24:46 +05:30
Casey Bodley
f0e30f67ab
Merge pull request #45064 from s-t-e-v-e-n-k/no-direct-use-of-nose
test: No direct use of nose

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 12:28:48 -04:00
Yuri Weinstein
3bc0354bc4
Merge pull request #45762 from NitzanMordhai/wip-nitzan-test-cls-lock-expiered-locks
cls_lock: check expired lock before unlock

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2022-05-11 07:34:42 -07:00
Yuri Weinstein
6322d1aaae
Merge pull request #45529 from NitzanMordhai/wip-nitzan-pglog-dups-not-trimmed
osd/PGLog.cc: Trim duplicates by number of entries

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-05-11 07:34:12 -07:00
Yuri Weinstein
7268e4574c
Merge pull request #44839 from NitzanMordhai/wip-nitzan-mon-ok-to-stop
mon/Monitor: Validate monitor ids during 'mon ok-to-stop' cmd

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-11 07:32:49 -07:00
Venky Shankar
6f459d8bf2
Merge pull request #46164 from rishabh-d-dave/vstart-minor-fixes
qa/vstart_runner: minor bug fix and minor adjustments

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-11 11:26:09 +05:30
Samuel Just
d22869b24f
Merge pull request #46217 from Matan-B/wip-matanb-c-shard_services
crimson/osd: Moving shard_services to PGBackend

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-10 21:50:27 -07:00
Kefu Chai
7f022510a7
Merge pull request #46208 from rzarzynski/wip-crimson-main-using-cleanups
crimson/osd: drop "using ..." from main.cc

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-05-11 10:41:06 +08:00
Venky Shankar
5e1ec0cb49
Merge pull request #46110 from dparmar18/fix_onecmd
cephfs-shell: make onecmd() print proper error msg

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 22:58:41 +05:30
Venky Shankar
8cfef7194c
Merge pull request #46047 from dparmar18/qa_testcephfs_shell_pep8_fixes
qa/cephfs/test_cephfs_shell: Fixed pep8 formatting violations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 22:57:16 +05:30
Ernesto Puerta
be1c083b64
Merge pull request #45808 from rhcs-dashboard/add-cpu/mem-usage
mgr/dashboard: introduce memory and cpu usage for daemons

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: ceph-jenkins <NOT@FOUND>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-05-10 18:06:21 +02:00
Ernesto Puerta
fd7d50c831
Merge pull request #46190 from rhcs-dashboard/smart-data-fix
mgr/dashboard: fix smart data error

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-05-10 17:55:58 +02:00
Venky Shankar
3ed1dfee84
Merge pull request #45894 from rishabh-d-dave/stop-sh-check-mdss
src/stop.sh: check if ceph-mds daemon(s) exist beforehand

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 20:28:01 +05:30
Kefu Chai
006cfa4df8
Merge pull request #46143 from tchaikov/wip-dashboard-angular
cmake: s/--prod/--configuration=production/

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-05-10 22:19:16 +08:00
Matan Breizman
9af99c3024 crimson/osd: Moving shard_services to PGBackend
ec and replicated backends are derived from PGBackend,
shard_services should be a member of the base class.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-05-10 14:16:05 +00:00
Casey Bodley
66331f4ba2
Merge pull request #46093 from pdvian/wip-admin-curl
examples: Add rgw script to access admin APIs using curl

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-10 08:24:41 -04:00
Ernesto Puerta
c959ae67ad
Merge pull request #46196 from rhcs-dashboard/smart-data-scsi
mgr/dashboard: smart data for devices with scsi protocol

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2022-05-10 14:17:12 +02:00
Ernesto Puerta
7752d87712
Merge pull request #46201 from rhcs-dashboard/daemon_count_hosts
mgr/dashboard: show daemon count instead of services in host page 

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-05-10 13:52:18 +02:00
Ernesto Puerta
e24adc952b
Merge pull request #46174 from rhcs-dashboard/physical-disks-selection
mgr/dashboard: devices with same UID causes multiselection

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-05-10 13:51:27 +02:00
Ernesto Puerta
84f7da7ada
Merge pull request #46159 from rhcs-dashboard/disable-external-grafana-snapshots
mgr/cephadm: Disable external snapshots in grafana

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-05-10 13:48:19 +02:00
Venky Shankar
338e18b893
Merge pull request #46169 from rishabh-d-dave/vr-run-mnt-cmd
qa/vstart_runner: fix bug in LocalFuseMount._run_mount_cmd()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 15:08:01 +05:30
Nizamudeen A
4fab663983 mgr/dashboard: smart data for devices with scsi protocol
In the dashboard, we've been showing smart data for hdd devices with ata
protocol only. Otherwise we show a No Smart Data found error which is
clearly misleading since Smart Data is returned even in the api call.

So this PR is trying to show the smart data for hdd devices
that uses scsi protocol too.

Fixes: https://tracker.ceph.com/issues/55574
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-05-10 12:55:18 +05:30
Nizamudeen A
ee78b47612 mgr/dashboard: show daemon count instead of services in host page
Fixes: https://tracker.ceph.com/issues/47218
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-05-10 12:14:33 +05:30
Rishabh Dave
c9807d35e0 qa/vstart_runner: fix args to os.path.join()
Result of os.path.join() before "./bin/ceph-mds" and after
"./bin/./ceph-mds".

Before -

	2022-05-05 19:36:11,100.100 DEBUG:__main__:> ./bin/./ceph-mds -i a

After -

	2022-05-05 19:38:48,179.179 DEBUG:__main__:> ./bin/ceph-mds -i a

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-10 11:03:24 +05:30
Rishabh Dave
3dd150959e qa/vstart_runner: minor bug fix and minor adjustments
The message regarding deletion of helper tools is printed for every
command. This message should be printed only when applicable.

Besides -

* Move XXX comments to _do_run() since it increases visibility of
  these messages.
* Move omission of arguments stuff to new method clear up the clutter.
* And remove shell as a parameter from _perform_checks_and_adjustments
  since it's redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-10 11:02:08 +05:30
Venky Shankar
f005520194
Merge pull request #37667 from rishabh-d-dave/vr-inherit-Remote
vstart_runner: let LocalRemote inherit Remote

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-10 10:30:22 +05:30
Samuel Just
ab6b68efe4
Merge pull request #46194 from athanatos/sjust/wip-55565
crimson: fix op tracking related bugs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-05-09 17:31:54 -07:00
Samuel Just
29a00b4482 crimson/common/operation: trim AggregateBlockingEvent::events entries
PglogBasedRecovery and BackfillRecovery reuse the same Operation
until their respective operations are complete.  Each recovery
operation adds an entry to AggregateBlockingEvent::events.  This
way, we only retain entries that are currently blocking.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-09 14:04:56 -07:00
Samuel Just
396ff7b702
Merge pull request #46127 from Matan-B/wip-matanb-cyan-clone
crimson/os: Add OP_CLONE to cyanstore

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-09 12:53:30 -07:00
Samuel Just
15b5a93a54
Merge pull request #46205 from rzarzynski/wip-crimson-cls_rgw-objapi
crimson/osd, rgw: implement the new Objclass API methods for cls_rgw

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-09 12:47:17 -07:00
Samuel Just
813ab9947d
Merge pull request #44418 from AmnonHanuhov/wip-fix_futurized_store
crimson/os: Don't limit the amount of returned keys per omap get call

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-09 12:46:25 -07:00
Radosław Zarzyński
86043d961d crimson/osd: drop "using std::string" from main.cc
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-09 18:12:43 +02:00
Radosław Zarzyński
fc4e9ff852 crimson/osd: drop "using config_t = ..." from main.cc
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-09 18:12:43 +02:00
Radosław Zarzyński
14c04fb03d crimson/osd, rgw: implement the new Objclass API methods for cls_rgw
They were introduced by 3877c1e37f.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-09 17:13:58 +02:00
Venky Shankar
d4f36d647c
Merge pull request #45732 from mchangir/mgr-snap_schedule-validate-fs-for-all-commands
mgr/snap_schedule: validate fs name argument

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-09 19:30:45 +05:30
dparmar18
c5f3c5122c cephfs-shell: make onecmd() print proper error msg
Rationale: Whenever a python exception occurred in cephfs-shell,
           it would often only be the exception message but doesn't
           say anything about the type of exception. For example if
           `ZeroDivisionError: division by zero` occurred, the onecmd()
           would print `division by zero` but will omit the type of
           exception. In this case it's easy to understand but let's
           say an `KeyError` exception occurred for a key `9999` which
           is not existent in the dictionary, onecmd() would print
           just `9999` in this scenario and it would be very difficult
           to interpret what type of error it is.

Fixes: https://tracker.ceph.com/issues/55536

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-09 19:19:39 +05:30
Dhairya Parmar
53aa69e373
Merge branch 'master' into qa_testcephfs_shell_pep8_fixes 2022-05-09 15:46:04 +05:30
dparmar18
c02ab23ede qa/cephfs/test_cephfs_shell: Fixed pep8 formatting violations
NOTE: Although most of the issues are fixed but a few function
      and variable names are unchanged in order to prevent
      ambiguity and preserve their meaning.

      They are:
      - functions: setUp(), test_ls_H_prints_human_readable_file_size(),
      - variables: ls_H_output, ls_H_file_size

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-05-09 15:40:08 +05:30
Nizamudeen A
f38e3e96b8 mgr/dashboard: fix smart data error
the error in the log was this

```
"/usr/share/ceph/mgr/dashboard/services/ceph_service.py", line 253, in _get_smart_data_by_device
May 06 07:38:39 occldlr750-1.occl208.lab conmon[2142938]:     svc_type, svc_id = daemon.split('.')
May 06 07:38:39 occldlr750-1.occl208.lab conmon[2142938]: ValueError: too many values to unpack (expected 2)
```

on the cluster, the output of `ceph device ls-by-host` looks like this

```
ceph: root@occldlr750-1 /]# ceph device ls-by-host occldlr750-1.occl208.lab
DEVICE                        DEV   DAEMONS                       EXPECTED FAILURE
DELLBOSS_VD_cbd004c975390010  sda   mon.occldlr750-1.occl208.lab
WDC_WUH721818AL5204_3FGZR3JT  sdda  osd.20
WDC_WUH721818AL5204_3FH4315T  sdbf  osd.94
WDC_WUH721818AL5204_3FHP58TT  sdec  osd.30
WDC_WUH721818AL5204_3FHSK8HT  sdu   osd.78
WDC_WUH721818AL5204_3FHVTS9T  sdfi  osd.47
WDC_WUH721818AL5204_3FHWJE8T  sdv   osd.23
WDC_WUH721818AL5204_3FHXHETT  sdcl  osd.11
WDC_WUH721818AL5204_3FHXKP1T  sdcj  osd.10
```

the first device is mon and its name is mon.occldlr750-1.occl208.lab.

In our dashboard code, when fetching the smart data we have a line like
this

`svc_type, svc_id = daemon.split('.')`

so for the mon the output of `daemon.split('.') will be ['mon', 'occldlr750-1', 'occl208', 'lab']. The svc_id gets split into three because of the split. I am changing that and giving the criteria as splitting only on the first occurence of the dot and the considering everything that comes after the dot as the svc_id of the device.

Fixes: https://tracker.ceph.com/issues/55571
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-05-09 11:14:49 +05:30
Liu-Chunmei
524ef52573
Merge pull request #46064 from liu-chunmei/crimson-pool-quota
Crimson pool quota

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-08 19:29:39 -07:00
Nizamudeen A
9fe18a6252 mgr/dashboard: devices with same UID causes multiselection
In the Physical Disks page, the uids for multiple devices are coming in
as same and that causes the selection to go berserk and select multiple
rows with same UID. The uid is generated in the frontend service call
itself. I just added some more parameters to it inorder to make it more
unique.

The second issue is the number of selected number getting multiplied
exponentially. Its because each time the table is updated or refreshed,
we push the row with the number of selected items we had before and that
causes the number of selection to multiply.

Fixes: https://tracker.ceph.com/issues/55523
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-05-08 20:20:35 +05:30
Matan Breizman
54b8d69232 crimson/os: Add OP_CLONE to cyanstore
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-05-08 08:27:25 +00:00
chunmei-liu
4b55a2b411 crimson/osd: check full
need check if pool exceeds its quota

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-05-07 16:47:33 -07:00
chunmei-liu
01e7122c97 crimson/errorator: change errorator template to int
need define EDQUOT which is not included in std::errc.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-05-07 16:36:21 -07:00
Samuel Just
d0708b5452 crimson/osd/pg_recovery: remove new_started, fix parallel_for_each call
Presumably typo from b7e2b11595.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-06 23:56:09 -07:00
Samuel Just
027d6c2940 crimson/common/operation: use std::list for AggregateBlockingEvent::events
std::vector's references aren't stable after emplace_back.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-06 23:56:09 -07:00
Samuel Just
df9827eba8 crimson/osd/pg_recovery: avoid using create_part_trigger() directly
AggregateBlockingEvent::TriggerI::maybe_record_blocking() should handle
invoking create_part_trigger().

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-06 18:05:15 -07:00
Samuel Just
83e9db3a8b crimson: retain reference to WaitForObjectRecovery until blocker resolves
PGRecovery::on_global_recover destroys the map entry without waiting for
the future returned from

    seastar::future<> wait_for_recovered(BlockingEvent::TriggerI& trigger) {

This commit changes WaitForObjectRecovery to be refcounted and retains a
reference until the future resolves.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-06 23:21:54 +00:00
Anthony D'Atri
74e57a6ab2
Merge pull request #46170 from elacunza/patch-1
doc/rados/operations: Add missing required parameters to stretch-mode CRUSH rule example
2022-05-06 08:42:41 -07:00
Prashant D
9b40fdad5a examples: Add rgw script to access admin APIs using curl
The rgw_admin_curl.sh script will allow end-user/developers to
access RGW admin APIs through curl command.

Signed-off-by: Prashant D <pdhange@redhat.com>
2022-05-06 11:14:12 -04:00