Commit Graph

128790 Commits

Author SHA1 Message Date
Ernesto Puerta
cdc9f742df
Merge pull request #44190 from rhcs-dashboard/grafana-regex
monitoring/grafana: improve grafana unit tests variable substitution

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-12-21 17:58:17 +01:00
Ernesto Puerta
5a22aec48e
Merge pull request #44362 from rhcs-dashboard/kcli-e2e-improvements
mgr/dashboard: cephadm e2e job: improvements 

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-12-21 17:53:41 +01:00
Ernesto Puerta
76eb7a57da
Merge pull request #43838 from rhcs-dashboard/fix-53128-master
mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-12-21 17:33:21 +01:00
Yingxin Cheng
53248f6a36 crimson/os/seastore: fix potential leak for onodes to live across transactions
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-21 22:36:33 +08:00
Ernesto Puerta
efb062e5c3
Merge pull request #43942 from rhcs-dashboard/host-drain-frontend
mgr/dashboard: Implement drain host functionality in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-12-21 13:33:51 +01:00
Nizamudeen A
524c340562 mgr/dashboard: Implement drain host functionality in dashboard
Fixes: https://tracker.ceph.com/issues/51587
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-12-21 16:23:17 +05:30
Willem Jan Withagen
174055edfb common: Fix missing utility include
See:
    https://en.cppreference.com/w/cpp/utility/move

Detected on FreeBSD/Clang/libc++:
/home/jenkins/workspace/ceph-master-compile/src/common/deleter.h:111:43: error: no member named 'move' in namespace 'std'
  impl(deleter next) : refs(1), next(std::move(next)) {}

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-12-21 11:38:54 +01:00
Kamoltat
abaab51dd8 Added ReleasesNotes and documentation
Add Release Notes and remove any `profile`
related stuff in the autoscaler documentation
and replace it with `bulk` flag.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-12-21 07:56:37 +00:00
Neha Ojha
1f28c06f2a
Merge pull request #42750 from aclamk/wip-aclamk-bluefs-log-incremental
os/bluestore: incremental update mode for bluefs log

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2021-12-20 17:15:13 -08:00
Ernesto Puerta
45b63ebf44
Merge pull request #44368 from ceph/fix-api-tests
mgr/dashboard: run-backend-api-tests.sh: Older setuptools

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2021-12-20 23:43:18 +01:00
Adam Kupczyk
2292a4d85a os/bluestore/bluefs: Add tracking of bluefs log in noop replay mode
Keep updating bluefs log when printing content of bluefs replay log.
Without this modification we only have initial content of log.
Log can be printed by 'ceph-bluestore-tool bluefs-log-dump'.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Adam Kupczyk
ae5c5b7655 os/bluestore/bluefs: Sync BlueFS log with its allocation delta
BlueFS log is the only file that we can append to.
When we append to file we must take into consideration previously commited allocations,
otherwise update will be miscalculated.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Adam Kupczyk
baed63ab12 test/objectstore/bluefs_test: Add test for continuation of previous BlueFS log
Added test that verifies that in update mode we properly pick up delta.
BlueFS log is the only file that can be appended to, but it is done in very indirect way.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Igor Fedotov
821aadaaa9 os/bluefs: allow incremental file metadata updates in bluefs log
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2021-12-20 21:55:37 +00:00
Kamoltat
b218335dcc qa/workunits/cephtool: modified test.sh
Included bulk flag in the test script
such it tests the `get` and `set` of
the flag

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-12-20 21:46:37 +00:00
Kamoltat
c194f4a3eb qa/workunits/mon/pg_autoscaler: modified test script
Modified test scrtipt to include `bulk` and
remove all `profile` options.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-12-20 21:46:37 +00:00
Kamoltat
d90a3746de pg_autoscaler/test: Modified unit-test for bulk flag
Modified the unit-test cases to account for bulk
flag and remove any `profile` related things.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-12-20 21:46:37 +00:00
Kamoltat
5f0f967c93 mon: osd pool create <pool-name> with --bulk flag
Creating the pool with `--bulk` will allow
the pg_autoscaler to use the `scale-down`
mode on.

Creating pool:

`ceph osd pool create <pool-name> --bulk`

Get var:

`ceph osd pool get <pool-name> bulk`

Set var:

`ceph osd pool set <pool-name> bulk=true/false/1/0`

Removed `autoscale_profile` and incorporate bulk flag
into calculating `final_pg_target` for each pool.

bin/ceph osd pool autoscale-status no longer has
`PROFILE` column but has `BULK` instead.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-12-20 21:46:37 +00:00
Patrick Donnelly
61de84a522
Merge PR #44315 into master
* refs/pull/44315/head:
	doc/cephfs: mds default cache memory limit is now 4GB

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-12-20 14:41:43 -05:00
David Galloway
4ab2df179b
pybind/mgr/dashboard/run-backend-api-tests: Older setuptools
https://github.com/pypa/setuptools/issues/2353

Signed-off-by: David Galloway <dgallowa@redhat.com>
2021-12-20 18:01:20 +01:00
Matt Benjamin
4ff723061f
Merge pull request #42416 from galsalomon66/wip-test-albin-error-status
RGW/s3select : new-s3select-responds presto-alignments output-serialization
2021-12-20 10:50:44 -05:00
Sebastian Wagner
2cfb681003
mgr/cephadm: Fix count for OSDs with OSD specs
osd counting is special

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-12-20 16:08:07 +01:00
Kefu Chai
8fd968d26c seastar: pick up upstream change which includes cryptopp fix
update seastar submodule to upstream master, so we don't need to
use our own fork.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-12-20 21:59:55 +08:00
Igor Fedotov
0960f981f7 os/bluestore: get rid of fake onode nref increment for pinned entry
Looks like this isn't necessary any more after fixing
https://tracker.ceph.com/issues/53002

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2021-12-20 15:15:15 +03:00
Alfonso Martínez
9aedeaf201 mgr/dashboard: cephadm e2e job: improvements
- Avoid jenkins env. error: Input file /workdir/./src/pybind/mgr/dashboard/ci/cephadm/ceph_cluster.yml not found.Leaving...
- More meaningful log output.

Fixes: https://tracker.ceph.com/issues/53666
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2021-12-20 09:12:37 +01:00
Aashish Sharma
2ba1a5a266 mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements
Ensure "Storage capacity" keeps the "Description : Value" approach ("Number of devices: X" and "Raw Capacity: Y" in different lines).Correct issue with "host by services" host count

Fixes: https://tracker.ceph.com/issues/53128
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-12-20 13:30:57 +05:30
Xuehan Xu
dd52180773 crimson/os/seastore/cache: add lba nodes' logical begin addresses to extent_info_t
LBAManager rely on lba nodes' logical start addresses to judge whether an lba extent
is still live

Fixes: https://tracker.ceph.com/issues/53612
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-12-20 14:39:57 +08:00
胡玮文
d4b239c635 mgr/dashboard: fix white screen on Safari
Safari do not support lookbehind in regular expression.

Fixes: https://tracker.ceph.com/issues/53665
Signed-off-by: 胡玮文 <huww98@outlook.com>
2021-12-20 12:54:29 +08:00
Kefu Chai
242e8f35e8
Merge pull request #44350 from Huber-ming/fix_annotation_misspelling
rgw: fix misspelling of 'effort'

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-19 15:53:45 +08:00
胡玮文
620f1ef061 common: fix fmt::format_to deprecated warning
Signed-off-by: 胡玮文 <huww98@outlook.com>
2021-12-19 15:39:46 +08:00
Kefu Chai
6e1a6f0aa1 mgr: move DaemonState methods into .cc
for faster compilation, and for better readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-12-19 15:02:45 +08:00
Kefu Chai
95a96cffc3 mgr: refactor DaemonState::set_metadata()
restructure DaemonState::set_metadata() using for_each_pair() for better
readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-12-19 15:02:45 +08:00
Kefu Chai
b5d638dc0f common/str_map: reimplement get_str_list() using for_each_pair
to avoid creating a temporary list<string> and then dropping it
on the floor after iterating through it for collecting the kv
pairs in it.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-12-19 15:02:45 +08:00
Kefu Chai
997715e259 cmake: use new CMP0127 policy
CMP0127 is a new policy introduced by CMake 3.22. without specifying
the policy, we'd have following warning when configuring the building
system using CMake 3.22:

CMake Warning (dev) at /usr/share/cmake-3.22/Modules/CMakeDependentOption.cmake:84 (message):
  Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
  Syntax.  Run "cmake --help-policy CMP0127" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
  CMakeLists.txt:255 (CMAKE_DEPENDENT_OPTION)
This warning is for project developers.  Use -Wno-dev to suppress it.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-12-19 15:02:10 +08:00
Kefu Chai
9f128e94b4
Merge pull request #44277 from fengchunsong/dpdk-rte-exit
msg/async/dpdk: replacing rte_exit with ceph_assert to avoid exit fai…

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-18 22:19:50 +08:00
galsalomon66
fe0c944777 using fmt for message formatting
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
abccc5d77b fix comments; enum-class; typo; documentation
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
b13b902aa7 typo
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
784c70c496 modification for error-handling and boto3
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
72e82d45ac adding/modifying documentation for output-serialization, s3select-response, syntax support, error-handling
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
9dc21c2575 s3select: point to s3select/master(after merging pr #90)
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:55 +02:00
galsalomon66
f05cdb71ee in the case Presto is push-down query, it also send output-serialization parameter('&#10;'), that should be handle as '\n'
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:54 +02:00
galsalomon66
7956fb30bf s3tests.yaml points to ceph-master
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:54 +02:00
gal salomon
f128f04c60 RGW: Implement continuation, progress, stats, end s3select response
RGW/S3select: Implement output-serializationi. user may request different CSV defintions
    for output (field delimiter, row delimiter, quote handling.

    RGW/S3select: Implement presto-alignments. presto-application sends
    queries with table-alias,case insensitive, and with no-semicolon at the
    end of statement.

    RGW/s3select: zero object size issue

    Signed-off-by: Albin Antony <aantony@redhat.com>

    Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-12-18 13:51:54 +02:00
Huber-ming
88571bb94d rgw: fix misspelling of 'effort'
Signed-off-by: Huber-ming <zhangsm01@inspur.com>
2021-12-18 14:43:34 +08:00
Joseph Sawaya
8742173e20
Merge pull request #43139 from josephsawaya/rook-orch-qa
qa/tasks/rook: test reapplication of drive groups stored in mgr
2021-12-17 12:50:18 -05:00
Sage Weil
b55781d412 Merge PR #44337 into master
* refs/pull/44337/head:
	mon: prevent new sessions during shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-12-17 09:59:48 -05:00
Casey Bodley
aa98a82c95
Merge pull request #43872 from Huber-ming/reshard
rgw: init bucket index only if putting bucket instance info succeeds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:42 -05:00
Casey Bodley
34cc746b75
Merge pull request #43817 from joke-lee/UploadPartCopy-src-not-exist
rgw: fix UploadPartCopy error code when src object not exist and src bucket not exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:32 -05:00
Casey Bodley
6e0a171258
Merge pull request #43259 from paulreece42/wip-rgw-xmlfix
rgw: allow CompletedMultipartUpload in XML

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-12-17 09:30:21 -05:00