Commit Graph

130465 Commits

Author SHA1 Message Date
Ville Ojamo
04b8cf09b3 doc/rados/api: fix invalid RST syntax and don't sudo for vim
Fix invalid syntax where "linenos:" was printed in the
final rendered documentation instead of being used as
formatting syntax.

There is no need to use sudo for editing a source file
so run vim without sudo.

Change the whole block to use bash prompt since it is
a command.

Also modify the preceding text to hopefully not bold it.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2022-05-02 11:50:52 +07:00
Samuel Just
3a21a91528
Merge pull request #45775 from liu-chunmei/seastore-zero
crimson: seastore add OP_ZERO support

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2022-04-12 20:20:54 -07:00
Samuel Just
d06ce17625 crimson: Implement ObjectDataHandler::zero using hole punching
Trim already treats Reserved regions as zero, let's use that
for zero as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-04-12 17:49:03 -07:00
Samuel Just
88319b475f crimson/os/seastore/object_data_handler: don't return empty buffers from split_pin*
Always return std::nullopt rather than an empty buffer -- this way users
can rely on this as an invariant.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-04-12 17:48:52 -07:00
Samuel Just
76e9262efb test/crimson/seastore: improve test_seastore zero() coverage
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-04-12 17:48:45 -07:00
chunmei-liu
b392b2a141 crimson: add seastore::zero unit test
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-04-12 17:48:39 -07:00
chunmei-liu
31dcd54dff crimson: seastore add OP_ZERO support
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-04-12 17:48:31 -07:00
Yuri Weinstein
ed556e6bc9
Merge pull request #45756 from rzarzynski/wip-common-no-cpp17-second_round
common/bl: fix FTBFS on C++11 due to C++17's if-with-initializer

Reviewed-by: Kefu Chai <kchai@redhat.com>
2022-04-12 13:51:58 -07:00
Yuri Weinstein
a31813d251
Merge pull request #45819 from ljflores/wip-anonymize-telemetry-host-names
mgr/telemetry: anonymize daemons in telemetry `perf_counters`

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2022-04-12 12:12:26 -07:00
Yuri Weinstein
61b975ae6d
Merge pull request #45802 from ljflores/wip-config-dump-yaml
ceph.in: clarify the usage of `--format` in the ceph command

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-04-12 12:10:48 -07:00
Neha Ojha
a32221cb9a
Merge pull request #45531 from ifed01/wip-ifed-alloc-replay-with-bin
os/bluestore: proper locking for Allocators' dump

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
2022-04-12 11:06:19 -07:00
Ilya Dryomov
3c556878cf
Merge pull request #45684 from CongMinYin/pwl-add-stats
librbd/cache/pwl: add pwl metrics in "rbd status" display

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-12 16:48:54 +02:00
Casey Bodley
d104ae541c
Merge pull request #45357 from cbodley/wip-54531
rgw: disable RGWDataChangesLog::add_entry() when log_data is off

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-04-12 09:34:58 -04:00
Igor Fedotov
d7aa5decf3 os/bluestore: proper locking for Allocators' dump methods
Plus renaming parametrized dump to foreach()
Fixes: https://tracker.ceph.com/issues/54973

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2022-04-12 12:16:55 +03:00
Igor Fedotov
55def1c5ad test/allocator_replay_test: introduce check for duplicates
This performs check for duplicates using free dump.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2022-04-12 12:09:44 +03:00
Igor Fedotov
82c1036ffe test/allocator_replay_test: introduce binary format for free list dump
Adding new command to export free dump to binary format plus capability
to use new format for replay.
This dramatically increases large dump loading.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2022-04-12 12:09:44 +03:00
Ilya Dryomov
b100d1fb5b
Merge pull request #45677 from ideepika/wip-ninja-default
ceph.spec: make ninja-build package install always

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2022-04-12 10:57:30 +02:00
Liu-Chunmei
5ee066c9b9
Merge pull request #45820 from liu-chunmei/crimson-do_osd_ops_params_t
crimson: keep do_osd_ops_params_t alive when call do_osd_ops

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-04-11 21:57:36 -07:00
Samuel Just
5b5b457691
Merge pull request #45137 from myoungwon/wip-51627-2
osd: return appropriate error if the object is not manifest

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-04-11 16:31:03 -07:00
chunmei-liu
211e5644d9 crimson: keep do_osd_ops_params_t alive when call do_osd_ops
otherwise stack-under-overflow

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-04-11 16:30:49 -07:00
Yuri Weinstein
8cee59ff93
Merge pull request #45744 from idryomov/wip-stretch-last-force-resend
mon/OSDMonitor: properly set last_force_op_resend in stretch mode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2022-04-11 15:58:23 -07:00
Yuri Weinstein
1b47a14f55
Merge pull request #45670 from pdvian/wip-fix-mgr-daemon-state
mgr, mon: Keep upto date metadata with mgr for MONs

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-04-11 15:57:36 -07:00
Yuri Weinstein
9de51de27d
Merge pull request #45599 from amathuria/amathuri-54994-fix
osd: add scrub duration for scrubs after recovery

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2022-04-11 15:56:54 -07:00
Yuri Weinstein
8b9cfe39e6
Merge pull request #45547 from nkshirsagar/master
Catch exception if thrown by __generate_command_map()

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-04-11 15:55:43 -07:00
Yuri Weinstein
15548e020e
Merge pull request #45505 from pdvian/wip-fix-daemon-version
mgr, mgr/prometheus: Fix regression with prometheus metrics

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-04-11 15:51:26 -07:00
Samuel Just
0380a2b446
Merge pull request #45822 from liu-chunmei/crimson-enametoolong
crimson: check -ENAMETOOLONG for Name, Locator, NameSpace

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-04-11 14:35:06 -07:00
Ernesto Puerta
9ff904a467
Merge pull request #45584 from votdev/issue_54983_default_rgw_daemon
mgr/dashboard: RGW users and buckets tables are empty if the selected gateway is down

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2022-04-11 20:50:56 +02:00
Ernesto Puerta
63b97035bd
Merge pull request #45790 from rhcs-dashboard/host-toggle-column-fix
mgr/dashboard: datatable in Cluster Host page hides wrong column on selection

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-04-11 20:48:48 +02:00
Ernesto Puerta
0a7d9ef280
Merge pull request #45007 from VallariAg/dashboard-complexity-cleanup
mgr/dashboard: reduce method (cyclomatic) complexity 

Reviewed-by: VallariAg <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-04-11 20:45:58 +02:00
Adam King
29f18c6f7b
Merge pull request #45070 from rkachach/fix_issue_52042
mgr/cephadm: Making default cephadm shell cmd easier

Reviewed-by: Adam King <adking@redhat.com>
2022-04-11 13:18:36 -04:00
Adam King
bb9034dba0
Merge pull request #45056 from ktdreyer/explain-cephadm-tox
cephadm: add comment explaining docker.io grep test

Reviewed-by: Adam King <adking@redhat.com>
2022-04-11 12:51:59 -04:00
Casey Bodley
52a8345421
Merge pull request #45815 from cbodley/wip-55232
test/rgw: use mock OpsLogSink instead of OpsLogSocket

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-04-11 12:42:51 -04:00
Casey Bodley
167255d382
Merge pull request #45770 from yuvalif/wip-yuval-fix-54416
test/multisite: dont use path when mrun outside of src tree

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-04-11 12:40:51 -04:00
Adam King
07a4e9ebbe
Merge pull request #45347 from mgfritch/cephadm-config-noreplace
cephadm: preserve `authorized_keys` file during upgrade

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2022-04-11 10:10:55 -04:00
Adam King
aae28b1006
Merge pull request #45589 from melissa-kun-li/bootstrap_registry_warning
cephadm: show error during bootstrap if private registry cred not provided 

Reviewed-by: Adam King <adking@redhat.com>
2022-04-11 10:07:50 -04:00
Adam King
b8d25e0c62
Merge pull request #45685 from rkachach/fix_issue_47905
mgr/cephadm: improving logging to send errors to stderr

Reviewed-by: Adam King <adking@redhat.com>
2022-04-11 10:07:07 -04:00
Ilya Dryomov
88db8db1b4
Merge pull request #45798 from adk3798/iscsi-only-pid-limit
cephadm: only apply unlimited pids-limit to iscsi and rgw

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 16:02:50 +02:00
Venky Shankar
34c6c468ea
Merge pull request #45689 from vshankar/wip-55110
mount.ceph: remove `ms_mode' mount option when switching to old-syntax

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-11 19:19:23 +05:30
Ilya Dryomov
a23b9c9938 qa: test_iscsi_pids_limit.sh: increase sleep time
It could take longer than 30 seconds to fork off 40000 processes on
a busy system.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 13:15:22 +02:00
Ilya Dryomov
e996fd8060 rbd: include persistent cache metrics in "rbd status" report
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 11:33:24 +02:00
Venky Shankar
6e28d3406d mount.ceph: remove `ms_mode' mount option when switching to old-syntax
... and switch to using v1 addresses (if users haven't specified those
explicitly). kernel versions <5.11 do not understand `ms_mode' mount
option which would result in mount failure.

Fixes: http://tracker.ceph.com/issues/55110
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-04-11 05:03:59 -04:00
Ernesto Puerta
4417d76fd0
Merge pull request #45791 from rhcs-dashboard/rm-true
build: install-deps failing in docker build

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-04-11 10:26:47 +02:00
Ilya Dryomov
9324ab9471 rbd: factor out get_percentage() helper
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 08:26:47 +02:00
Ilya Dryomov
d64a3ae265 librbd/cache/pwl: no need to set clean and empty in remove_pool_file()
It is redundant -- the only caller sets both since commit 6593e31fff
("librbd/cache/pwl: correct cache state").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 08:26:47 +02:00
Ilya Dryomov
016882925a librbd/cache/pwl: avoid inconsistencies in ImageCacheState
When empty and/or clean bools are updated in I/O handling code paths,
ImageCacheState becomes inconistent for a short while: e.g. with clean
transitioned to true, dirty_bytes counter could still be positive
because the counters are updated only in periodic_stats().  Move to
updating the counters in update_image_cache_state(Context*) to avoid
this.

update_image_cache_state(Context*) now requires m_lock -- most call
sites already hold it anyway.  The only problematic call site was
AbstractWriteLog::shut_down() callback chain: perf_stop() needed to
be moved to the very end since perf counters must be alive now for
update_image_cache_state() to work.

Don't override expect_op_work_queue() in unit tests: completing
context in the same thread now results in a deadlock on m_lock in
all test cases that call AbstractWriteLog::init().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 08:26:47 +02:00
Ilya Dryomov
7678ee2490 librbd/cache/pwl: handle invalid ImageCacheState json
get_json_format() and create_image_cache_state() attempt to get
particular keys which could result in an unhandled std::runtime_error
exception.  Conversely, ImageCacheState constructor just swallows that
exception which could leave the newly constructed object incorrectly
initialized.  Avoid doing parsing in the constructor and introduce
init_from_config() and init_from_metadata() methods instead.

While at it, move everything out from under "persistent_cache" key.
Also fix init_state_json_write test case which stopped working now
that types are enforced by json_spirit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 08:26:47 +02:00
Yin Congmin
769f3a06ec librbd/cache/pwl: add basic metrics to ImageCacheState
Add basic metrics to ImageCacheState and persist them, including
allocated_bytes, cached_bytes, dirty_bytes, free_bytes and hit/miss
info.

Leverage periodic_stats() timer to call update_image_cache_state.
In order to avoid outputting too much debug information, the original
statistics output log level is changed to 5.

Switch to json_spirit for encoding because encode_json encodes bool as
"true"/"false" string.

Remove rbd_persistent_cache_log_periodic_stats option because we need
to always update cache state.

[ idryomov: add cached_bytes and hits_partial; report misses and
  miss_bytes instead of respective totals; naming ]

Fixes: https://tracker.ceph.com/issues/50614
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 08:26:47 +02:00
zdover23
c82efef051
Merge pull request #45832 from anthonyeleven/anthonyeleven/min-alloc-size-53752
doc/rados/configuration: document min_alloc_size values and space amplification

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2022-04-11 04:35:54 +10:00
Anthony D'Atri
fadcbbefb8 doc/rados/configuration: document min_alloc_size values and space amplification
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-04-10 03:27:34 -07:00
Yaarit Hatuka
2f4cc770e7 mgr/telemetry: fix daemon anonymization in perf_counters
Anonymized daemons now appear with a SHA1 digest instead of their
original identifier, e.g.:

    "perf_counters": {
        "mon.1b1b829ba9298527f4934053a4742a1710937007": {
            "mon": {
                "election_call": {
                    "value": 1
                },
                ...
                "session_trim": {
                    "value": 0
                }
            },
        ...
        }
    ...
    }

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2022-04-08 21:44:54 -04:00