Commit Graph

371 Commits

Author SHA1 Message Date
Sage Weil
09ef9b850f mon/ConfigMap: search nested sections
For an entity a.b.c.d, search all dot-delineated prefix sections.  This
enables you to establish a hierarchical set of options for clients, such
as radosgw daemons.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-01 16:51:54 -05:00
Sage Weil
713dbc9722 mgr/telemetry: add stats about crush map
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-01 07:59:53 -05:00
Abhishek Lekshmanan
f62c6e8cba mgr/telemetry: add rgw metadata
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-30 16:56:38 -05:00
Sage Weil
2b7ac440ab mgr/telemetry: include fs size (files, bytes, snaps)
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 14:46:53 -05:00
Sage Weil
f4c7366994 mgr/telemetry: mds cache stats
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 14:46:53 -05:00
Sage Weil
a42dc0a3d0 mgr/telemetry: add some rbd metadata
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 14:46:53 -05:00
Sage Weil
702175f493 mgr/telemetry: note whether osd cluster_network is in use
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 14:25:00 -05:00
Sage Weil
9e6fbaaaa0 mgr/telemetry: add host counts
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 14:25:00 -05:00
Sage Weil
87670fdc3c mgr/telemetry: add more pool metadata
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 14:24:59 -05:00
Sage Weil
3453930d43 mgr/telemetry: include min_mon_release and msgr v1 vs v2 addr count
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 13:38:44 -05:00
Sage Weil
7f6aad677b mgr/telemetry: add CephFS metadata
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-29 13:38:44 -05:00
Sage Weil
29c97547a9 Merge PR #30859 into master
* refs/pull/30859/head:
	auth: EACCES, not EPERM
	mon: shunt old tell commands from cli interface to asok
	mon: allow mgr to tell mon.foo smart
	mon: include quorum features in quorum_status
	qa/workunits/mon/caps.sh: fix test
	ceph_test_rados_api_cmd: fix MonDescribe test
	Merge branch 'vstart-fs-auth' of git://github.com/batrick/ceph into wip-cleanup-mon-asok
	test/pybind/test_ceph_argparse: fix tests
	vstart: add volume client keys to keyring
	vstart: use fs authorize to create master client key
	vstart: redirect some output to stderr
	vstart: output command strings to stderr
	qa/workunits/cephtool/test.sh: fix 'quorum enter' caller
	qa: change mon_status calls to quorum_status or tell commands
	mon: fix 'heap ...' command
	mon: consolidate 'sync force' commands
	mon: allow asok commands to return an error code
	mon: move 'quorum enter|exit' and 'mon_status' to asok
	mon: fix 'smart' asok command
	mon: remove old 'config set' and 'injectargs'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-10-23 21:05:42 -05:00
Gregory Farnum
e190825f26
Merge pull request #30650 from athanatos/sjust/wip-dmclock-server-only
dmclock server side refactor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-10-23 13:35:13 -07:00
Samuel Just
6eaf324083 PendingReleaseNotes: note mClockScheduler config changes
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-10-23 13:33:59 -07:00
Sage Weil
da533bdd4d Merge branch 'vstart-fs-auth' of git://github.com/batrick/ceph into wip-cleanup-mon-asok 2019-10-21 11:43:01 -05:00
Kefu Chai
8ae022a3ce
Merge pull request #30830 from tchaikov/wip/rpm-install-deps/python_provide
install-deps, rpm: use python_provide macro and cleanups

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-10-15 00:05:06 +08:00
Sage Weil
e3260a81d5 mon: consolidate 'sync force' commands
Replace the 'ceph [mon] sync force' commands and just use the asok
sync_force command instead.  This is a low-level command that nobody should
reasonsbly using except in an emergency, so do not bother with trying to
maintain compatibility; it's a bit rediculous that we had 3 variations of
this to being with!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-12 12:05:36 -05:00
Kefu Chai
9ae33df4ca PendingReleaseNotes: revise the note on python3.4 to python3
as RHEL/CentOS 7.7 offers python3 (python3.6), EPEL7 removed the
duplicated python3-* (python36-*) packages, see
https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org/message/GUSLTD5AJWFEGBFK5EYFVH3LHQVHUBAP/

and EPEL7 only supports the latest RHEL7.x release, which is now 7.7

so should not mention EPEL7 in this note,

for the reason why EPEL removed these packages, see EPEL's guideline:
https://fedoraproject.org/wiki/EPEL/GuidelinesAndPolicies#Policy

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-10 11:04:54 +08:00
Sage Weil
c9bfbb7a47 Merge PR #30724 into master
* refs/pull/30724/head:
	mgr/telemetry: bump content revision and add a release note
	telemetry/server: add device report endpoint
	mgr/telemetry: include device telemetry
	mgr/devicehealth: factor _get_device_metrics out of show_device_metrics
	mgr/devicehealth: pull out MAX_SAMPLES

Reviewed-by: Dan Mick <dmick@redhat.com>
2019-10-09 21:30:44 -05:00
Sage Weil
47d3406fb2 mgr/telemetry: bump content revision and add a release note
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-09 21:30:13 -05:00
Sage Weil
bbc7bb5a22 Merge PR #30217 into master
* refs/pull/30217/head:
	crimson: common/admin_socket kludge so that it builds
	mon/MonClient: fix sending mon command to a specific rank
	src/.gitignore: ignore .tox
	mon/MonClient: interpret numeric mon target name as rank
	mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
	qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
	common/ceph_context: make 'config unset' idempotent
	qa/tasks/dump_stuck: mon.a, not mon.0
	qa/suites/rados/singleton/all/admin-socket: fix test
	common/config: EPERM setting config option after startup
	qa/workunits/cephtool/test.sh: fix tell output error check
	common/admin_socket: pass Formatter from generic infrastructure
	common/admin_socket: pass ostream to call() for error output
	os/bluestore: fix asok hook return value
	rgw: fix asok return value
	common/ceph_context: return error code from asok commands
	test/pybind/test_rados: fix accidental mon tell test
	mon: print entity_name along with caps to debug log
	PendingReleaseNotes: notes about asok changes
	mgr/MgrClient: empty target string for 'tell' means active mgr
	common/admin_socket: report error code as part of output string
	osd: change trigger_[deep_]scrub tommands to a pg tell command
	osd: remove old command workqueue, threadpool
	osd: drop MMonCommand handling
	osdc/Objecter: resend OSD tell commands on EAGAIN
	osd: route tell commands to asok; migrate commands
	osd: use unique_ptr<Formatter> for asok_command
	common/ceph_context: add generic asok 'injectargs'
	common/admin_socket: allow dup prefixes
	common/admin_socket: refactor with sync and async execute_command variants
	common/admin_socket: pass input bufferlist
	osd: transition to call_async() for asok
	common/admin_socket: support alternative call_async()
	mon/MonClient: send tell commands out of band via MCommand
	mon: accept tell commands via MCommand and send them to asok handler
	common/admin_socket: return int from hook call()
	mgr/DaemonServer: route MCommand (for octopus+) to asok commands
	do not use 'ceph tell mgr'
	pybind/ceph_argparse: disambiguate mgr tell and CLI commands
	ceph: make 'ceph tell mgr.*' send to the active mgr
	ceph: send 'ceph tell mgr.X' to the right mgr
	librados: add rados_mgr_command_target
	mgr/MgrClient: add start_command variant that takes a target
	common/admin_socket: drop unregister_command(); use per-hook variant
	common/admin_socket: drop explicit prefix arg to register_command
	common/admin_socket: simplify command routing
	common/admin_socket: add ability to process MCommand via asok queue
	common/admin_socket: pass cmdvec to execute_command
	common/admin_socket: use pipe for general wakeup
	include/compat: add flags arg to pipe_cloexec
	common/admin_socket: drop unused args

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-06 09:08:28 -05:00
Sage Weil
5d4fe85b74 PendingReleaseNotes: notes about asok changes
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:02 -05:00
Kefu Chai
86b9036c6c
Merge pull request #30614 from tchaikov/wip-no-more-osdmap-full
librados,osd,mon: remove traces of CEPH_OSDMAP_FULL

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-10-04 10:27:21 +08:00
Kefu Chai
a532aaf479 PendingReleaseNotes: document full/nearfull changes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-04 10:26:08 +08:00
Xie Xingguo
f28f35f79b
Merge pull request #30591 from xiexingguo/wip-balancer-throttler
mgr/balancer: upmap_max_iterations -> upmap_max_optimizations; behave as it is per pool

Reviewed-by: Sage Weil <sage@redhat.com>
2019-10-01 07:42:25 +08:00
xie xingguo
d7ea56f3df mgr/balancer: upmap_max_iterations -> upmap_max_optimizations
With osd_calc_pg_upmaps_aggressively on we might have to iterate
hundreds or thousands of times to figure out an optimization,
but upmap_max_optimizations will remain a hard limit for the
total optimizations that can be returned.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-28 08:31:45 +08:00
Sage Weil
dff5697464 Merge PR #30525 into master
* refs/pull/30525/head:
	qa/tasks/ceph.conf.template: disable power-of-2 warning
	qa/standalone/mon/health-mute: use power of 2 for pg_num
	osd/OSDMap: remove remaining g_conf() usage
	PendingReleaseNotes: add note for 14.2.5 so we can backport this
	osd/OSDMap: health alert for non-power-of-two pg_num

Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-27 12:08:37 -05:00
Sage Weil
f1f8e9a520 PendingReleaseNotes: add note for 14.2.5 so we can backport this
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 09:26:45 -05:00
Sage Weil
6e46b1c0e5 osd/OSDMap: health alert for non-power-of-two pg_num
Fixes: https://tracker.ceph.com/issues/41647
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 09:26:33 -05:00
Patrick Donnelly
30909f5a6a
Merge PR #29824 into master
* refs/pull/29824/head:
	qa: whitelist new FS_INLINE_DATA_DEPRECATED health warning
	mds: add a HEALTH_WARN message when inline_data is enabled
	mds: log a warning message when mds is started on an fs with inline_data
	mon: deprecate CephFS inline_data support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2019-09-24 04:32:28 -07:00
Sage Weil
e1569a18c3 common: default pg_autoscale_mode=on for new pools
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-22 16:58:33 -05:00
Jeff Layton
9c406d0ab3 mon: deprecate CephFS inline_data support
The plan is to start deprecating this feature now so that we can remove
it in a future release. Change it to require the
--yes-i-really-really-mean-it flag, and to emit a custom
warning when that isn't specified.

For now, we leave the testing in place since we do want to be notified
if something breaks before we're ready to rip it out completely.

Fixes: https://tracker.ceph.com/issues/41311
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-19 09:15:13 -04:00
David Zafman
dd2782c15a Revert "common: default pg_autoscale_mode=on for new pools"
This reverts commit 91e4fc24e7.

Fixes: https://tracker.ceph.com/issues/41900

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-09-17 14:51:08 -07:00
Kefu Chai
7c1b53dc49
Merge pull request #30350 from liewegas/wip-bluestore-poolstats-configs
os/bluestore: simplify per-pool-stat config options

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-09-16 19:37:27 +08:00
Sage Weil
7e96024254 os/bluestore: simplify per-pool-stat config options
The previous bluestore_no_per_pool_stats_tolerance had a lot of possible
values, not all of which make sense to users.  Replace with a single new
option, bluestore_fsck_error_on_no_per_pool_stats, which controls whether
the lack of per-pool stats is an error or a warning.  On repair, we will
unconditionally convert to per-pool stats.

This brings us in sync with the newer
bluestore_fsck_error_on_no_per_pool_omap.

Note that one part of the ceph_test_objectstore test is dropped since it
is no longer possible to create a store with legacy stats.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-12 13:25:46 -05:00
Sage Weil
91e4fc24e7 common: default pg_autoscale_mode=on for new pools
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-12 10:42:36 -05:00
David Zafman
b3e1c58b0e osd: Replace active/pending scrub tracking for local/remote
This is similar to how recovery reservations are split between
local and remote.

It was the case that scrubs_pending was used for reservations at
the replicas as well as at the primary while requesting reservations
from the replicas.  There was no need for scrubs_pending to turn
into scrubs_active at the primary as nothing treated that value
as special.  scrubber.active = true when scrubbing is
actually going.

Now scurbber.local_reserved indicates scrubs_local incremented
Now scrubber.remote_reserved indicates scrubs_remote incremented

Fixes: https://tracker.ceph.com/issues/41669

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-09-10 13:33:27 -07:00
David Zafman
8dffd68365 osd: Add dump_scrub_reservations to show scrub reserve tracking
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-09-10 13:32:29 -07:00
David Zafman
b98950e707 osd: Rename dump_reservations to dump_recovery_reservations
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-09-10 13:32:29 -07:00
Yan Jun
aed94cf203 PendingReleaseNotes: note about changes in ceph osd erasure-code-profile set
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2019-09-09 20:02:08 +08:00
David Zafman
5f83a6158b osd doc mon mgr: To milliseconds for config value, user input and threshold out
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-09-04 17:13:32 +00:00
David Zafman
f4a0be2e87 doc: Add documentation and release notes
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-08-26 15:25:34 +00:00
Casey Bodley
f0575a7144
Merge pull request #26787 from soumyakoduri/bucket_name_validation
[rgw]:Validate bucket names as per revised s3 spec

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-16 10:53:54 -04:00
Soumya Koduri
eb6eddbe8d Validate bucket names as per revised s3 spec
As per amazon s3 spec -
https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

* The s3 bucket names should not contain upper case letters or underscore.
* Name cannot end with dash or have consecutive periods, or dashes adjacent
  to periods.
* Each label in the bucket name must start and end with a lowercase
  letter or a number.
* Name cannot exceed 63 characters.

This change is to enforce these rules if rgw_relaxed_s3_bucket_names is set to
'false' which is by default.

Fixes: https://tracker.ceph.com/issues/36293

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2019-08-08 16:54:12 +05:30
Sage Weil
3e7c185bd4 mon: make mon summary more concise in 'ceph -s'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-05 22:02:31 -05:00
Sage Weil
9e0916160d mon/MgrMap: make print_summary (used by 'ceph -s') more concise
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-05 09:20:12 -05:00
Joao Eduardo Luis
ff080391c1 PendingReleaseNotes: add ceph osd info
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-07-15 17:13:37 +00:00
Sage Weil
35c0d75888 osd: add hdd and ssd variants for osd_recovery_max_active
Semi-arbitrarily set the SSD max to 10 (instead of 3).  This should be
tuned based on some real data.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-06-20 16:24:51 -05:00
Sage Weil
1f73631e75 PendingReleaseNotes: note about change to ISO 8601 throughout
Signed-off-by: Sage Weil <sage@redhat.com>
2019-05-29 14:12:15 -05:00
Dmitriy Rabotjagov
c336882cf6 mgr/zabbix: Fix raw_bytes_used key name
This patch fixes raw_bytes_used key which was renamed to stored_raw.
Also added key percent_used and fixed zabbix template to be fully
compatible with zabbix 3.0

Fixes: https://tracker.ceph.com/issues/39644
Signed-off-by: Dmitriy Rabotjagov <noonedeadpunk@ya.ru>
2019-05-28 13:10:49 +03:00