Make dumping of reservation info congruent between scrub and recovery
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Describe how to install teuthology, use vstart_runner.py to run tests
locally and briefly describe internals of vstart_runner.py
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This is similar to how recovery reservations are split between
local and remote.
It was the case that scrubs_pending was used for reservations at
the replicas as well as at the primary while requesting reservations
from the replicas. There was no need for scrubs_pending to turn
into scrubs_active at the primary as nothing treated that value
as special. scrubber.active = true when scrubbing is
actually going.
Now scurbber.local_reserved indicates scrubs_local incremented
Now scrubber.remote_reserved indicates scrubs_remote incremented
Fixes: https://tracker.ceph.com/issues/41669
Signed-off-by: David Zafman <dzafman@redhat.com>
feature: Health warnings on long network ping times, add "dump_osd_network" to get a report
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Executed ceph-conf --dump-all on a freshly installed v14.2.2 (nautilus)
cluster. Compared the global defaults to the keys/values specified in
mon-config-ref.rst. Checked options.cc to make sure the obsolete keys
are no longer used.
Fixes: https://tracker.ceph.com/issues/41516
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
Signed-off-by: Myna <mynaramana@gmail.com>
(cherry picked from commit a20ba26721)
Note: This documentation fix was merged to nautilus via
https://github.com/ceph/ceph/pull/29191 without being merged to master, first.
This commit forward-ports the fix to master.
* refs/pull/28855/head:
doc: document scrub summary in ceph status output
test: extend scrub control test to validate mds task status
mds: send scrub state changes to cluster log.
mds: periodically sent mds scrub status to ceph manager
mgr, mon: allow normal ceph services to register with manager
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Currently dashboard provides a Ceph command to specify location or
contents of SAML2 IdP XML (`idp_metadata` argument). This loose
interface is implemented by trying to:
- First, opens HTTPS connection to whatever that argument contains (it
might be a proper remote URL, a local file or XML contents).
- Then, tries to open the local file
- Finally, assumes the input argument is an XML and proceeds to parse
it.
However, as the XML can have an undefined length, when fed as a filename
it results in FreeBSD raising a OSError exception (`Max filename length
exceeded`, 1K). This essentially means that this handling results in
unexpected behaviour, as it pushes the validation & error handling to
the underlying methods.
In this fix, some preliminary validation is performed. Especifically:
- Is the input argument a potential filename?
- Is the input argument complying with URL syntax?
Only if the above checks fail, the input argument is fed into the XML
parser.
Additionally, previous syntax is deprecated, so now, `idp_metadata`
enforces 2 syntaxes:
- Raw XML contents
- URL specification (http, https, and file schemas accepted). For local
file, URL 'file://<path>' should be used instead.
Fixes: https://tracker.ceph.com/issues/41358
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
osd: scrub error on big objects; make bluestore refuse to start on big objects
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
I think someday the docs for how health alerts work (here) and the
enumeration of all actual alerts should be restructured. For now this
si the simplest placde to fit this!
Signed-off-by: Sage Weil <sage@redhat.com>t
* refs/pull/29292/head:
os/bluestore: warn on no per-pool omap
os/bluestore: fsck: warning (not error) by default on no per-pool omap
os/bluestore: fsck: int64_t for error count
os/bluestore: default size of 1 TB for testing
os/bluestore: behave if we *do* set PGMETA and PERPOOL flags
os/bluestore: do not set both PGMETA_OMAP and PERPOOL_OMAP
os/bluestore: fsck: only generate 1 error per omap_head
os/bluestore: make fsck repair convert to per-pool omap
os/bluestore: teach fsck to tolerate per-pool omap
os/bluestore: ondisk format change to 3 for per-pool omap
mon/PGMap: add data/omap breakouts for 'df detail' view
osd/osd_types: separate get_{user,allocated}_bytes() into data and omap variants
mon/PGMap: fix stored_raw calculation
mon/PGMap: add in actual omap usage into per-pool stats
osd: report per-pool omap support via store_statfs_t
os/bluestore: set per_pool_omap key on mkfs
osd/osd_types: count per-pool omap capable OSDs
os/bluestore: report omap_allocated per-pool
os/bluestore: add pool prefix to omap keys
kv/KeyValueDB: take key_prefix for estimate_prefix_size()
os/bluestore: fix manual omap key manipulation to use Onode::get_omap_key()
os/bluestore: make omap key helpers Onode methods
os/bluestore: add Onode::get_omap_prefix() helper
os/bluestore: change _do_omap_clear() args
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
As per amazon s3 spec -
https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
* The s3 bucket names should not contain upper case letters or underscore.
* Name cannot end with dash or have consecutive periods, or dashes adjacent
to periods.
* Each label in the bucket name must start and end with a lowercase
letter or a number.
* Name cannot exceed 63 characters.
This change is to enforce these rules if rgw_relaxed_s3_bucket_names is set to
'false' which is by default.
Fixes: https://tracker.ceph.com/issues/36293
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
rgw : Bucket mv, bucket chown and user rename utilities
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
* refs/pull/29337/head:
mon: enable telemetry module by default
mgr/telemetry: force re-opt-in if the report contents change
mgr/telemetry: less noise in the log
mgr/telemetry: wake up serve on config change
mgr/telemetry: track telemetry report revisions
Reviewed-by: Neha Ojha <nojha@redhat.com>
* refs/pull/29492/head:
doc: Fix for new ceph-devel mailing list
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
doc: pg_num should always be a power of two
Reviewed-By: Jan Fajerski <jfajerski@suse.com>
Reviewed-By: Sage Weil <sage@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
We already have a robust set of opt-in and opt-out procedures; no need to
require the user to enable the module.
Signed-off-by: Sage Weil <sage@redhat.com>
* refs/pull/29334/head:
qa/tasks/mgr/dashboard/test_mgr_module: adjust expected schema
mgr/telemetry: separate out cluster config vs running daemons
mgr/telemetry: include any config options that are customized
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
Split *_stateless_service, because every orchestrator
did some method dispatching internally anyway.
Also: Added test
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
existing user are moved under the new user namespace specified by "--new-uid".
It calls bucket link and bucket chown to link the buckets and objects to the new
user namespace. Access and secret keys of the user(and the subusers) are preserved.
Usage: "radosgw-admin user rename --uid=<> --new-uid=<>"
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Improve and add to documentation for "bucket move" functionality;
including use moving to multi-tenancy and further deprecating bucket-id
which is no longer necessary.
Fixes: http://tracker.ceph.com/issues/35885
Signed-off-by: Marcus Watts <mwatts@redhat.com>
This just adds the command line option and related bits.
Underlying functionality will be a later commit.
Fixes: http://tracker.ceph.com/issues/35885
Signed-off-by: Marcus Watts <mwatts@redhat.com>
In jewel, "rgw keystone implicit tenants" only applied to swift. As of
luminous), this option applies to s3 also.
Sites that used this feature with jewel now have outstanding data that
depends on the old behavior.
The fix here is to expand "rgw keystone implicit tenants" so that it
can be set to any of "none", "all", "s3" or "swift" (also 0=false=none,
1=true=all). When set to "s3" or "swift", the actual id lookup
is also partitioned.
Formerly "rgw keystone implicit tenants" was a legacy opt.
This change converts it to the new style of option,
including support for dynamically changing it.
Fixes: http://tracker.ceph.com/issues/24348
Signed-off-by: Marcus Watts <mwatts@redhat.com>
python-common: Python common package
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
like
```
ceph osd pool set <pool-name> crush_rule <rule-name>
```
where `<rule-name>` is a string instead of a number.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/28847/head:
doc/mgr/telemetry: update
mgr/telemetry: move contact info to an 'ident' channel
mgr/telemetry: accept channel list to 'telemetry show'
mgr/telemetry: always generate new report for 'telemetry show'
mgr/telemetry: add 'device' channel and call out to devicehealth module
mgr/telemetry: add telemetry channel 'device'
mgr/telemetry: add separate channels
Reviewed-by: Dan Mick <dmick@redhat.com>
* refs/pull/29034/head:
doc/mgr/crash: document missing commands, options
qa/suites/rados/singleton/all/test-crash: whitelist RECENT_CRASH
qa/suites/rados/mgr/tasks/insights: whitelist RECENT_CRASH
qa/tasks/mgr/test_insights: crash module now rejects bad crash reports
mgr/telemetry: fix remote into crash do_ls()
mgr/crash: don't make these methods static
mgr/BaseMgrModule: handle unicode health detail strings
mgr/crash: verify timestamp is valid
qa/suites/mgr: whitelist RECENT_CRASH
mgr/crash: remove unused var
mgr/crash: remove unused import 'six'
qa/workunits/rados/test_crash: health check
mgr/crash: improve validation on post
mgr/crash: automatically prune old crashes after a year
mgr/crash: raise RECENT_CRASH warning for recent (new) crashes
mgr/crash: add 'crash ls-new'
mgr/crash: add option and serve infra
mgr/crash: keep copy of crashes in memory
mgr/pg_autoscaler: adjust style to match built-in tables
mgr/crash: make 'crash ls' a nice table with a NEW column
mgr/crash: nicely format 'crash info' output
mgr/crash: add 'crash archive <id>', 'crash archive-all' commands
Reviewed-by: Neha Ojha <nojha@redhat.com>
1. Disable redirection on standby managers. A HTTP error (500) will be returned instead of a redirection.
$ ceph config set mgr mgr/dashboard/standby_behaviour "error"
2. Configure the HTTP error status code.
$ ceph config set mgr mgr/dashboard/standby_error_status_code 503
Signed-off-by: Volker Theile <vtheile@suse.com>
The ceph-release-notes script renders strings as bytes and these get exported
as b'<str>'. . . the script needs some more work to play nicely with Python 3.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
* we've moved to cmake 3.5, so no need to hack for newer cmake
* nss dependency is dropped, no need to install it anymore
* pass clang and clang++ using cmake variable to be more consistent
* pass CMAKE_EXE_LINKER_FLAGS as `/usr/local/opt/llvm/lib` is
not in default library paths.
* mention https://github.com/boostorg/atomic/issues/15
Signed-off-by: Kefu Chai <kchai@redhat.com>
Clean up the caching sections to match the current implementation and
clarify which sections apply only to librbd.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
* refs/pull/29011/head:
doc/releases/nautilus: move note to 14.2.2
doc/releases/nautilus: typo
doc/releases/nautilus.rst: pool stats fix was merged for 14.2.2 after all
doc: release notes for nautilus 14.2.2
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Introduced in 4872cc5aa3
`_ceph_set_module_option` also accepts `None`, not just strings.
Fixes: http://tracker.ceph.com/issues/40779
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Include hardware details and update language for modern tools.
Fixes: http://tracker.ceph.com/issues/39620
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Current extensible tier flush a chunked object when all chunks mark as dirty.
So, to make the state of object consistent, tier-flush operation is needed.
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
* Updated troubleshooting.rst as kernel-client-troubleshooting.rst
is dropped.
* Added troubleshooting references in fuse.rst and kernel.rst
Signed-off-by: Jos Collin <jcollin@redhat.com>
* refs/pull/28643/head:
doc: removed old EOL Ceph versions from the list
doc: added openSUSE distros to the list and updated the releases
doc: updated the OS recommendations to reflect the Nautilus release
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Removed references to older OpenStack releases. Updated
documentation in regards to using QCOW2 as a virtual machine
disk type.
Fixes: https://tracker.ceph.com/issues/18197
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
* to reference the script to generate corpus
* use qa/workunits/rgw/run-s3tests.sh for producing rgw corpus
Signed-off-by: Kefu Chai <kchai@redhat.com>
The S3 action is case-sensitive and was fixed in #21916, but there were
two more occurrences left.
Signed-off-by: Thomas Kriechbaumer <thomas@kriechbaumer.name>
Now you can silence alerts through the dashboard. You can now create,
recreate, edit and expire a silence. You can create a silence based on a
selected alert.
The silence form will help you create a silence that silences an alert.
It is provided with functionality to check if the silences, that
you are about to create, will or will not match an active alert or even
a rule.
It also provides help choosing the right values for the right chosen
matcher attribute name, through the use of type ahead values.
The dashboard will now use the Prometheus and the Alertmanager API
Fixes: https://tracker.ceph.com/issues/36722
Signed-off-by: Stephan Müller <smueller@suse.com>
doc/rados/configuration: update to be in sync with ConfUtils changes
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
* refs/pull/28864/head:
doc/releases/releases.yaml: set target EOL to Jun 1 for L and M
doc/releases: update schedule to describe a 12-month cycle
Reviewed-by: Noah Watkins <nwatkins@redhat.com>
as crimson's logging levels are now connected those of ceph, there is no
need to note down the difference between them anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
This commit makes chunk_scrub as a chunk_pool-based operation.
chunk_scrub procedure:
1. find references in the chunk_pool
2. check whether the reference exists in a base_pool
3. fix references
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
The lio setup and teardown functionality was moved from rbd-target-gw to
rbd-target-api in ceph-iscsi.
Signed-off-by: Mike Christie <mchristi@redhat.com>
The ceph-ansible ceph-iscsi support does not allow setting up iscsi
objects. It must be done through gwcli like is done in the container
case. This drops the iscsi object related settings and admin info for
it.
Signed-off-by: Mike Christie <mchristi@redhat.com>
Use the more standard --limit iscsigws argument instead of running the
playbook command on the iscsigws.yml file directly.
Signed-off-by: Mike Christie <mchristi@redhat.com>
osd: add hdd and ssd variants for osd_recovery_max_active
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
mon: Improve health status for backfill_toofull and recovery_toofull
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Neha Ojha <nojha@redhat.com>
rgw: add S3 object lock feature to support object worm
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Treat backfull_toofull as a warning condition because it can resolve itself.
Includes test case for PG_BACKFILL_FULL
Includes test case for recovery_toofull / PG_RECOVERY_FULL
Fixes: https://tracker.ceph.com/issues/39555
Signed-off-by: David Zafman <dzafman@redhat.com>
rgw: update the "radosgw-admin reshard status"
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
