Commit Graph

115975 Commits

Author SHA1 Message Date
Venky Shankar
b0d540e82f doc: cephfs mirroring design document
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-08 05:31:35 -04:00
Jan Fajerski
0cc5604843 ceph-volume: don't exit before empty report can be printed
get_plan() called exit in case of an empty plan. This prevented a report
being printed under these circumstances. Avoid exit in this case. Also
adds tests to ensure an empty report is printed.

Fixes: https://tracker.ceph.com/issues/47760

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-10-08 08:50:59 +02:00
Kefu Chai
718d9c5997
Merge pull request #37561 from tchaikov/wip-cmake-node-mirror
cmake: support Node.js and npm registry mirror

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-10-08 14:50:30 +08:00
Josh Durgin
821b66e67c
Merge pull request #37570 from ceph/wip-install-ceph-deploy
doc: add a link to ceph-deploy document

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-10-07 17:56:22 -07:00
Patrick Donnelly
149a4a6cfd
Merge PR #37446 into master
* refs/pull/37446/head:
	mds: remove dead mdsmap broadcast code

Reviewed-by: Jos Collin <jcollin@redhat.com>
2020-10-07 13:35:18 -07:00
Patrick Donnelly
998149bef5
Merge PR #37497 into master
* refs/pull/37497/head:
	client,mds: Drop execute permission bits

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-07 12:39:04 -07:00
Patrick Donnelly
035ff8b827
Merge PR #37571 into master
* refs/pull/37571/head:
	qa/tasks/cephfs/nfs: Check ganesha daemon status after restarting mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-07 12:38:04 -07:00
Patrick Donnelly
ebdb6da46b
Merge PR #37579 into master
* refs/pull/37579/head:
	SubmittingPatches: use "doc" prefix in title of doc-only commits

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-07 12:29:03 -07:00
Nizamudeen A
6b71e4aaf2 mgr/dashboard: Add short descriptions to the telemetry report preview
A small info icon with a popover on the telemetry report preview form step2 for
Report ID and Report Preview.

Report ID: "A randomized UUID to identify a particular cluster over the course of several telemetry reports."
Report preview: "The actual telemetry data that will be submitted."

Fixes: https://tracker.ceph.com/issues/47610
Signed-off-by: Nizamudeen A <nia@redhat.com>
2020-10-07 23:08:50 +05:30
Varsha Rao
1552f7239c mgr/volumes/nfs: Fix wrong error message for pseudo path
Fixes: https://tracker.ceph.com/issues/47783
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-10-07 22:31:01 +05:30
Jason Dillaman
cb35c6c3bc librbd: avoid failing IO with -ESHUTDOWN when disabling exclusive-lock
When dynamically disabling the exclusive-lock feature with in-flight IO,
it was previously possible for IO to fail with -ESHUTDOWN when it
attempts to acquire the lock due to the PreReleaseRequest setting the
lock required flag. There is also the possibility for a race with the
first IO racing with exclusive-lock shutdown when the lock was not
already acquired.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-10-07 11:04:41 -04:00
J. Eric Ivancich
bd6f163f36 rgw: fix setting of namespace in ordered and unordered bucket listing
The namespace is not always set correctly during bucket listing. This
can, for example, cause the listing of incomplete multipart uploads,
which are in the _multipart_ namespace, to not paginate correctly, and
cause entries to be re-listed.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-10-07 11:02:15 -04:00
Jan Fajerski
cdaaf754a4 Merge PR #37577 into master
* refs/pull/37577/head:
	PendingReleaseNotes: add note about batch refactor

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-10-07 15:19:54 +02:00
J. Eric Ivancich
ddf52016fa rgw: allow rgw-orphan-list to note when rados objects are in namespace
Currently namespaces and locators are ignored when `rados ls` is run
by rgw-orphan-list to record RADOS's known objects.

However there have been cases where RADOS objects have a locator, and
when one is included in the listing, the script does not handle it
correctly. Now when objects have locators, we will prevent their
output from entering the .intermediate file.

Additionally we do not expect RGW data objects to be in RADOS
namespaces, so when a namespaced object is detected, we'll error out
with a message.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-10-07 08:51:50 -04:00
Willem Jan Withagen
cb122ad1e5 include: explicitly define all types needed for libc++ iterator
There is a serious difference between iterator in
    libstdc++ (gcc) versus libc++ (clang)

This generates a rather big and vague set of warnings and notes.
The crux is the warnings about `requirement '!__is_forward_iterator....`
Extra information can also be found on:
    https://www.fluentcpp.com/2018/05/08/std-iterator-deprecated/

```
/home/jenkins/workspace/ceph-master-compile/src/librbd/deep_copy/ObjectCopyRequest.cc:142:15: error: no matching constructor for initialization of 'io::Extents' (aka 'vector<pair<unsigned long, unsigned long> >')
  io::Extents image_extents{read_op.image_interval.begin(),
              ^            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins/workspace/ceph-master-compile/src/librbd/deep_copy/ObjectCopyRequest.cc:769:35: note: in instantiation of member function 'librbd::deep_copy::ObjectCopyRequest<librbd::ImageCtx>::send_read' requested here
template class librbd::deep_copy::ObjectCopyRequest<librbd::ImageCtx>;
                                  ^
/usr/include/c++/v1/vector:516:14: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::size_type' (aka 'unsigned long') for 1st argument
    explicit vector(size_type __n, const allocator_type& __a);
             ^
/usr/include/c++/v1/vector:518:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::size_type' (aka 'unsigned long') for 1st argument
    vector(size_type __n, const value_type& __x);
    ^
/usr/include/c++/v1/vector:559:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'const std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >' for 1st argument
    vector(const vector& __x, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:568:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'initializer_list<std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::value_type>' (aka 'initializer_list<std::__1::pair<unsigned long, unsigned long> >') for 1st argument
    vector(initializer_list<value_type> __il, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:579:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >' for 1st argument
    vector(vector&& __x, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:521:9: note: candidate template ignored: requirement '!__is_forward_iterator<interval_set<unsigned long, std::map>::iterator>::value' was not satisfied [with _InputIterator = interval_set<unsigned long, std::map>::iterator]
        vector(_InputIterator __first,
        ^
/usr/include/c++/v1/vector:536:9: note: candidate template ignored: requirement 'is_constructible<std::__1::pair<unsigned long, unsigned long>, unsigned long &>::value' was not satisfied [with _ForwardIterator = interval_set<unsigned long, std::map>::iterator]
        vector(_ForwardIterator __first,
        ^
/usr/include/c++/v1/vector:502:40: note: candidate constructor not viable: requires single argument '__a', but 2 arguments were provided
    _LIBCPP_INLINE_VISIBILITY explicit vector(const allocator_type& __a)
                                       ^
/usr/include/c++/v1/vector:514:14: note: candidate constructor not viable: requires single argument '__n', but 2 arguments were provided
    explicit vector(size_type __n);
             ^
/usr/include/c++/v1/vector:558:5: note: candidate constructor not viable: requires single argument '__x', but 2 arguments were provided
    vector(const vector& __x);
    ^
/usr/include/c++/v1/vector:565:5: note: candidate constructor not viable: requires single argument '__il', but 2 arguments were provided
    vector(initializer_list<value_type> __il);
    ^
/usr/include/c++/v1/vector:571:5: note: candidate constructor not viable: requires single argument '__x', but 2 arguments were provided
    vector(vector&& __x)
    ^
/usr/include/c++/v1/vector:519:5: note: candidate constructor not viable: requires 3 arguments, but 2 were provided
    vector(size_type __n, const value_type& __x, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:496:5: note: candidate constructor not viable: requires 0 arguments, but 2 were provided
    vector() _NOEXCEPT_(is_nothrow_default_constructible<allocator_type>::value)
    ^
/usr/include/c++/v1/vector:529:9: note: candidate constructor template not viable: requires at least 3 arguments, but 2 were provided
        vector(_InputIterator __first, _InputIterator __last, const allocator_type& __a,
        ^
/usr/include/c++/v1/vector:543:9: note: candidate constructor template not viable: requires at least 3 arguments, but 2 were provided
        vector(_ForwardIterator __first, _ForwardIterator __last, const allocator_type& __a,
        ^
1 error generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2020-10-07 14:02:27 +02:00
Jan Fajerski
485f0d797e PendingReleaseNotes: add note about batch refactor
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-10-07 13:12:14 +02:00
Adam Kupczyk
d302238839 os/bluestore: fix memory accounting in TwoQBufferCacheShard
Variable buffer_bytes has been redefined in TwoQBufferCacheShard,
causing PriCache to see always 0 usage when 2q cache was selected,
as it looks at it through BufferCacheShard::_get_bytes().

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2020-10-07 06:34:04 -04:00
Kefu Chai
38ced60485 SubmittingPatches: use "doc" prefix in title of doc-only commits
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-07 17:43:57 +08:00
Joshua Schmid
99ebc0c458
Merge pull request #37498 from jan--f/cadm-new-batch-report
pybind/mgr/orchestrator: fix preview for new c-v batch output
2020-10-07 09:56:10 +02:00
Mykola Golub
e41e0a1217
Merge pull request #37556 from dillaman/wip-rbd-status
rbd: cleanup code and output formatting in status action

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-10-07 10:42:04 +03:00
Joshua Schmid
6d4896e5da
Merge pull request #37521 from jschmid1/remove_orch_osd_spec_mention
docs: <orch osd spec> is no longer present
2020-10-07 09:37:42 +02:00
Kiefer Chang
381e4ac5ac
qa/suites/rados/cephadm/dashboard: fix creating OSD failure
The iSCSI disks report a larger optimal I/O size, which causes
ceph-volume to miscalculate the extent count. Workaround this by
reporting 0.

Fixes: https://tracker.ceph.com/issues/47742
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-07 14:45:24 +08:00
Kefu Chai
2c6483f1b5
Merge pull request #37574 from tchaikov/wip-crimson-doc
doc/crimson: cleanups

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2020-10-07 13:45:35 +08:00
Kefu Chai
c7da693961 doc/crimson/poseidonstore: reformat
to render it as expected

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-07 12:54:42 +08:00
Kefu Chai
2263511c93 doc: move dev/crimson.rst into dev/crimson/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-07 12:54:42 +08:00
Samuel Just
699c2740fe
Merge pull request #36343 from myoungwon/wip-doc-poseidon-store
doc: introduce PoseidonStore

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-10-06 17:28:43 -07:00
Anthony D'Atri
33931a8330 doc/dev: doc/dev/osd_internals caps, formatting, clarity
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2020-10-06 14:06:46 -07:00
Alfonso Martínez
cdeaea2a8f mgr/dashboard: get rgw daemon zonegroup name from mgr
Fixes: https://tracker.ceph.com/issues/47676
Signed-off-by: Alfonso Martinez <almartin@redhat.com>
2020-10-06 21:52:27 +02:00
Josh Durgin
935359d8b9
Merge pull request #37567 from liewegas/wip-stable-mod
include: ceph_stable_mod in public domain

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-10-06 11:54:04 -07:00
Varsha Rao
97959d249a qa/tasks/cephfs/nfs: Check ganesha daemon status after restarting mgr
Fixes: https://tracker.ceph.com/issues/47591
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-10-06 22:56:05 +05:30
Kefu Chai
fe785597e3
doc: add a link to ceph-deploy document 2020-10-07 00:20:58 +08:00
Kefu Chai
befe3bc77a
Merge pull request #37565 from tchaikov/wip-crimson-cleanup
crimson: do not capture unused variables

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2020-10-06 23:21:51 +08:00
Kefu Chai
55216a78ab crimson/os: mark class with "final" dtor "final"
to silence warning like

scheduler.cc:145:30: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
  ~ClassedOpQueueScheduler() final {};
                             ^
/home/jenkins-build/build/workspace/ceph-perf-crimson/ceph-pr/src/crimson/osd/scheduler/scheduler.cc:52:7: note: mark 'ClassedOpQueueScheduler<T>' as 'final' to silence this warning
class ClassedOpQueueScheduler : public Scheduler {
      ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 22:22:25 +08:00
Kefu Chai
6336f72244 crimson/os: mark dtor of interface class "virtual"
we use a smart_ptr in Seastore to manage the life cycle of
`TransactionManager` which is in turn a subclass
`ExtentCallbackInterface`, so should declare the dtor of latter as
virtual.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 22:22:25 +08:00
Jason Dillaman
00b86644cd rbd: cleanup code and output formatting in status action
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-10-06 09:35:24 -04:00
Sage Weil
fa5d9f8846 include: ceph_stable_mod in public domain
Signed-off-by: Sage Weil <sage@newdream.net>
2020-10-06 08:30:08 -05:00
Kefu Chai
37e7425552
Merge pull request #37395 from tchaikov/wip-doc-crimson
doc/dev/crimson: add instruction on running install-deps.sh for build…

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-10-06 19:15:34 +08:00
Kefu Chai
427b244b84 crimson: do not capture unused variables
also silence warnings like:

Signe-off-by: Kefu Chai <kchai@redhat.cop_btree_node.h:103:36: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
                                  [this, ec, len, &extents] (auto i) {
                                   ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 19:12:33 +08:00
Yuval Lifshitz
0ba5fba942 rgw/kafka: fix potential crash when accessing tags
issues were detected by pvs-studio static analyzer

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2020-10-06 13:41:35 +03:00
Kefu Chai
16ae762eb2 seastar: pick up document fixes
pick up changes to address the split script failure with HTML generated
with the latest pandoc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 17:59:11 +08:00
Kefu Chai
a9ed9b15c2 doc/dev/crimson: add links to seastar documents
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 17:59:11 +08:00
Lenz Grimmer
67efea0e79
Merge pull request #37523 from Exotelis/wip-42404
mgr/dashboard: Show warning when replicated size is 1 

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-10-06 11:11:49 +02:00
Kefu Chai
3c3c2f6ea0 doc/dev/crimson: add instruction on running install-deps.sh for building crimson
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 17:06:35 +08:00
myoungwon oh
9261c747f0 doc: introduce Poseidon store
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Jiwoong Park <jw12.park@samsung.com>
Signed-off-by: Kyuho Son <kyuho.son@samsung.com>
Signed-off-by: Sungkyu Park <sk921.park@samsung.com>
2020-10-06 16:22:53 +09:00
Alfonso Martínez
bc1a33c28e mgr/dashboard: api doc: fix endpoint responses doc generation.
Fixes: https://tracker.ceph.com/issues/47615
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-10-06 09:10:13 +02:00
Kefu Chai
9a4fe29204 cmake: read $NPM_REGISTRY for npm registry mirror
in case the offical npm registry is slow or unreachable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 15:01:11 +08:00
Kefu Chai
13f0667e5a cmake: extract add_npm_options()
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 15:01:11 +08:00
Kefu Chai
67fa1fbd74 cmake: read $NODE_MIRROR for Node.js mirror
in case the offical Node.js https://nodejs.org/dist/ is slow or
unreachable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-06 15:01:11 +08:00
Samuel Just
5d7d30f8e3
Merge pull request #37134 from myoungwon/wip-tier-evict
osd: add tier_evict

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-10-05 23:06:21 -07:00
Kiefer Chang
ee14ef7c60
mgr/dashboard: fix error when typing existing paths in the Ganesha form
- The `CephFS.ls_dir()` implementation in the backend had changed, the code in the
  UI endpoint `/ui-api/nfs-ganesha/lsdir` needs to adapt.
- Add fs_name as resource_id in `/ui-api/nfs-ganesha/lsdir/<fs_name>` to distinguish FS.
- Add more checks and unit tests.

Fixes: https://tracker.ceph.com/issues/47372
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-06 11:25:55 +08:00