Commit Graph

115902 Commits

Author SHA1 Message Date
Patrick Donnelly
daa2f85156
Merge PR #29951 into master
* refs/pull/29951/head:
	test: add tests for validating MDS metrics via `perf stats` module
	test: Filesystem class helpers to grow and shrink MDS cluster
	mgr/stats: mds performance stats module
	mds: support sending empty perf metrics to ceph-manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-12 18:50:33 -07:00
zdover23
0f3bce753a
Merge pull request #37541 from anthonyeleven/osd-internals-tweaks
doc/dev: doc/dev/osd_internals capitalization, formatting, clarity

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-10-13 06:01:05 +10:00
Patrick Donnelly
473d8e9a60
Merge PR #37593 into master
* refs/pull/37593/head:
	qa/tasks/cephfs: Refactor test_volumes

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-12 07:42:13 -07:00
Tatjana Dehler
1edaa17181
Merge pull request #37154 from bk201/wip-47397
mgr/dashboard: fix the error when exporting CephFS path "/" in NFS exports

Reviewed-by: Laura Paduano lpaduano@suse.com
Reviewed-by: Stephan Müller smueller@suse.com
Reviewed-by: Varsha Rao varao@redhat.com
2020-10-12 14:32:39 +02:00
Venky Shankar
a95b364e3e test: add tests for validating MDS metrics via perf stats module
Fixes: http://tracker.ceph.com/issues/24285
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Venky Shankar
5afdf26943 test: Filesystem class helpers to grow and shrink MDS cluster
These routines were test specific. Make them a part of Filesystem
class.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Venky Shankar
7523aef6e8 mgr/stats: mds performance stats module
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Venky Shankar
e739b831b7 mds: support sending empty perf metrics to ceph-manager
Right now, there are no per-mds metrics that are tracked and
sent by mds. However, such metrics will get added soon. So,
send empty performance metrics to ceph-manager for now.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Ramana Raja
d1d5d61884 qa/tasks/cephfs: Refactor test_volumes
... into smaller test classes. This enables breaking up the volumes yaml
into smaller yaml fragments.

Fixes: https://tracker.ceph.com/issues/47160
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-10-12 13:54:40 +05:30
Kefu Chai
130c2715d9
Merge pull request #37628 from changchengx/mail_update
mailmap: update Intel employee mail/org

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-12 15:43:30 +08:00
Changcheng Liu
1e8c30a1e1 mailmap: update Intel employee mail/org
1. "changcheng.liu@aliyun.com" need be classified into intel until now.
   This reverts part of commit: df07e9f3
2. add "Yuan Lu <yuan.y.lu@intel.com>" in mailmap

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-12 10:10:06 +08:00
Matt Benjamin
118c5ee78b
Merge pull request #36942 from rosinL/wip-fix-librgw
rgw/rgw_file: Fix the incorrect lru object eviction
2020-10-11 11:14:06 -04:00
Kefu Chai
63b40dbc5b
Merge pull request #37580 from aclamk/wip-bluestore-fix-2q-cache
os/bluestore: fix memory accounting in TwoQBufferCacheShard

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-10-10 19:42:11 +08:00
Kefu Chai
774653ab60
Merge pull request #37325 from badone/wip-angular-cli-analytics-dont-prompt
mgr/dashboard: Don't prompt for analytics during @angular/cli install

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-10-10 19:41:07 +08:00
Kefu Chai
7341f96808
Merge pull request #37542 from SUSE/wip-fix-47745
cephadm: allow uid/gid == 0 in copy_tree, copy_files, move_files

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-10-10 19:38:33 +08:00
Kefu Chai
72d2eb7d32
Merge pull request #37559 from wjwithagen/wjw-fix-iterator
include: explicitly define all types needed for libc++ iterator

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-10 19:37:16 +08:00
Kefu Chai
d3f1b1988a
Merge pull request #37104 from sebastian-philipp/cephadm-serve.py-2
mgr/cephadm: move functions that do remote calls to serve.py

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-10-10 19:35:48 +08:00
Patrick Donnelly
07ea8f5bca
Merge PR #37441 into master
* refs/pull/37441/head:
	qa: fix proc exit status check

Reviewed-by: Jos Collin <jcollin@redhat.com>
2020-10-09 18:41:24 -07:00
Lenz Grimmer
a333b8db2c
Merge pull request #36979 from tspmelo/wip-npm-update-20-8
mgr/dashboard: Update npm packages

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-10-09 15:56:00 +02:00
luo rixin
f209733872 rgw/rgw_file: Fix the incorrect lru object eviction
In func lookup_fh, when RGWFileHandle not be found in fh_cache, it
need to recycle an object and create an new RGWFileHandle. When there
are multi threads use lookup_fh to find and create RGWFileHandle concurrently,
it must to make sure evict lru object from the partiton of fh_cache which new
RGWFileHandle will be inserted to.

Fixes: https://tracker.ceph.com/issues/47235

Signed-off-by: luo rixin <luorixin@huawei.com>
2020-10-09 20:44:53 +08:00
Lenz Grimmer
6fadf9e147
Merge pull request #37449 from ceph/47676-zone-empty-realm-id
mgr/dashboard: get rgw daemon zonegroup name from mgr

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-10-09 14:32:10 +02:00
Lenz Grimmer
661aa031ee
Merge pull request #37375 from ceph/47615-fix-endpoint-responses
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-10-09 13:43:52 +02:00
Lenz Grimmer
16e8df214c
Merge pull request #37064 from bk201/wip-47372
mgr/dashboard: fix error when typing existing paths in the Ganesha form

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-10-09 13:42:30 +02:00
Lenz Grimmer
b1a0cad22a
Merge pull request #37596 from rhcs-dashboard/telemetry_link_cleanup
mgr/dashboard: Improve HTML formatting of the telemetry report preview

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-10-09 13:40:32 +02:00
Tiago Melo
ad4bbcd32a mgr/dashboard: Update npm packages
Angular 10.1 includes extraction of TS translations,
so @locl/cli is no longer needed.

Updating Angular automatically removed the usage of base tsconfig.

Downgraded css-loader to be able to use css modules.

angular-tree-component has been deprecated and
replaced by @circlon/angular-tree-component

Fixes: https://tracker.ceph.com/issues/47289

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-10-09 10:07:11 +00:00
Jan Fajerski
65eafe0b54 Merge PR #32778 into master
* refs/pull/32778/head:
	msg: Use sockets instead of pipes for wakeup events on win32
	common: Convert remote errors on win32
	common,msg: Initialize Windows WSA and TLS
	common,os,kv: Define aligned_free
	common: avoid CLOCK_*_COARSE warnings on win32
	global: Windows support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-10-09 11:22:01 +02:00
Patrick Donnelly
2cb09bd4bd
Merge PR #36554 into master
* refs/pull/36554/head:
	mgr/volumes: Make number of cloner threads configurable

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Shyamsundar R <srangana@redhat.com>
2020-10-08 13:25:15 -07:00
Patrick Donnelly
5c9f77cb65
Merge PR #36457 into master
* refs/pull/36457/head:
	qa/tasks/cephfs: fix fs authorize cmd args
	mds: add root_squash mode in MDS auth caps
	vstart_runner: add write_file and sudo_write_file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-08 13:23:34 -07:00
Patrick Donnelly
eef744a88f
Merge PR #36556 into master
* refs/pull/36556/head:
	doc: cephfs mirroring design document

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-08 12:43:39 -07:00
Neha Ojha
343107766e
Merge pull request #37483 from dzafman/wip-46405
osd/osd-rep-recov-eio.sh: TEST_rados_repair_warning:  return 1

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-10-08 11:44:00 -07:00
Nathan Cutler
6d1f1f63b7
Merge pull request #37500 from ricardoasmarques/fix-bootstrap-container-init
cephadm: Fix error setting 'mgr/cephadm/container_init' config

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-10-08 20:04:02 +02:00
J. Eric Ivancich
f0e539b304
Merge pull request #37582 from ivancich/wip-fix-bucket-list-namespace
rgw: fix setting of namespace in ordered and unordered bucket listing

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-08 13:01:36 -04:00
J. Eric Ivancich
3f522f0712
Merge pull request #37572 from ivancich/wip-orphan-list-namespace-locator
rgw: allow rgw-orphan-list to note when rados objects are in namespace

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-08 13:00:51 -04:00
J. Eric Ivancich
de6ae6f6cc
Merge pull request #37511 from ivancich/wip-paginate-rgw-bucket-list
rgw: radosgw-admin should paginate internally when listing bucket

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-10-08 12:59:55 -04:00
Nizamudeen A
bd067a6038 mgr/dashboard: Improve HTML formatting of the telemetry report preview
Convert the string "Community Data License Agreement - Sharing - Version 1.0" to a clickable link

Fixes: https://tracker.ceph.com/issues/47790
Signed-off-by: Nizamudeen A <nia@redhat.com>
2020-10-08 17:20:34 +05:30
Lenz Grimmer
98a8f533b8
Merge pull request #37584 from rhcs-dashboard/report-preview-helper
mgr/dashboard: Add short descriptions to the telemetry report preview

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-10-08 13:15:58 +02:00
Jan Fajerski
f68197eca4 Merge PR #37591 into master
* refs/pull/37591/head:
	ceph-volume: don't exit before empty report can be printed

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-10-08 11:52:56 +02:00
Venky Shankar
b0d540e82f doc: cephfs mirroring design document
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-08 05:31:35 -04:00
Jan Fajerski
0cc5604843 ceph-volume: don't exit before empty report can be printed
get_plan() called exit in case of an empty plan. This prevented a report
being printed under these circumstances. Avoid exit in this case. Also
adds tests to ensure an empty report is printed.

Fixes: https://tracker.ceph.com/issues/47760

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-10-08 08:50:59 +02:00
Kefu Chai
718d9c5997
Merge pull request #37561 from tchaikov/wip-cmake-node-mirror
cmake: support Node.js and npm registry mirror

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-10-08 14:50:30 +08:00
Josh Durgin
821b66e67c
Merge pull request #37570 from ceph/wip-install-ceph-deploy
doc: add a link to ceph-deploy document

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-10-07 17:56:22 -07:00
Patrick Donnelly
149a4a6cfd
Merge PR #37446 into master
* refs/pull/37446/head:
	mds: remove dead mdsmap broadcast code

Reviewed-by: Jos Collin <jcollin@redhat.com>
2020-10-07 13:35:18 -07:00
Patrick Donnelly
998149bef5
Merge PR #37497 into master
* refs/pull/37497/head:
	client,mds: Drop execute permission bits

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-07 12:39:04 -07:00
Patrick Donnelly
035ff8b827
Merge PR #37571 into master
* refs/pull/37571/head:
	qa/tasks/cephfs/nfs: Check ganesha daemon status after restarting mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-07 12:38:04 -07:00
Patrick Donnelly
ebdb6da46b
Merge PR #37579 into master
* refs/pull/37579/head:
	SubmittingPatches: use "doc" prefix in title of doc-only commits

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-07 12:29:03 -07:00
Nizamudeen A
6b71e4aaf2 mgr/dashboard: Add short descriptions to the telemetry report preview
A small info icon with a popover on the telemetry report preview form step2 for
Report ID and Report Preview.

Report ID: "A randomized UUID to identify a particular cluster over the course of several telemetry reports."
Report preview: "The actual telemetry data that will be submitted."

Fixes: https://tracker.ceph.com/issues/47610
Signed-off-by: Nizamudeen A <nia@redhat.com>
2020-10-07 23:08:50 +05:30
J. Eric Ivancich
bd6f163f36 rgw: fix setting of namespace in ordered and unordered bucket listing
The namespace is not always set correctly during bucket listing. This
can, for example, cause the listing of incomplete multipart uploads,
which are in the _multipart_ namespace, to not paginate correctly, and
cause entries to be re-listed.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-10-07 11:02:15 -04:00
Jan Fajerski
cdaaf754a4 Merge PR #37577 into master
* refs/pull/37577/head:
	PendingReleaseNotes: add note about batch refactor

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-10-07 15:19:54 +02:00
J. Eric Ivancich
ddf52016fa rgw: allow rgw-orphan-list to note when rados objects are in namespace
Currently namespaces and locators are ignored when `rados ls` is run
by rgw-orphan-list to record RADOS's known objects.

However there have been cases where RADOS objects have a locator, and
when one is included in the listing, the script does not handle it
correctly. Now when objects have locators, we will prevent their
output from entering the .intermediate file.

Additionally we do not expect RGW data objects to be in RADOS
namespaces, so when a namespaced object is detected, we'll error out
with a message.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-10-07 08:51:50 -04:00
Willem Jan Withagen
cb122ad1e5 include: explicitly define all types needed for libc++ iterator
There is a serious difference between iterator in
    libstdc++ (gcc) versus libc++ (clang)

This generates a rather big and vague set of warnings and notes.
The crux is the warnings about `requirement '!__is_forward_iterator....`
Extra information can also be found on:
    https://www.fluentcpp.com/2018/05/08/std-iterator-deprecated/

```
/home/jenkins/workspace/ceph-master-compile/src/librbd/deep_copy/ObjectCopyRequest.cc:142:15: error: no matching constructor for initialization of 'io::Extents' (aka 'vector<pair<unsigned long, unsigned long> >')
  io::Extents image_extents{read_op.image_interval.begin(),
              ^            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins/workspace/ceph-master-compile/src/librbd/deep_copy/ObjectCopyRequest.cc:769:35: note: in instantiation of member function 'librbd::deep_copy::ObjectCopyRequest<librbd::ImageCtx>::send_read' requested here
template class librbd::deep_copy::ObjectCopyRequest<librbd::ImageCtx>;
                                  ^
/usr/include/c++/v1/vector:516:14: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::size_type' (aka 'unsigned long') for 1st argument
    explicit vector(size_type __n, const allocator_type& __a);
             ^
/usr/include/c++/v1/vector:518:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::size_type' (aka 'unsigned long') for 1st argument
    vector(size_type __n, const value_type& __x);
    ^
/usr/include/c++/v1/vector:559:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'const std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >' for 1st argument
    vector(const vector& __x, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:568:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'initializer_list<std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >::value_type>' (aka 'initializer_list<std::__1::pair<unsigned long, unsigned long> >') for 1st argument
    vector(initializer_list<value_type> __il, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:579:5: note: candidate constructor not viable: no known conversion from 'interval_set<unsigned long, std::map>::iterator' to 'std::__1::vector<std::__1::pair<unsigned long, unsigned long>, std::__1::allocator<std::__1::pair<unsigned long, unsigned long> > >' for 1st argument
    vector(vector&& __x, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:521:9: note: candidate template ignored: requirement '!__is_forward_iterator<interval_set<unsigned long, std::map>::iterator>::value' was not satisfied [with _InputIterator = interval_set<unsigned long, std::map>::iterator]
        vector(_InputIterator __first,
        ^
/usr/include/c++/v1/vector:536:9: note: candidate template ignored: requirement 'is_constructible<std::__1::pair<unsigned long, unsigned long>, unsigned long &>::value' was not satisfied [with _ForwardIterator = interval_set<unsigned long, std::map>::iterator]
        vector(_ForwardIterator __first,
        ^
/usr/include/c++/v1/vector:502:40: note: candidate constructor not viable: requires single argument '__a', but 2 arguments were provided
    _LIBCPP_INLINE_VISIBILITY explicit vector(const allocator_type& __a)
                                       ^
/usr/include/c++/v1/vector:514:14: note: candidate constructor not viable: requires single argument '__n', but 2 arguments were provided
    explicit vector(size_type __n);
             ^
/usr/include/c++/v1/vector:558:5: note: candidate constructor not viable: requires single argument '__x', but 2 arguments were provided
    vector(const vector& __x);
    ^
/usr/include/c++/v1/vector:565:5: note: candidate constructor not viable: requires single argument '__il', but 2 arguments were provided
    vector(initializer_list<value_type> __il);
    ^
/usr/include/c++/v1/vector:571:5: note: candidate constructor not viable: requires single argument '__x', but 2 arguments were provided
    vector(vector&& __x)
    ^
/usr/include/c++/v1/vector:519:5: note: candidate constructor not viable: requires 3 arguments, but 2 were provided
    vector(size_type __n, const value_type& __x, const allocator_type& __a);
    ^
/usr/include/c++/v1/vector:496:5: note: candidate constructor not viable: requires 0 arguments, but 2 were provided
    vector() _NOEXCEPT_(is_nothrow_default_constructible<allocator_type>::value)
    ^
/usr/include/c++/v1/vector:529:9: note: candidate constructor template not viable: requires at least 3 arguments, but 2 were provided
        vector(_InputIterator __first, _InputIterator __last, const allocator_type& __a,
        ^
/usr/include/c++/v1/vector:543:9: note: candidate constructor template not viable: requires at least 3 arguments, but 2 were provided
        vector(_ForwardIterator __first, _ForwardIterator __last, const allocator_type& __a,
        ^
1 error generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2020-10-07 14:02:27 +02:00