Commit Graph

101387 Commits

Author SHA1 Message Date
Kefu Chai
35dbbf9a9b
Merge pull request #29854 from tchaikov/wip-sloppy-cond
common/condition_variable_debug: do not assert() if sloppy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2019-08-26 22:38:50 +08:00
Kefu Chai
00ccadff26
Merge pull request #29866 from joscollin/wip-fix-stupid
os/bluestore: silence StupidAllocator reorder warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-26 22:36:41 +08:00
Alfredo Deza
846d6c775a
Merge pull request #29890 from rhcs-dashboard/wip-signed-master
github: display phrase for signed-off check

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-08-26 08:20:57 -04:00
Kefu Chai
a3fdfe6d67
Merge pull request #29831 from tchaikov/wip-python-int
mgr/BaseMgrModule: use PyInt_Check() to compatible with py2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-26 20:06:21 +08:00
Lenz Grimmer
0301fc9061
mgr/dashboard: Edit Pool Placement Group E2E Test (#29779)
mgr/dashboard: Edit Pool Placement Group E2E Test

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-26 11:36:53 +00:00
Lenz Grimmer
fb63142ab0
Merge pull request #29707 from rafaelquint/dashdoc
mgr/dashboard: Extend "Writing End-to-End Tests" section (describe vs it)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-26 11:33:41 +00:00
Ernesto Puerta
dcf40c7d01
github: display phrase for signed-off check
https://github.com/ceph/ceph-build/pull/1379 will introduce a
trigger-phrase for pr-commit check (DCO/signed-off).

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-26 11:16:25 +02:00
Kefu Chai
20bebe242b
Merge pull request #29840 from tchaikov/wip-install-deps.sh-without-python-saml
install-deps.sh,deb,rpm: move python-saml deps into debian/control an…

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-08-26 16:40:02 +08:00
xie xingguo
c165c5286a osd/osd_types: add_next_event - add special handler for lost_revert
For unfound objects, we might append LOST_REVERT log entries,
which shall allow these objects to be reverted to the newest
available version later.
However, we are currently lack of support to rewind the clean_regions
portion too when marking unfound objects as lost with inc-recovery mode
enabled. Hence we must mark these unfound-revert objects as fully dirty
to make sure they can be correctly recovered.

E.g.,:
- primary is pulling object A from replica 1
- object A is corrupted on replica 1
- object A is now unfound
- mark object A as lost, replica 1 will persist a wrong
  missing item for object A..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-26 13:13:27 +08:00
xie xingguo
81c4ed2b1d osd/osd_types: drop 'new_object' from constructor
There is no consumer.
Actually, I think this field is only meaningful to be used
to indicate whether we should initiate an inc-recovery or not.
If not, then we shall fall back to triggering a full-recovery
instead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-26 13:13:27 +08:00
xie xingguo
e70f8ddb5a osd/osd_types: drop 'new_object' from missing.add
because below here we know we'll always mark object as fully dirty.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-26 13:13:27 +08:00
xie xingguo
679f0777a7 osd/osd_types: always call mark_fully_dirty for missing.add
In general we shall build missing set (and hence clean_regions)
based on pg log. However, currently there are still 5 cases we might call
missing.add to add a new pg_missing_item into the missing set
explicitly (or replace an existing pg_missing_item entirely):

1. we explicitly build missing set on startup, in which case
we know we are trying to be compatiable with pre-kraken versions,
so it should be ok for us to disable inc-recovery.

2. we are currently processing authoritative log, and there are
some divergent objects detected. For simplicity (and correctness),
we should disable inc-recovery entirly for these objects.

3. we are re-building missing set, e.g., due to the global
CEPH_OSDMAP_RECOVERY_DELETES policy changing.
In this case we know we are at the end of upgrading from a
pervious version that is lack of CEPH_OSDMAP_RECOVERY_DELETES support.
Hence it should be the recommended option to disable inc-recovery
simultaneously since these objects should be lack of inc-recovery support
too.

4. we are adding or re-adding missing object into primary's missing_loc.
It doesn't matter whether we have a correct clean_regions there
since we never actually refer to that field from missing_loc
when we actually start to perform object recovery later.

5. we are auto-repairing a corrupted object and hence the need of
adding it to the corresponding missing set first, e.g, by leveraging
the existing recovery procedure. In this case, we always disable
inc-recovery to make sure this object can be fully (and correctly)
recovered later.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-26 13:12:55 +08:00
Mykola Golub
732fe579b6
Merge pull request #29808 from dillaman/wip-41356
rbd-mirror: ignore errors relating to parsing the cluster config file

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2019-08-25 20:23:51 +03:00
James McClune
820ada1e58 doc: default values for mon_health_to_clog_* were flipped
On a freshly installed nautilus cluster (i.e. 14.2.2), the default values are:

mon_health_to_clog_interval = 3600
mon_health_to_clog_tick_interval = 60.000000

Fixes: https://tracker.ceph.com/issues/41403
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2019-08-25 11:35:41 -04:00
Kefu Chai
a576f9bbc2
Merge pull request #29651 from rzarzynski/wip-crimson-cls
crimson, osd: add support for Ceph Classes, part 1

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-25 15:25:41 +08:00
Kefu Chai
083c3a5d1f
Merge pull request #29595 from bk201/wip-41151
mgr/test_orchestrator: Allow initializing dummy data

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-08-25 15:13:33 +08:00
Kefu Chai
0d2891a058
Merge pull request #29787 from NancySu05/fix-osd
osd: add log information to record the cause of do_osd_ops failure

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-25 15:11:51 +08:00
Jos Collin
510679f61a
os/bluestore: silence StupidAllocator reorder warning
warning: ‘StupidAllocator::last_alloc’ will be initialized after [-Wreorder]

Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-08-25 10:36:53 +05:30
Patrick Donnelly
24fbb04331
Merge PR #29542 into master
* refs/pull/29542/head:
	mds: trim cache on regular schedule

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-08-24 21:35:14 -07:00
Patrick Donnelly
2db3dfdd8a
Merge PR #29760 into master
* refs/pull/29760/head:
	mgr/volumes: cleanup FS subvolume or subvolume group path
	mgr/volumes: give useful error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-24 21:31:58 -07:00
Patrick Donnelly
7b58274fda
Merge PR #29843 into master
* refs/pull/29843/head:
	qa/tasks: drop object inherit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-24 21:31:27 -07:00
huangjun
01eae69876 tests/osd: fix typo in unittest_osdmap
Signed-off-by: huangjun <huangjun@xsky.com>
2019-08-25 09:57:47 +08:00
Patrick Donnelly
4b72cc60eb
mds: trim cache on regular schedule
Do this outside the standard tick interval as it needs to be driven more
frequently to keep up with client workloads that generate a lot of
capabilities.

Fixes: https://tracker.ceph.com/issues/41141
Fixes: https://tracker.ceph.com/issues/41140
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-24 16:03:46 -07:00
Xie Xingguo
dce9704c28
Merge pull request #29754 from xiexingguo/wip-inc-recovery-3
osd: misc inc-recovery compat fixes 

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-24 13:51:21 +08:00
Kefu Chai
79c0fcf823
Merge pull request #28344 from iotcg/rdma
check rdma configuration and fix some logic problem

Reviewed-by: Roman Penyaev <rpenyaev@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-24 11:37:03 +08:00
Radoslaw Zarzynski
6435c6c278 crimson/osd: dissect pg listing implementation from OpsExecuter.
Additionally improve const-correctness for PGBackend.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:34:00 +02:00
Radoslaw Zarzynski
c67efe955b crimson/osd: do_pgnls_common isn't the member of OpsExecuter anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:34:00 +02:00
Radoslaw Zarzynski
2a827f5e0e crimson/os, crimson/osd: make xattr retrieval const.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:34:00 +02:00
Radoslaw Zarzynski
7232b10920 crimson/os, crimson/osd: make list_objects() const.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:34:00 +02:00
Radoslaw Zarzynski
edbf82e47f crimson/osd: adopt PGLSPlainFilter.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:34:00 +02:00
Radoslaw Zarzynski
9c01ab59be osd: share PGLSPlainFilter with crimson.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:59 +02:00
Radoslaw Zarzynski
2c0f089710 objclass, osd: dissect PGLSFilter into the osd_types.h header.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:59 +02:00
Radoslaw Zarzynski
10fca231e2 crimson/osd: implement PGLSFilter.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:59 +02:00
Radoslaw Zarzynski
50f042d661 crimson/osd: OpsExecuter differentiates read and modifying operations.
This is necessary to pass ClsHello.BadMethods.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:59 +02:00
Radoslaw Zarzynski
d2640e5b45 crimson/osd: implement CEPH_OSD_OP_GETXATTR and cls_cxx_getxattr().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:59 +02:00
Radoslaw Zarzynski
7631691f9f crimson/osd: differentiate ENOENT during do_osd_call().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:58 +02:00
Radoslaw Zarzynski
e374b22086 crimson/osd: implement cls_cxx_read2().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:58 +02:00
Radoslaw Zarzynski
6c7e7ad7ce crimson/osd: implement cls_cxx_setxattr().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:58 +02:00
Radoslaw Zarzynski
edced75b15 crimson/osd: implement CEPH_OSD_OP_SETXATTR.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:58 +02:00
Radoslaw Zarzynski
ca3ad1dbd1 crimson/os: port get_max_attr_name_length() from MemStore.
This method is an enabler for CEPH_OSD_OP_SETXATTR.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:58 +02:00
Radoslaw Zarzynski
71ccd4f2b1 objclass, osd: drop unused variable.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:57 +02:00
Radoslaw Zarzynski
eee4b843cf crimson/osd: implement cls_cxx_write_full().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:57 +02:00
Radoslaw Zarzynski
44e14775a0 crimson/osd: introduce OpsExecuter to uniform calling CEPH_OSD_OPS.
OSD has two entry points for executing CEPH_OSD_OP_*:
  1. the MOSDOp message handler,
  2. the Object Class API (cls_* and cls_cxx_* functions).

We definitely want to address these two users without code
duplication. However, exposing the entire PG to Obj Class
would break encapsulation. Moreover, there is difference
in life times between PG and sequence-of-operations-from-
MOSDOp.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:57 +02:00
Radoslaw Zarzynski
aaadbe6df7 crimson/osd: implement cls_cxx_stat().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 03:33:57 +02:00
Radoslaw Zarzynski
3d8b28bfa0 crimson/osd: implement CEPH_OSD_OP_STAT.
This commit is enabler of cls_cxx_stat(), and thus the RecordHello
and ClsHello unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 01:27:44 +02:00
Radoslaw Zarzynski
c76ed81343 crimson/osd: implement CEPH_OSD_OP_CALL.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 01:27:44 +02:00
Radoslaw Zarzynski
ed8eee05cd crimson/osd: rename crimson_error and introduce namespace.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 01:27:44 +02:00
Radoslaw Zarzynski
2407f576ef crimson/osd: exceptions derive from std::system_error now.
This change is be useful especially for CEPH_OSD_OP_CALL
which will be brought by further commits. The issue here
is that the Ceph Classes will be based on existing iface
handling errors with usual, int-based ret codes. These
codes needs to be glued with the existing handling  in
pg::do_osd_ops().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 01:27:44 +02:00
Radoslaw Zarzynski
a9c29ae5ac crimson: put extra debugs into do_osd_op().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 01:27:44 +02:00
Radoslaw Zarzynski
57330088d3 osd: switch ClassHandler::get_method() to std::string.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-08-24 01:27:43 +02:00