Commit Graph

79192 Commits

Author SHA1 Message Date
Jianpeng Ma
2303c73261 os/bluestore/BlueStore: ASAP wake up _kv_finalize_thread.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-09 22:38:33 +08:00
Jos Collin
2953986b52 Merge pull request #18157 from batrick/ptl-tool-branch-checkout
ptl-tool: checkout branch after creation

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-10-09 03:27:27 +00:00
Sage Weil
96ddf5c3a0 Merge pull request #17708 from liewegas/wip-pg
osd: initial minimal efforts to clean up PG interface
2017-10-08 21:47:49 -05:00
Sage Weil
95fb8fbad7 Merge pull request #18125 from cernceph/dvanders_conf
mon: mark mon_allow_pool_delete as observed

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-08 21:47:26 -05:00
Sage Weil
6f72b404f4 Merge pull request #18144 from hjwsm1989/wip-21425-part2
qa/suites/rest/basic/tasks/rest_test: whiltelist OSD_DOWN

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-08 21:47:02 -05:00
Sage Weil
2dbbb351e2 Merge pull request #18159 from liewegas/wip-21712-b
os: fix 0-length zero semantics, add tests

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-08 21:46:27 -05:00
Sage Weil
b588eaf2b0 os/bluestore: 0-length zero should not change object size
Fixes: http://tracker.ceph.com/issues/21712
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-08 21:46:03 -05:00
Sage Weil
979b7c9cd2 os/filestore: make 0-length zero avoid touching object length
Fixes: http://tracker.ceph.com/issues/21712
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-08 21:46:03 -05:00
Sage Weil
9ad1f4f10f os/ObjectStore: 0-length zero does not change length of object
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-08 21:46:03 -05:00
Kefu Chai
3b9564f667 Merge pull request #18141 from adamemerson/wip-log-time
log: Use the coarse real time clock in log timestamps

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-09 09:35:46 +08:00
Kefu Chai
987809c2aa Merge pull request #17976 from adamemerson/wip-warning-sign
mgr,osd,kv: Fix various warnings for Clang and GCC7

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-09 09:34:50 +08:00
Adam C. Emerson
afaf48a263 log: Mark timestamps for coarseness and print appropriately
Have each timestamp know whether it's coarse or fine. Print six figures
for fine stamps and three for coarse.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-06 20:49:53 -04:00
Adam C. Emerson
0f8606b4e0 log: Add log_coarse_timestamps option
And set up an observer in CephContext to coarsen/refine time stamps as
requested.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-06 20:49:53 -04:00
Adam C. Emerson
a0a4c483bc config: Add templated _get_val for use in observers
When notified by a config observer, the config lock is already held,
create a _get_val template to manipulate newstyle options when
notified.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-06 20:49:53 -04:00
Adam C. Emerson
31de198955 log: Use log_clock, defaulting to coarse time
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-06 17:29:06 -04:00
Adam C. Emerson
f00547bb7d log: Add optional msg parameter to create_entry and test with it
The tests should, really, get their time from the same source as the
log, and also probably use the same interface to creating entries that
other people do. So add an optional message argument and have the
tests use that rather than using the new operator directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-06 17:29:06 -04:00
Adam C. Emerson
7b0168c5dd log: Create log_clock to replace use of utime_t
We create a new time type to support using coarse timestamps (for
performance) by default while still allowing the user to switch to
fine timestamps.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-06 17:29:06 -04:00
Sage Weil
582ee8d4b7 osd/PGLog: drop old compat coll/oid args to read_log_and_missing
The oldest version we care about is 10, and these were for <8.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 15:37:51 -05:00
Sage Weil
b4e4ad3b85 Merge pull request #18122 from liewegas/wip-monthrash-peering
qa/suites/rados/monthrash: tolerate PG_AVAILABILITY during mon thrashing
2017-10-06 15:32:45 -05:00
Yuri Weinstein
5dc2deac30 Merge pull request #18047 from gregsfortytwo/wip-20416-bitwise-assert
osd: make the PG's SORTBITWISE assert a more generous shutdown

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-06 12:48:35 -07:00
Sage Weil
2347ecb961 ceph-objectstore-tool: bring past_intervals up to date on import
We can't blinding fast-forward the pg epoch to the latest map without
also updating the pg history and PastIntervals.  Do that at import time.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 14:26:56 -05:00
Sage Weil
366a3d598e ceph-objectstore-tool: hide DIAGNOSTIC stuff behind debug/verbose mode
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 14:26:56 -05:00
Sage Weil
31312797be ceph-objectstore-tool: drop unused biginfo_oid
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 14:26:56 -05:00
Sage Weil
36817117d6 ceph-objectstore-tool: refuse to import PG older that OSD's oldest_epoch
We don't have a way to construct a valid PastIntervals history.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 14:26:56 -05:00
Patrick Donnelly
ae5d61bfd5
ptl-tool: checkout branch after creation
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-06 11:52:04 -07:00
Sage Weil
b6a5c09dba ceph-objectstore-tool: remove rm-past-intervals op
The OSD doesn't rebuild this on demand anymore.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
0f654d0f8d ceph-objectstore-tool: do not clear same_interval_since and PastIntervals
We dropped the OSD recalculation code for this.  Import it.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
10a5a838f8 ceph-objectstore-tool: print pg_info_t to stdoutt
Helpful for debugging

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
7816ab5a8e osd: print summary for forced backfill/recovery to debug
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
05d52061ac osd/PG: drop OSD friend, finally!
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
f6d3077225 osd: drop redundant is_active() check before scrub_sched()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
7debd61f8b osd/PG: make update_snap_mapper_bits public
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
25769b2736 osd/PG: make on_removal() public
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
c4616eec28 osd/PG: make get_primary() public
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
17e6d40080 osd/PG: add get_pool()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
070181a9e5 osd: use pg_id.pool() for pg pool id
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
24af955d02 osd/PG: make sched_scrub() public
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
88c13308b6 osd/PG: get_must_scrub()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
d14cc56157 osd/PGBackend: remove scrub_supported()
Everybody supports scrub.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
2a3a3fdae7 osd/PG: change 'debug dump_missing'
- json
- move into PG class

No users in the qa suite.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
79e7dbb80c osd/PG: move bits of delete_dir (pg removal) into PG
We don't want to expose things like SnapMapper, OSDriver.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
b8f33aab64 osd/PG: define public interface for heartbeat peers
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
d36c468647 osd: use accessors for pg resurrection
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
7a4f30d8a6 osd/PG: simplify start_recovery_ops return value meaning
Instead of complex check in caller, return simple bool indicating whether
find_unfound() should be called with a rctx.

There is one remaining condition that can probably be simplified:

  if (!recovering.empty() ||
      work_in_progress || recovery_ops_active > 0 || deferred_backfill)
    return !work_in_progress && have_unfound();

but we will leave it for now.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
e5405feada osd/PG: simplify unfound recovery case; rename method
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
82077e7116 osd/PG: drop unneeded write_if_dirty
Nothing in start_recovery_ops or stuck_on_unfound dirties state.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
b3b8119585 osd/PG: move a few assertions, flag change into start_recovery_ops()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
81f93729ff osd/PG: move some more split machinery into PG
and move other split pieces into the public interface.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
fa7818aefa osd/PG: move write_if_dirty inside handle_{loaded,create}
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
Sage Weil
043816396c osd/PG: get_need_up_thru(), get_same_interval_since()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00