Commit Graph

71734 Commits

Author SHA1 Message Date
Sage Weil
d0b2e606f7 Merge pull request #14826 from tchaikov/wip-prime-pg-temp
mon/OSDMonitor: skip prime_pg_temp if mapping is prior to osdmap

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-04-28 11:27:22 -05:00
Sage Weil
e1065fed07 Merge pull request #14818 from tchaikov/wip-rocksdb
rocksdb: sync with upstream

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-04-28 11:26:54 -05:00
Yuri Weinstein
ebdce966de Merge pull request #14817 from TsaiJin/wip-set-boolean-variable-with-boolean-value
osd: ReplicatedBackend::prep_push() remove redundant variable assignments

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-28 08:38:06 -07:00
Yuri Weinstein
75c1818f1e Merge pull request #14835 from tchaikov/wip-19743
mgr: do shutdown using finisher so we can do it in the right order

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-28 08:34:19 -07:00
Yuri Weinstein
5f6db0e81a Merge pull request #14838 from liewegas/wip-mgr-dirs
debian/ceph-base.dirs: create bootstrap-mgr dirs

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-04-28 08:32:52 -07:00
Sage Weil
e3998f0e30 osd/PG: fix upgrade()
We need to update the infover_key on any upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
993e2d5fac osd/osd_types: encode all_participants for pi_compact
The all_participants is only updated on add_interval and
cannot be rebuilt from the intervals vector alone since
we only keep the smallset subset of peers to probe that
we need.  And it wasn't being encoded/decoded.  Or
dumped.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
f9f8f6fda0 osd/PG: clear past intervals on a osdmap gap
If we are crossing an osdmap gap, clear our old past
interval since the information is stale: someone else
took the pg clean after everything we know if the maps
were trimmed.

This avoids an assert later in the past intervals check.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
a2d2232f39 osd/PG: clear past_intervals as needed on upgrade
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
abc615a883 osd/PG: clear past_intervals from update_history
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
76ada64067 osd/PG: dirty_big_info=true when we clear past_intervals
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
43485257a9 osd/PG: consolidate all history updates in update_history
...and take care of reg and unreg_scrub.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
539c072f81 osd: fix build_past_intervals_parallel
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
5a5903d0f5 src/test/vstart-wrapper.sh: run with -d
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Sage Weil
3b148cddc9 osd/PG: store last_interval_started in pg_info_t too
We have a last_epoch_started value in pg_info_t; store
the corresponding last_interval_started value alongside
it.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:32:17 -04:00
Kefu Chai
7b676f4f0f Merge pull request #14384 from hrchu/radosgw-admin-key
doc: rgw: Rewrite the key management

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-04-28 23:32:16 +08:00
Sage Weil
f26aabaddf osd: populate PastIntervals on new PGs
When we get a pg create from the mon we don't get
PastIntervals with it; generate it from scratch as
needed.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:40 -04:00
Sage Weil
ef4012bf35 osd/osd_types: track pg_history last_interval_{starte,clean}
These will track last_epoch_{started,clean} but match
the first epoch in the interval instead of the epoch when
the event happened.  We didn't end up need this now, but
I suspect it will be useful in the future.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:40 -04:00
Sage Weil
484d45bf2e osd/osd_types: more compact compact_interval_t printer
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
e8267fa0ba osd/osd_types: allow get_bounds() on empty
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
bb2a2fe267 osd/osd_types: simplify bounds
The bounds are based on last_epoch_clean, which can
happen at any point during an interval (usually not the
beginning!).  Instead of trying to ensure that the
PastIntervals include the oldest interval, just ensure
that they go at least as far back as last_epoch_clean.
This means that we might have *more* intervals, but given
that all we ever do is *clear* past_intervals when we
go clean, I don't think there is much value in trying
assert more.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
73b64ec702 osd/osd_types: fix encoding for PastIntervals
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
2dd9d2a1d4 osd/osd_types: print and encode empty PastIntervals
These happen when they are optionally sent across the
wire.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
5d92d29ac4 messages/MOSDPGLog: show past_intervals if present
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
e7ccadec51 messages/MOSDPGNotify: streamline luminous+ encoding
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
bc7ddd3e8d messages/MOSDPGInfo: streamline luminous+ encoding
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
da4937c68d osd/osd_types: make pg_notify_t operator<< slightly less gross
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
59a92aeee0 osd/PG: fix operator<< and init debug
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
bb99b51eff osd/osd_types: fix PastIntervals::update_type()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
2fd345188b osd/osd_types: fix pi_compact_rep empty() condition
We may have no intervals but still be non-empty (have a first and
last), because during that period there were no osds.  This easily
happens when a pool is created before osds are up.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
1c15c18896 osd/osd_types: fix pi_compact_rep bounds; rename fields
The second bound is non-inclusive.  Use 'first' and 'last' instead of
'start' and 'end' to reinforce that.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
f2a13dc0d7 osd/PG: put interval bounds [) in debug/log output
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
0e94e17740 osd/PG: log peering intervals at high debug level
Because we are no longer explicitly tracking all past intervals
I think it is a good idea to log these more aggressively.  This will
give us potentially vital information when debugging peering problems
(says someone who just debugged a peering problem in which the past
interval information provided a vital clue).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Sage Weil
a9d7911913 unittest_osd_types: minor fix to past_intervals tests
Set [up_]primary to valid values (not that it is actually
relevant to the test!).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:39 -04:00
Samuel Just
6505175684 test/osd/types: add tests specifically for prior set generation
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:39 -04:00
Samuel Just
44264de239 test/osd/types.cc: fix check_new_interval test
Removes the asserts relating to past_intervals state.
They don't really make sense with the new representation,
we'll add new tests for that next.

Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:39 -04:00
Samuel Just
5a9c3be3c7 osd_types: refactor PriorSet so it can be tested without an OSDMap
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:39 -04:00
Samuel Just
6f1783d9ba osd_types: just pass last_epoch_started to PriorSet
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
acb713b48c osd_types: fill in generate_past_intervals for new types
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
867fd4e790 osd_types: implement PastIntervals pi_compact_rep
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
3ed85f0f2e PG: accommodate new past_intervals encoding
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
aeee4bc8ce RadosDump: accommodate new past_intervals encoding
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
8e4856041d messages/: accomodate the new past_intervals encoding
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
f52313c461 osd_types: generalize PastIntervals to support multiple representations
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
0baebe166b osd/: hide pg_interval_t encoder
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
065bb89ca6 osd/: refactor past_intervals logic into PastIntervals
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
e4c7ecea4a PG::GetInfo: remove logic for detecting intervals without complete peers
c7d92d1d3f introduced this back when
the acting set could contain incomplete peers during backfill.  That
hasn't been true since dumpling.  Now, any interval where the acting
set contains an incomplete peer cannot possibly go active.  Thus, it
can't change last_epoch_started or history.last_epoch_started.  Thus,
even though choose_acting omits incomplete peers, the answer can't
change.

Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Samuel Just
7485f9e7cc test/osd_types.cc: fix emacs line at the top
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-04-28 11:30:38 -04:00
Sage Weil
3ab8dff07f qa/suites/upgrade/jewel-x: add cache tiering + snaps workload
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:30:38 -04:00
Sage Weil
d063c3dc73 qa/suites/upgrade/kraken-x/stress-split-erasure-code: fix
Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-28 11:14:45 -04:00