Commit Graph

98028 Commits

Author SHA1 Message Date
Samuel Just
d9f4e8b389 osd/: move fulfill_info etc
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:23 -07:00
Samuel Just
b927419dd4 osd/: clarify PeeringState deletion interfaces
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
16d388bdae osd/: move min_peer_*_features and helpers
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
84ebb8ccf5 osd/: move proc_* and log handling methods
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
3e738e7b64 osd/: move share_pg_info to PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
35c65805b6 osd/: move waiting_for_flushed|peering handling into on_activate_complete
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
049d4196bb osd/: move search_for_missing, discover_all_unfound, build_might_have_unfound
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
8885b0e54c PeeringState: restructure activate to avoid snap_trimq
In particular, don't use snap_trimq when manipulating
info.purged_snaps.  Pass the resulting to_trim set
into on_activate.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:22 -07:00
Samuel Just
749a13d864 osd/: start moving activate to PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:21 -07:00
Samuel Just
027ab5ab36 osd/: move try_mark_clean/finish_recovery to PG::on_clean
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:21 -07:00
sjust@redhat.com
bc839f453d osd/: move choose_acting and callees into PeeringState
Also marks IsPGRecoverablePredicate users const to allow
simply reusing the instance on MissingLoc.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:21 -07:00
Samuel Just
33c231e759 osd/: remove OSDService::check_osdmap_full, fix caller
PeeringState was the only remaining caller -- both the interface in
PGBackend and the pass-through in PrimaryLogPG were actually unused.
Further, the WaitLocalRecoveryReserved user does not appear to actually
require the fresh-off-of-the-wire osdmap available from OSDService.
As such, moved the logic into OSDMap itself and simply used the
PG local osdmap.

Note, the above is a change in behavior that probably could use a second
opinion.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:21 -07:00
Samuel Just
cabe5b52cc osd/: move the backfill space reservation back into PG
PeeringState really only needs an interface for requesting
it.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:21 -07:00
Samuel Just
4f8fbd1dca osd/: move needs_recover/backfill, all_unfound_are_queried_or_lost
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:21 -07:00
Samuel Just
b52f2bf999 osd/: clarify recovery queueing from PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:20 -07:00
Samuel Just
7cbbb6065f osd/: move Active purged_snaps handling back to PG
Note: this patch only sets dirty_info/dirty_big_info when
info.purged_snaps is actually changed.  I *think* that's
right, but that part deserves particular attention during
review.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:20 -07:00
Samuel Just
c259ad5708 PeeringState: mechanically rename a ton of pg-> to ps->
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:20 -07:00
Samuel Just
2d2b3c18b8 PeeringState: introduce a macro for the ps, pl, and pg declarations
This way I can mechanically rename all of the PeeringState member
variable references.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:20 -07:00
Samuel Just
c66e46fe6b osd/: move build_prior and related state to PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:20 -07:00
sjust@redhat.com
74329bc8f3 PeeringState: clean up references in Primary, Start, Reset
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:20 -07:00
sjust@redhat.com
7424a57d5f osd/: move io reservation machinery into PeeringState
This patch recasts the reservation and backoff interfaces
on PeeringListener in terms of events to queue rather than
explicit callbacks (PG handles implementing via callbacks).

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:15 -07:00
Samuel Just
6fd7b5a1bd osd/: remove remaining scrubber references from PeeringState
Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:14 -07:00
sjust@redhat.com
5ec8f6863b osd/: rephrase [un]reg_next_scrub as on_info_history_change
As PeeringState won't be as careful about usage of
on_info_history_change, add a scrub_registered flag
to protect against double-(un)registers.

Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:14 -07:00
sjust@redhat.com
e3f42e0d0d osd/: merge update_store_with_options with on_pool_change
Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:14 -07:00
sjust@redhat.com
7f56513f80 osd/: simply requeue_map_waiters in handle_activate, remove take_waiters
This seems clearer.

Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:14 -07:00
sjust@redhat.com
6e28b271fe osd/: move check_past_interval_bounds into PeeringState
Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:14 -07:00
sjust@redhat.com
40f71cda0e osd/: move start_peering_interval and callees into PeeringState
Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
sjust@redhat.com
1d08ac58aa osd/: move should_restart_peering to PeeringState
Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
sjust@redhat.com
4884a28cd3 osd/: move check_full_transition to PeeringState
Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
10842f7c38 PeeringState: route recoverystate_perf calls through PeeringListener
Signed-off-by: Samuel Just <rexludorum@gmail.com>
2019-05-01 11:22:13 -07:00
Samuel Just
fbf06885fb osd/: move flush state into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
482eccd13e osd/: move check_recovery_sources and missing_loc into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
a22acd6032 osd/: extract MissingLoc into its own file
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
e42ca4204b osd/: move update_history and purge_strays into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
d995596433 osd/: move proc_replica_info and some callees into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
3b158e6308 osd/: move the is_* and acting/up/backfill helpers into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
sjust@redhat.com
8f3b4336a4 osd/: Move osdmap handling into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
sjust@redhat.com
e3fa38b984 osd/: move most peering state to PeeringState
This patch moves the 40 something peering state variables over into
PeeringState while leaving references to them in PG.  The following
patches will move over the users until all users are in PeeringState.
Then, the PG references will be removed.  A subsequent patch will also
move the recovery_state member to be the last initialize and first
destructed.

Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
365fe8638b PeeringState: begin abstracting out PG, add dpp for logging
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
d96c9072b7 PGStateUtils: remove PG*, move into PGStateHistory into PeeringState
I don't think there's any need to worry about the pg locking from
PGStateHistory.  NamedState::begin/exit and dump_pgstate_history are the
only users, so the only calls should be under the peering event handler,
the asok command, or the PG constructor and destructor.  The first two
already have the lock.  The last should be safe as well as long as
the state machine states are constructed and destructed after and
before the PGStateHistory instance. As such, this patch removes most
of that state leaving the epoch generation as an interface
implemented by PG.

The snap trimming state machine was already excluded, so that this
patch leaves it disabled.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
1e4166a049 osd/: pull NamedState and PGStateHistory out into PGStateUtils
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
442bfd80a1 PeeringState: fix pg->info->pgid.pgid references
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
5f202e82c2 PeeringState: adjust logging to avoid using pg
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
12b60ed53a osd/: mechanically rip PeeringState and friends out into their own file
The next step is to rationalize the PeeringState<->PG relationship.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
5e36acdb6f osd/: mechanically rename RecoveryMachine/State/Ctx to Peering*
I'm going to extract this logic and reuse it in crimson.  Recovery* has
always been a confusing name as it implements neither log-based recovery
nor backfill.  Rather, it's mainly the buisiness logic for agreeing on
an authoritative log and some ancillary things such as scrub/backfill
reservation.

$ for i in $(git grep -l 'RecoveryMachine'); do sed -i 's/RecoveryMachine/PeeringMachine/g' $i; done
$ for i in $(git grep -l 'RecoveryState'); do sed -i 's/RecoveryState/PeeringState/g' $i; done
$ for i in $(git grep -l 'RecoveryCtx'); do sed -i 's/RecoveryCtx/PeeringCtx/g' $i; done

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Sage Weil
1974f01113 ceph_test_objectstore: add very_large_write test
Write 2GB to verify the blockdevice aio splitting.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-05-01 10:24:52 -05:00
Willem Jan Withagen
3eb760580f common: Clang requires a default constructor, but it can be empty
Just do what the error messages ask for.

Error from Clang:
```
In file included from /home/jenkins/workspace/ceph-master/src/cls/rbd/cls_rbd.cc:28:
In file included from /home/jenkins/workspace/ceph-master/src/objclass/../include/types.h:21:
In file included from /home/jenkins/workspace/ceph-master/src/include/uuid.h:9:
In file included from /home/jenkins/workspace/ceph-master/src/include/encoding.h:17:
In file included from /usr/include/c++/v1/set:426:
In file included from /usr/include/c++/v1/__tree:16:
/usr/include/c++/v1/memory:2241:41: error: call to implicitly-deleted default constructor of '__compressed_pair_elem<ceph::BitVector<'\x02'>::NoInitAllocator, 1>'
      : _Base1(std::forward<_Tp>(__t)), _Base2() {}
                                        ^
/usr/include/c++/v1/vector:437:7: note: in instantiation of function template specialization 'std::__1::__compressed_pair<unsigned int *, ceph::BitVector<'\x02'>::NoInitAllocator>::__compressed_pair<nullptr_t, true>' requested here
      __end_cap_(nullptr)
      ^
/usr/include/c++/v1/vector:496:5: note: in instantiation of member function 'std::__1::__vector_base<unsigned int, ceph::BitVector<'\x02'>::NoInitAllocator>::__vector_base' requested here
    vector() _NOEXCEPT_(is_nothrow_default_constructible<allocator_type>::value)
    ^
/home/jenkins/workspace/ceph-master/src/common/bit_vector.hpp:163:3: note: in instantiation of member function 'std::__1::vector<unsigned int, ceph::BitVector<'\x02'>::NoInitAllocator>::vector' requested here
  BitVector();
  ^
/home/jenkins/workspace/ceph-master/src/cls/rbd/cls_rbd.cc:3289:16: note: in instantiation of member function 'ceph::BitVector<'\x02'>::BitVector' requested here
  BitVector<2> object_map;
               ^
/usr/include/c++/v1/memory:2179:39: note: explicitly defaulted function was implicitly deleted here
  _LIBCPP_INLINE_VISIBILITY constexpr __compressed_pair_elem() = default;
                                      ^
/usr/include/c++/v1/memory:2172:50: note: default constructor of '__compressed_pair_elem<ceph::BitVector<'\x02'>::NoInitAllocator, 1, true>' is implicitly deleted because base class 'ceph::BitVector<'\x02'>::NoInitAllocator' has no default constructor
struct __compressed_pair_elem<_Tp, _Idx, true> : private _Tp {
                                                 ^
1 error generated.

```

Fixes: http://tracker.ceph.com/issues/39561
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-05-01 15:57:24 +02:00
Josh Durgin
e4ac0a054e
Merge pull request #21729 from badone/wip-add-ceph_dump_log.py
scripts: ceph_dump_log.py

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-04-30 16:29:06 -07:00
Sage Weil
876d01dd58 Merge PR #27802 into master
* refs/pull/27802/head:
	ceph_telemetry: accept older formats and transform

Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-30 14:04:54 -05:00
J. Eric Ivancich
edddca27fa rgw: add editor directive comments to rgw services source files
So that both vi and emacs help to correctly format the source files,
add the directives to each of the source files. In addition, use
`#pragma once` in the header files instead of the macro work-around.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-04-30 14:54:55 -04:00