Commit Graph

84877 Commits

Author SHA1 Message Date
Sage Weil
3a331c8be2 osd: kill disk_tp, recovery_gen_wq
Progress!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:50 -05:00
Sage Weil
e5c336851c osd: move recovery contexts to normal wq
We have a specific PGRecoveryContext type/event--even though we are just
calling a GenContext--so that we can distinguish the event type properly.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
26938d54d4 osd: remove _ookup_lock_pg_with_map_lock_held()
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
60ea5e87b6 osd: new MOSDScrub2 message with spg_t, fast dispatch
Send new message to mimic+ OSDs.  Fast dispatch it at the OSD.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
62f79cae1b osd/PG: request scrub via a state machine event
Continuing effort to make PG interactions event based.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
fe5e361467 osd: use peering events for forced recovery
The mgr code is updated to send spg_t's instead of pg_t's (and is slightly
refactored/cleaned).

The PG events are added to the Primary state, unless we're also in the
Clean substate, in which case they are ignored.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
bd8d198c07 osd/OSDMap: get_primary_shart() variant that returns primary *and* shard
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
ae210722b4 osd: prime pg_slots for to-be-split children
Once we know which PGs are about to be created, we instantiate their
pg_slot and mark them waiting_pg, which blocks all incoming events until
the split completes, the PG is installed, and we call wake_pg_waiters().

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:49 -05:00
Sage Weil
718b6baa3c osd: remove obsolete slow dispatch path for most messages
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
025ca7e1e3 osd: fast dispatch M[Mon]Command
These just get dumped onto a work queue.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
f325b7dbc7 osd: fast dispatch ping
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
6710d34872 mon/OSDMOnitor: send MOSDPGCreate2 to mimic+ osds
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
974896a819 osd: handle MOSDPGCreate2 messages (fast dispatch!)
Add a new MOSDPGCreate2 message that sends the spg_t (not just pg_t) and
includes only the info we need.  Fast dispatch it.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
b4fcd6d702 osd/OSDMapMapping: a getter that returns a spg_t
Note whether a pool is erasure so that we can generate an appropriate
spg_t for a mapping.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
39945d299e osd: send pg creations through normal pg queue
Queue a null event tagged with create_info, elimiating the special
legacy path.

These are still not fast dispatch because we need an spg (not pg) to queue
and event, and we need a current osdmap in order to calculate that.  That
isn't possible/a good idea in fast dispatch.  In a subsequent patch we'll
create a new pg create message that includes the correct information and
can be fast dispatched, allowing this path to die off post-nautilus.

Also, improve things so that we ack the pg creation only after the PG has
gone active, meaning it is fully replicated (by at least min_size PGs).

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
294bc5d631 osd: fix max pg check for peer events
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:48 -05:00
Sage Weil
9ab2400109 osd: use atomic for pg_map_size
This avoids the need for pg_map_lock in the max pg check.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:47 -05:00
Sage Weil
828060749a osd/PGPeeringEvent: note mon- vs peer-initiated pg creates
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:47 -05:00
Sage Weil
9dc71e653a osd: fast dispatch peering events (part 2)
This actually puts the remaining peering events into fast dispatch.  The
only remaining event is the pg create from the mon.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:47 -05:00
Ricardo Dias
f736dbdef6
mgr/dashboard: TaskManager: implement finished tasks as a list
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-04-04 11:19:16 +01:00
Ricardo Dias
dd003e5ff0
mgr/dashboard: fix exception serialization for tasks
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-04-04 11:19:00 +01:00
Ramana Raja
4eeea7c0f3 doc: fix typos and other minor errors in cephfs-nfs doc
Signed-off-by: Ramana Raja <rraja@redhat.com>
2018-04-04 15:12:02 +05:30
Gu Zhongyan
bad4e929e5 mgr/zabbix: Fix wrong log message
Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
2018-04-04 17:29:49 +08:00
Lenz Grimmer
386c71362e
Merge pull request #21087 from ricardoasmarques/wip-date-pipe
mgr/dashboard: Add CdDatePipe

Reviewed-by: Volker Theile <vtheile@suse.com>
2018-04-04 11:29:44 +02:00
Ricardo Dias
00ba38ff59
Merge pull request #21060 from ricardoasmarques/wip-view-cache-status-for
mgr/dashboard: Support aditional info on 'cd-view-cache'

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-04-04 07:52:39 +01:00
Ricardo Dias
2fdd65ed3e
Merge pull request #21045 from votdev/remove_useless_code
mgr/dashboard: Remove unused code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-04-04 07:33:01 +01:00
Patrick Donnelly
ebd0fcd067
Merge PR #16779 into master
* refs/pull/16779/head:
	mds: cleanup MDCache::open_snaprealms()
	mds: make sure snaptable version > 0
	mds: don't consider CEPH_INO_LOST_AND_FOUND as base inode
	mds: replace MAX() with std::max()
	tools/cephfs: make cephfs-data-scan create snaprealm for base inodes
	qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
	qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
	mds: don't warn unconnected snaplrealms in cluster log
	mds: update CInode/CDentry's first according to global snapshot seq
	qa/cephfs: add tests for snapclient cache
	qa/cephfs: add tests for snaptable transaction
	mds: add asok command that dumps cached snap infos
	qa/cephfs: add tests for multimds snapshot
	client: don't mark snap directory complete when its dirstat is empty
	qa/workunits/snaps: add snaprealm split test
	mds: make sure mds has uptodate mdsmap before checking 'allows_snaps'
	client: fix incorrect snaprealm when adding caps
	qa/workunits/snaps: add hardlink snapshot test
	mds: add incompat feature and bump protocol for snapshot changes
	mds: detach inode with single hardlink from global snaprealm
	mds: record hardlink snaps in inode's snaprealm
	mds: attach inode with multiple hardlinks to dummy global snaprealm
	mds: cleanup rename code
	mds: ensure xlocker has uptodate lock state
	mds: simplify SnapRealm::build_snap_{set,trace}
	mds: record global last_created/last_destroyed in snaptable
	mds: pop projected snaprealm before inode's parent changes
	mds: keep isnap lock in sync state
	mds: handle mksnap vs resolve_snapname race
	mds: cleanup snaprealm past parents open check
	mds: rollback snaprealms when rolling back slave request
	mds: send updated snaprealms along with slave requests
	mds: explict notification for snap update
	mds: send snap related messages centrally during mds recovery
	mds: synchronize snaptable caches when mds recovers
	mds: introduce MDCache::maybe_finish_slave_resolve()
	mds: notify all mds about prepared snaptable update
	mds: record snaps in old snaprealm when moving inode into new snaprealm
	mds: cache snaptable in snapclient
	mds: recover snaptable client when mds enters resolve state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 15:41:19 -07:00
Zack Cerza
dd5e15a33d qa/suites/cephmetrics: Add new facets
For EPEL and containerized prometheus/grafana

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-04-03 15:44:33 -06:00
Zack Cerza
177b444b8d qa/suites/cephmetrics: Update ceph-ansible config
This is taken from the ceph-ansible suite.

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-04-03 15:44:33 -06:00
Patrick Donnelly
d4ee7b5056
Merge PR #20915 into master
* refs/pull/20915/head:
	Add config doc for exporting CephFS over NFS

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2018-04-03 12:52:48 -07:00
David Zafman
5cfb8241f4 osd: Fix stale scrub stats when a primary takes over
Fixes: http://tracker.ceph.com/issues/23267

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-03 12:51:06 -07:00
Patrick Donnelly
b1f8e9f948
Merge PR #21003 into master
* refs/pull/21003/head:
	client: lookup . on non-directory inode
	client: avoid may_lookup for lookup . and lookup ..

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-04-03 11:38:43 -07:00
Patrick Donnelly
a09b37721c
Merge PR #21221 into master
* refs/pull/21221/head:
	client: fix _read return for invalid offset

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 11:36:58 -07:00
Patrick Donnelly
4fd23a1eae
client: fix _read return for invalid offset
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 11:34:35 -07:00
Patrick Donnelly
30fb3129bc
Merge PR #21088 into master
* refs/pull/21088/head:
	client: fixup warning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 11:33:55 -07:00
John Wilkins
9b8c048192
Merge pull request #21050 from falcon78921/wip-doc-falcon78921
doc: updated add primary storage documentation for latest CloudStack release (4.11)

reviewed-by: John Wilkins <jowilkin@redhat.com>
2018-04-03 09:55:06 -07:00
David Galloway
34e0ecd105
Merge pull request #21218 from ceph/wip-dgall-crontab
test:qa:infra - Run update daily and use bash
2018-04-03 12:27:34 -04:00
David Galloway
07186a5b87 test:qa:infra - Run update daily and use bash
bash has the path for `source` to activate the virtualenv.  sh doesn't.

[skip ci]

Signed-off-by: David Galloway <dgallowa@redhat.com>
2018-04-03 11:39:13 -04:00
Sage Weil
2284e133af osd: fast dispatch peering events (part 1)
This is a big commit that lays out the infrastructure changes to fast
dispatch the remaining peering events.  It's hard to separate it all out
so this probably doesn't quite build; it's just easier to review as a
separate patch.

- lock ordering for pg_map has changed:
  before:
    OSD::pg_map_lock
      PG::lock
        ShardData::lock

  after:
    PG::lock
      ShardData::lock
        OSD::pg_map_lock

- queue items are now annotated with whether they can proceed without a
pg at all (e.g., query) or can instantiate a pg (e.g., notify log etc).

- There is some wonkiness around getting the initial Initialize event to
a newly-created PG.  I don't love it but it gets the job done for now.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
ac142c3cc0 osd: queue null events without PG lock
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
f9aea5da93 osd: move part of wake_pg_waiters into helper
We'll need this shortly.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
8e8c7cce1f osd: use MTrim peering event for trimming
This is simpler and cleaner than handling log trimming as a special case.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
cf5cd222ce osd: fast dispatch backfill and recovery reservation events
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
25da186ab8 osd: move M{Backfill,Recovery}Reserve event logic into message
Better encapsulation!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
3b904547fb messages/MOSDPeeringOp: add
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
aea80d9afb osd/PG: move peering event type out of PG class
We will create these directly from peering Messages shortly.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
beb8dd5b1a osd/PG: keep epoch, not map ref, of last osdmap for lsat persisted epoch
No need to pin the map in memory!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
a6fef5a61b osd/PG: remove old update_store_on_load()
This isn't needed post-luminous.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:12:35 -05:00
Sage Weil
643253c326 Merge tag 'v13.0.2'
v13.0.2
2018-04-03 10:08:22 -05:00
Sage Weil
e2dde5ff01 buffer: kill push_front
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-03 10:07:05 -05:00