Commit Graph

81 Commits

Author SHA1 Message Date
Adam C. Emerson
8777980283 osd: Build ceph-osd without using namespace declarations in headers
This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-04-06 11:15:06 -04:00
Xuehan Xu
0c0d07fbe7 OSD: make pglog related "logv" parameters rvalue references
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-02-19 16:05:05 +08:00
Samuel Just
a335a2599d osd/PGBackend: rename submit_transaction field roll_forward_to
This field is actually just a lower bound on committed, client
visible log versions.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-12-19 17:35:36 -08:00
Samuel Just
48ce9f4570 osd/: convert boost::optional users to std::optional
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-09 17:22:26 -07:00
David Zafman
57abdb11fa osd, test: Add num_shards_repaired to osd_stat_t for pushes with repair set 3(3)
Fixes: http://tracker.ceph.com/issues/38616

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-25 16:03:36 -07:00
Zengran Zhang
0f33fc9c16 osd: ec saves a write access to the memory under most circumstances
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
2019-01-21 19:19:16 +08:00
David Zafman
0474498684 osd: Deny reservation if expected backfill size would put us over backfill_full_ratio (EC)
Erasure Coded Pools

Fixes: http://tracker.ceph.com/issues/19753

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-18 09:30:44 -08:00
Adam C. Emerson
9d239edf2c osd: Use ceph_assert for asserts.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-08-27 10:27:22 -04:00
Sage Weil
27e91a99f5
Merge pull request #21273 from jdurgin/wip-23195
osd/ECBackend: only check required shards when finishing recovery reads

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-24 17:20:25 -05:00
Josh Durgin
b162a5478d osd/ECBackend: recover from EIO based on the minimum data necessary
Discount shards that already returned EIO, and use minimum_to_decode()
to request just what is necessary to recover or read the originally
requested extents of the object.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2018-04-20 19:42:14 -04:00
Josh Durgin
0f9474d4e8 osd/ECBackend: store the original needed shards in a ReadOp
This avoids extra logic recomputing them due to EIO or redundant
reads.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2018-04-20 19:28:59 -04:00
Wilson E. Alvarez
ce6e043a44
src: Added const references to various function parameters
Added const references to various function parameters in order to avoid
copying data unnecessarily and enhancing performance

Signed-off-by: Wilson E. Alvarez <wilson.e.alvarez1@gmail.com>
2018-04-06 17:43:04 -04:00
songweibin
860ebfedcc osd: correct several spell errors in comments
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-03-27 19:54:48 +08:00
Sage Weil
01b68f969c osd/PGBackend: drop on_applied callback for submit_transaction
This removes a ton of tracking for ReplicatedBackend.  ECBackend needs
to keep most of it so that it can track in-flight applies on legacy
peer OSDs.  We can remove this post-nautilus.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
ee314b9351 osd/ECBackend: unify commit+applied msg; drop applied callback
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
26e450753c osd/PGBackend: drop on_local_applied_async
Unused!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
5c1a3c644a Revert "osd/ECBackend: fix on_write ordering w/ sync onreadable callbacks"
This reverts commit 1908c06149.

Hoping this helps resolve the failed assertion

src/osd/ExtentCache.h: 371: FAILED assert(tid == 0)

Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-29 15:24:19 -06:00
Sage Weil
bf16f59887 osd: piecewise scrub
Perform scrub in stages, with each unit of work requeuing an item in the
work queue.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-16 06:54:06 -06:00
Adam C. Emerson
28d35abf89 rados: Switch MIN/MAX for std::min/max and use intarith templates
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-01-12 14:32:52 -05:00
Sage Weil
1908c06149 osd/ECBackend: fix on_write ordering w/ sync onreadable callbacks
When we call handle_sub_write after a write completion, we may
do a sync read completion and then call back into check_ops().  Attaching
the on_write events to the op we're applying means that we don't ensure
that the on_write event(s) happen before the next write in the queue
is submitted (when we call back into check_ops()).

For example, if we have op A, on_write event W, then op B, a sync
applied completion would mean that we would queue the write for A, call
back into SubWriteApplied -> handle_sub_write_reply -> check_ops and then
process B... before getting to W.

Resolve this by attaching the on_write callback to a separate Op that is
placed into the queue, just like any other Op.  This keeps the ordering
logic clean, although it is a bit ugly with the polymorphism around Op
being either an Op or an on_write callback.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-12 11:21:49 -06:00
Sage Weil
7d33ceb65e osd/PGBackend: remove unused on_flushed()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-04 12:45:17 -06:00
Neha Ojha
c8419dee19 osd/PG: const cleanup for recoverable/readable predicates
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-11-17 16:20:00 +05:30
Myna V
33d79f303d Introducing sub-chunks. Replaced the old decode, minimum_to_decode
with new decode, minimum_to_decode in ErasureCodeInterface.
Updated ECBackend, ECUtil to use the new functions.Fixed the test cases
to use the new functions.
Fixed the review comments.

Authors: Myna, Elita.

Signed-off-by: Myna Vajha <mynaramana@gmail.com>
2017-10-30 07:57:05 +05:30
Brad Hubbard
2094fda765 osd: Add a flag to ScrubMap to signal check needed
If we know none of the scrub maps have errors early in
be_large_omap_check() we can return without doing a lot of unnecessary
work.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-24 17:27:57 +10:00
Sage Weil
d14cc56157 osd/PGBackend: remove scrub_supported()
Everybody supports scrub.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-06 13:08:18 -05:00
David Zafman
390d12f71a osd: For recovery get all possible shards to read on errors
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-28 23:31:25 -07:00
songweibin
9757957b25 osd: fix spell errors
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2017-08-31 12:35:02 +08:00
Josh Durgin
96f9ff1be8 osd/PrimaryLogPG,PGBackend: handle deletes during recovery
Deletes are the same for EC and replicated pools, so add logic for
handling MOSDPGRecoveryDelete[Reply] to the base PGBackend
class.

Within PrimaryLogPG, add parallel paths for starting deletes,
recover_missing() and prep_object_replica_deletes(), and update the
local and global recovery callbacks to deal with lacking an
ObjectContext after a delete has been performed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-07-17 02:00:35 -04:00
David Zafman
8185535846 osd: Handle read failures when pushing at primary
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-06-23 08:02:51 -07:00
Sage Weil
7d4af0531d blkin: add traces to ECBackend
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-05 14:05:23 -04:00
liuchang0812
acec741706 osd: add override in headers files
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2017-03-25 12:55:38 +08:00
Sage Weil
cb76f60cae osd/ECBackend: constify
Signed-off-by: Sage Weil <sage@redhat.com>
2017-02-20 14:22:05 -05:00
Sage Weil
1699bf7b71 common/hobject: eliminate wonky compartors
Signed-off-by: Sage Weil <sage@redhat.com>
2017-02-11 10:45:16 -05:00
Sage Weil
a03c5be452 Merge pull request #12149 from adamemerson/wip-g_ceph_context-exterminate
global: start removing g_ceph_context

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-12-23 08:40:48 -06:00
Adam C. Emerson
1d5e967a05 osd: Remove g_conf and g_ceph_context
So begins the ruthless annihilation of g_ceph_context.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2016-12-22 13:55:37 -05:00
huangjun
9e19b86721 osd/ECBackend: remove unused variable "ReadCB"
Signed-off-by: huangjun <hjwsm1989@gmail.com>
2016-12-17 10:00:06 +08:00
Samuel Just
6f8e4b3810 src: rename ReplicatedPG to PrimaryLogPG
It's been confusing for a long time that EC pools are implemented by
ReplicatedPG.  What PG/ReplicatedPG really implement is the concept
of a PG where consistency is managed by the primary via a log.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-12-14 10:18:27 -08:00
Sage Weil
2041a187bf Merge pull request #12305 from athanatos/wip-ec-rbd-fixes
osd: fixes to make rbd on ec work

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-05 11:45:07 -05:00
Samuel Just
7750616139 ECBackend: ops must consistently use or not use the cache
If the state is flipped, ops already in the pipeline need to
continue to maintain it.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-12-02 15:56:08 -08:00
David Zafman
94bfb853cf Merge pull request #12142 from athanatos/wip-17966
osd: Fix read error propogation in ECBackend

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
2016-11-23 11:24:17 -08:00
Sage Weil
61fca96c29 assert(0) -> ceph_abort()
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-21 23:37:31 -05:00
Samuel Just
fea2bdfc2a ECBackend: propogate read errors through callbacks
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-21 15:08:53 -08:00
Samuel Just
51c2c5a7b4 ECBackend::ReadOp: fill in complete in constructor, not in do_read_op
Otherwise, send_all_remaining_reads creates duplicate entries.

Fixes: http://tracker.ceph.com/issues/17966
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-21 12:51:07 -08:00
Samuel Just
1e95f2ce64 ECBackend: integrate cache and rmw pipeline
Implements the rmw pipeline and integrates the cache.

HashInfo now maintains a projected size for use during the planning
phase of the pipeline.

(Doesn't build without subsequent patches, not worth stubbing out
the interfaces)

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:41:33 -08:00
Samuel Just
74ddea4b2b ECBackend: remove unused hobject argument to read_request_t
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:19 -08:00
Samuel Just
eb07d692b0 ECBackend: deduplicate start_remaining_read_ops and start_read_ops
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:18 -08:00
Samuel Just
735af7c6fa osd/: 's/trim_rollback_to/roll_forward_to/g'
trim_rollback_to was a not terrible name before in that all
it ever did is (possibly) trim the stashed version of the
object.  However, now, it's going to encompass, in general,
the roll_forward part of a tpc (which will still be to
delete the stashed object in cases where that is
appropriate).

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:18 -08:00
Samuel Just
2c99f864df osd/: switch all users of PGTransaction to use the new structure
This patch removes ReplicatedBackend::PGTransaction and implemenations
and switches over all users.  Happily, do_osd_ops loses the mod_desc
cruft and OpContext::pending_attrs.  PGTransaction doesn't really
have a natural way to implement append, however.  In reality, I think
this is probably an improvement, but it does mean that copy_from's
final transaction is now filled in by a lambda rather than by
appending a transaction fragment.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:17 -08:00
David Zafman
c51d70e1e8 osd: Handle recovery read errors
Fixes: http://tracker.ceph.com/issues/13937

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-10-27 22:42:10 -07:00
David Zafman
36fd68c966 osd: CLEANUP: Remove unused pending_read member
Remove unused struct

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-10-27 22:42:10 -07:00