Commit Graph

30965 Commits

Author SHA1 Message Date
Samuel Just
0bba79b722 ReplicatedPG: support async reads on ec pools
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:17 -08:00
Samuel Just
647e75ae78 src/: add CEPH_OSD_OP_READ_SYNC
EC pools won't support SYNC reads.  This means that TMAP and friends
along with reads from an object class will not be supported.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
3ef6282c8d ReplicatedPG: use pgbackend->objects_read_sync() as appropriate
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
2fe5e1f2d2 osd/: add sync/async read methods to PGBackend
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
66bc91cf0a ObjectStore: _collection_move_rename needs to increment ops
Bugfix.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
a3e4a844ef PGLog,PG: reset can_rollback_to when we get a log entry without a transaction
We cannot rollback past an entry which we didn't actually
apply locally.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
347d968a9d PGLog: we cannot rollback past a recovered object entry
We don't recover the older versions, so we can't do a rollback.
In general, this is ok since we had enough copies to recover it
in the first place.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
25eced59a8 PGLog: create interface allowing interface user to cleanup/rollback
We need to be able to allow the PGLog interface user to provide
logic for rolling back and trimming log entries.  To that end,
serveral PGLog methods now take a LogEntryHander.

In PGLog::merge_old_entry, if prior_version > info.log_tail and
the object is not missing, we must have rolled back the prior
log entry.  Thus, we don't skip the entry.

To simplify the code, _merge_old_entry has been split out as
a const helper.  This way, proc_replica_log can be reexpressed
as merging the divergent replica log entries with the fully
merged authoritative log.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
c98cb1ce74 ReplicatedPG: Fill in rollback info for log events
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
ecae37c33b osd/: introduce xattr caching for ec pools
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
155b4b069f PGBackend,ReplicatedBackend: add support for rolling back log entries
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:16 -08:00
Samuel Just
0528974584 osd_types: add a can_rollback_to line to pg_log_t
This allows us to determine how far back the log can be rolled.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
63f64de118 osd_types: add ObjectModDesc into pg_log_entry_t
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
a7a6a2f153 osd_types: ObjectModDesc
Contains information needed to roll back a pg_log_entry_t.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
53c26bc335 osd/: move client op handling into ReplicatedBackend
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
c58c8990d9 PGBackend: allow queue_transaction to specify an op
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
1a4694fdde ReplicatedPG: factor out should_send_op
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
26b804f266 ReplicatedPG: split out op_applied_replica
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
09717e6d5e PG/ReplicatedPG: move snap map management into append_log
Both the replica and the primary should use the same append_log
mechanism to update the snap map.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
5732f0c99d ReplicatedBackend: implement RPGTransaction
RPGTransaction is essentially a wrapped ObjectStore::Transaction.
The coll_t argument is elided, tempness is instead encoded in the
hobject.  RPGTransaction tracks which temp objects are created and
cleared so we can update the ReplicatedBackend tracking and possibly
create the temp collection as needed.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
bf9390f3e2 hobject_t/ReplicatedPG: tempness is now an hobject thing
PGBackend implmentations will have complete control over the temp
collection.  Rather than specifying the collection when sending
ops into the PGBackend, hobjects themselves will be temp or not.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
c60185c01a PGBackend: add PGTransaction
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:15 -08:00
Samuel Just
d3c8e85a44 ReplicatedBackend: update the collection_list mechanisms to skip stashed objects
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
4d56e3c831 test/osd/: verify length of buffer returned in read
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
0885880b30 test/osd/TestRados.cc: skip 0 weight ops
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
d0933eef60 test/osd: add append support to WriteOp
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
ca91fb4c5d test/osd: adapt RadosModel to not do omap ops on ec_pools
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
7fe22da29f test/osd: restructure Object/RadosModel in prep for append
Attribute handling no longer has special support in ContentsGenerator.
The most recent operation information is now stored in a special
attr rather than at the beginning of the object.  ObjectDesc layers
include their own ContentsGenerators to allow more flexibility.

Also, writes truncate to the new object size rather than simply
causing reads to stop at that object size.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
69ad880eed test/osd/Object: create encoders for ContDesc
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
8494aa84d5 src/test/osd: remove tmap test
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
03c5344f74 PGLog: allow for empty pg log in update_range if log_tail == eversion_t()
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
1b0335ed66 common: add option to disable filestore sharded object check
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
1b5992b0f0 FileStore::lfn_link: handle link within a collection
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:14 -08:00
Samuel Just
6d05fa23be PG: prefer older last_update for ec_pool
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:13 -08:00
Samuel Just
393a035b86 OSDMonitor: add debug_fake_ec_pool
This flag will cause ReplicatedPG to act as though the
pool were actually an EC pool in that operations will
be restricted to operations which can be locally rolled
back thereby allowing us to test the ReplicatedPG local
log rollback mechanisms independent of EC.  It will also
cause ReplicatedPG to use the async read mechanism on
the PGBackend implementation once it is implemented.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:39:13 -08:00
Samuel Just
a1ee1ee716 PGLog::rewind_divergent_log: we can rewind to tail, fix assert
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:26 -08:00
Samuel Just
a2006507a0 PGLog: don't move up log.tail
Moving up log.tail unnecessarily risks backfilling
a replica after a split.  Also, it disrupts the
property that replicas from the most recent interval
which performed writes must have overlapping logs.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:25 -08:00
Samuel Just
1f9457c53f ReplicatedBackend: remove pushing entry for each object as it completes
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:25 -08:00
Samuel Just
b7d100b697 FileStore::_collection_move_rename: remove source before closing guard
Fixes a bug in _collection_move_rename replays.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:25 -08:00
Samuel Just
06f7a98a03 ReplicatedBackend.h: don't need to be active for pushes
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:25 -08:00
Samuel Just
518774dc0c ObjectStore: improve name of the queue_transaction which cleans up the transaction
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:25 -08:00
Samuel Just
c03d027cf8 hobject: admit that gen_t is actually version_t
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-22 14:38:25 -08:00
Gregory Farnum
d6b592c087 Merge pull request #1079 from ceph/wip-tier-bits
osd: a few tiering fixes

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-01-22 13:44:46 -08:00
Sage Weil
8eaef4140b Merge pull request #1126 from ceph/wip-osdmapenc-fix
MOSDMap: reencode maps if target doesn't have OSDMAP_ENC

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-22 08:20:08 -08:00
Ilya Dryomov
8060afd77c MOSDMap: reencode maps if target doesn't have OSDMAP_ENC
Reencode both full and incremental maps if target doesn't know how to
decode OSDMAP_ENC maps (CEPH_FEATURE_OSDMAP_ENC bit is not set).  This
fixes a compatibility bug that was introduced in 3d7c69fb09 ("OSDMap:
add a CEPH_FEATURE_OSDMAP_ENC feature, and use new encoding").

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-01-22 17:34:31 +02:00
tamil
54caa0192b removing rgw_tests.sh
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
2014-01-21 18:44:57 -08:00
tamil
4550fb5ff9 Merge branch 'master' of github.com:ceph/ceph 2014-01-21 17:50:22 -08:00
tamil
8b8ede7a13 modified the port to 7280 in the script instead of the default 80
Signed-off-by: tamil <tamil@tamil-VirtualBox.(none)>
2014-01-21 17:50:00 -08:00
Sage Weil
81ff12ec70 Merge pull request #1124 from ceph/wip-cache-mode-fix
mon/MonCommands: 'invalidate+forward' -> 'forward'

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-21 17:30:34 -08:00
tamil
850b4f426f script for rgw bucket quota testing
Signed-off-by: tamil <tamil@tamil-VirtualBox.(none)>
2014-01-21 15:56:37 -08:00