Commit Graph

37739 Commits

Author SHA1 Message Date
Sage Weil
5877c069f6 osd: drop infos_oid arg from PG::read_info()
Only needed for legacy.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
a599d88615 osd: drop legacy arg to PG::peek_map_epoch
The function can figure it out itself, and it's only needed for legacy.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
f93b293775 ceph_objectstore_tool: use '_remove' pgmeta omap key to indicate removal too
We could conceivably mark the PG for backfill, but to do this correctly
we have to do the log rollback, which uses various methods in PG to update
snap metadata that we simply don't have access to from this tool.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
cd462d173c osd: move pg removal flag into a static helper
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
David Zafman
f056bdf939 ceph_objectstore_tool: Prevent tool from transitioning struct_v on rm-past-intervals
Signed-off-by: David Zafman <dzafman@redhat.com>
2014-12-16 17:07:58 -08:00
David Zafman
fbf6408964 ceph_objectstore_tool: Clean-up unused struct_ver in tool's metadata section
Exported struct_ver not used by write_info() or write_pg()

Signed-off-by: David Zafman <dzafman@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
d6df6eabab osd: retire meta/infos object entirely
Do not create it; remove it on upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
9d1466cb6e osd: prevent upgrade from older than v7
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
d1973f3ba0 osd/PGLog: drop support for ancient logs
Assume that users have at least upgraded to dumpling first.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
8cb25cbdd6 test/osd/osd-test-helper.sh: enable filestore log
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
6074bdd155 osd: move pg log and info to a per-pg object
Move all PG metadata (info, biginfo, epoch, and log) to a single per-PG
object.  This should make it easier for ObjectStore implementations to
avoid lock contention in the write path.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
8fa94071f9 osd/PGLog: explicitly pass log_oid collection
Do not assume the log_oid is ine the 'meta' collection.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
9819221799 osd/PGLog: take ghobject_t instead of hobject_t
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
be86cc739c osd: move PG collection creation into static PG method
This way we properly create the pgmeta object and set the infokey attr on
it.  Break it into two steps, one part before split, one after.

Disable the collection_empty() check in split so that it doesn't trip over
this object.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
5a5a4a8775 osd: don't need to pass object names via ctor
Calculate biginfo on the fly.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
35fcfceff2 osd/PG: define pgmeta_oid member
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
4d22adf676 osd: skip pgmeta object on remove_dir
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
e247cd4be4 ceph-objectstore-tool: skip pgmeta object on export
We will (eventually) export the contents of this object explicitly as
PG metadata.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
9ebdb99a5e osd/PGBackend: skip pgmeta objects
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
a56f10af82 osd/osd_types: spg_t::make_pgmeta_oid()
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
033a533a72 common/hobject_t: add pgmeta named ctor and is_pgmeta() check
There will be exactly one of these per PG.  It slots into the flat

  (pool, hash, name...)

namespace but is not a legal rados name (since it is the empty string).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
c765de6b9f common/hobject_t: fix whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
95eccd8e95 osd: add PGMETA ondisk feature
Once you start using the pgmeta objects there is no going back.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
18214928d1 os/FileStore: move to v4, trigger DBOjectMap upgrade
From FileStore's perspective, nothing changed from v3 to v4.  However, we
need to pass the do_upgrade flag to DBOjectMap open to trigger it's
upgrade process.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
69e169d96b os/DBObjectMap: new version v2; drop support for upgrading from v1
- Don't support upgrade from v1 anymore
- Add upgrade for v1 -> v2

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
462bad3221 ceph_test_rados: generate mix of small and large xattrs
This ensures that we exercise the large xattr handling paths in the OSD.

(Specifically, we want to trigger DBObjectMap usage on EC objects so that
we can test the fix for #10171).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
456255b728 os/DBObjectMap: include hash in header key for EC objects
We were inadvertantly dropping the hash portion of the header key for EC
objects.  This is usually not noticeable, but turns up quite quickly with
the new pgmeta objects, which share the same name ('') across PGs with
different hash values

Fixes: #10171
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
ca1b216b47 Merge pull request #2137 from yuyuyu101/pg-sparse-recovery
ReplicatedPG: Make recovery using fiemap and fix perf counter

sage-2014-12-16_08:43:28-powercycle-wip-sage-testing-distro

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 15:15:02 -08:00
Sage Weil
9323ab29cd Merge pull request #3000 from yuyuyu101/wip-10166
Wip 10166: fiemap or FileStore::do_sparse_copy_range bug

sage-2014-12-16_08:43:28-powercycle-wip-sage-testing-distr

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 15:14:20 -08:00
Sage Weil
f8d37a2cc4 Merge remote-tracking branch 'gh/master' into wip-watch-notify
Conflicts:
	PendingReleaseNotes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh During <jdurgin@redhat.com>
2014-12-16 14:02:32 -08:00
Samuel Just
868d7496f5 Merge pull request #3033 from dachary/wip-10201-osd-scrub-repair
osd scrub repair uses ceph_objecstore_tool + shell toolbox

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-16 13:54:11 -08:00
Josh Durgin
9d61dec011 Merge pull request #3181 from ceph/wip-10299
librbd: complete all pending aio ops prior to closing image

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2014-12-16 13:44:21 -08:00
Sage Weil
eef1705354 Merge pull request #2300 from ceph/wip-osdmap
add osdmap checksums

    sage-2014-12-16_06:27:10-rados:singleton-wip-sage-testing2-distro-basic-multi
    sage-2014-12-15_13:18:23-rados:thrash-wip-sage-testing2-distro-basic-multi

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-16 08:49:44 -08:00
Sage Weil
e27d3bda5c Merge pull request #3166 from ceph/wip-librados-fadvise
fadvise for librados, librbd + osd guts

    sage-2014-12-16_06:27:10-rados:singleton-wip-sage-testing2-distro-basic-multi
    sage-2014-12-15_13:18:23-rados:thrash-wip-sage-testing2-distro-basic-multi
2014-12-16 08:46:26 -08:00
Sage Weil
dacab7b7a7 Merge pull request #3080 from ceph/wip-fadvise
osd: reenable fadvise; use it more often

sage-2014-12-16_06:27:10-rados:singleton-wip-sage-testing2-distro-basic-multi
sage-2014-12-15_13:18:23-rados:thrash-wip-sage-testing2-distro-basic-multi

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 08:38:40 -08:00
John Wilkins
8241f114d1 Merge pull request #3165 from nilamdyuti/wip-doc-ceph
doc: Updates man page for Ceph

Reviewed-by: John Wilkins <jowilkin@redhat.com>
2014-12-16 08:13:53 -08:00
Loic Dachary
f614fae100 Merge pull request #3184 from dachary/wip-10325-cauchy
erasure-code: relax cauchy w restrictions

Reviewed-by:  David Zhang <david.z1003@yahoo.com>
2014-12-16 15:10:54 +01:00
Loic Dachary
bb80437f24 erasure-code: relax cauchy w restrictions
A restriction that the w parameter of the cauchy technique is limited to
8, 16 or 32 was added incorrectly while refactoring parameter parsing in
the jerasure plugin and must be relaxed.

http://tracker.ceph.com/issues/10325 Fixes: #10325

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2014-12-16 13:56:31 +01:00
Wido den Hollander
058f433930 man: Deprecate preferred OSD setting for cephfs
In the source code of cephfs there is no reference to it anymore,
but it's still in the man page. Leave it there for now, but make
it more clear that it is deprecated and unsupported.
2014-12-16 13:07:36 +01:00
Josh Durgin
cbc6c7cab0 Merge pull request #2970 from ceph/wip-swift-storage-policy
rgw: support swift storage policy api

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2014-12-15 23:36:40 -08:00
Josh Durgin
75a165db63 Merge pull request #3163 from ceph/wip-9254
rgw: http headers need to end with \r\n

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sweil@redhat.com>
2014-12-15 23:35:27 -08:00
Jason Dillaman
37a99694b1 librbd: complete all pending aio ops prior to closing image
It was possible for an image to be closed while aio operations
were still outstanding.  Now all aio operations are tracked and
completed before the image is closed.

Fixes: #10299
Backport: giant, firefly, dumpling
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2014-12-15 23:25:04 -05:00
Alfredo Deza
9ff914405c librados: do not choke on asphyxiate doc generation
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2014-12-15 14:23:31 -05:00
Sage Weil
a282f7e119 Merge pull request #3142 from ceph/wip-10220
mon: suspend proposing paxos values when flushing pending writes

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-15 10:44:51 -08:00
Loic Dachary
1396e55d77 Merge pull request #3182 from ceph/wip-9405
librbd: fixed garbage output from test LibRBD.TestIOPP

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-15 19:18:01 +01:00
Jason Dillaman
394fd4c212 librbd: fixed garbage output from test LibRBD.TestIOPP
buffer::list::c_str does not actually provide a C-style, NULL
terminated string.  As a result, its use for console output
resulted in stray garbage characters being printed.

Fixes: #9405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2014-12-15 12:44:44 -05:00
Sage Weil
3b5ff219fb Merge pull request #3161 from anandgbhat/wip-jemalloc
jemalloc-support: Added --with-jemalloc configure option

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-15 09:37:57 -08:00
Loic Dachary
0b8512eb0b Merge pull request #3180 from ktdreyer/doc-corpus-spelling
doc: fix spelling in corpus page

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-15 18:20:09 +01:00
Joao Eduardo Luis
18534615f1 mon: Paxos: reset accept timeout before submiting work to the store
Otherwise we may trigger the timeout while waiting for the work to be
committed to the store -- and it would only take the write to take a bit
longer than 10 seconds (default accept timeout).

We do wait for the work to be properly committed to the store before
extending the lease though.

Fixes: #10220

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-12-15 16:49:55 +00:00
João Eduardo Luís
639096d9f7 Merge pull request #3146 from ceph/wip-9323
mon: allow full flag to be manually cleared

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
2014-12-15 16:36:08 +00:00