Commit Graph

38050 Commits

Author SHA1 Message Date
Xiaoxi Chen
59b70fe20b Cleanup: Drop hobject_incorrect_pool
This field in MOSDSubOp, used to support very old version
OSD(MOSDSubOp header version < 7, which was introduced in
mid 2012 by commit 2fe9816305)

Now since two years pasted, we don't need it anymore.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2014-12-17 21:50:07 +08:00
Wido den Hollander
0c9d55d404 pybind/cephfs: Fix setxattr function. Pass value as character pointer
This passes the value of the xattr correctly and makes it work.
2014-12-17 14:01:44 +01:00
Loic Dachary
9b73c76209 Merge pull request #3190 from dachary/wip-test-helpers
tests: ignore xmlstarlet extra empty lines
2014-12-17 09:00:42 +01:00
Loic Dachary
023a8aca8c tests: ignore xmlstarlet extra empty lines
With xmlstarlet 1.3.0 (which is on Ubuntu precise)

xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and
contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n

will output an extra empty line which does not show on xmlstarlet
1.5.0 (which is on Ubuntu trusty and Fedora 20).

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2014-12-17 08:02:17 +01:00
Sage Weil
e582135c57 os/FileStore: better debug on omap_setkeys
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:59 -08:00
Sage Weil
f6ca2bcb74 os/FileStore: better debug on unlink
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:59 -08:00
Sage Weil
43f0bcb19f os/MemStore: drop support for collection attrs
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:59 -08:00
Sage Weil
17e4e1974a os/ObjectStore: deprecated collection_getattr et al
Drop the functional tests for this too.

Return EOPNOTSUPP by default, and tolerate this in the callers.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:59 -08:00
Sage Weil
56232d96cf ceph_test_objectstore_workloadgen: set pgmeta omap instead of collection xattr
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:59 -08:00
Sage Weil
82f452cc7d osd/PG: drop PG/collection attrs from ScrubMap
The collection attrs are often version specific.  We should compare the
semantic PG metadata, if anything, not these!

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:59 -08:00
Sage Weil
a1a3e156b6 os/ObjectStore: deprecated collection attr operations
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
db3daaf11b osd/PG: drop PG::read_info support for ancient PGs
This is rewritten regularly by any vaguely-active OSD; no need for this
any more.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
880135abfb osd: drop snap_collections
This is obsolete since cuttlefish.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
5877c069f6 osd: drop infos_oid arg from PG::read_info()
Only needed for legacy.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
a599d88615 osd: drop legacy arg to PG::peek_map_epoch
The function can figure it out itself, and it's only needed for legacy.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
f93b293775 ceph_objectstore_tool: use '_remove' pgmeta omap key to indicate removal too
We could conceivably mark the PG for backfill, but to do this correctly
we have to do the log rollback, which uses various methods in PG to update
snap metadata that we simply don't have access to from this tool.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
cd462d173c osd: move pg removal flag into a static helper
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:58 -08:00
David Zafman
f056bdf939 ceph_objectstore_tool: Prevent tool from transitioning struct_v on rm-past-intervals
Signed-off-by: David Zafman <dzafman@redhat.com>
2014-12-16 17:07:58 -08:00
David Zafman
fbf6408964 ceph_objectstore_tool: Clean-up unused struct_ver in tool's metadata section
Exported struct_ver not used by write_info() or write_pg()

Signed-off-by: David Zafman <dzafman@redhat.com>
2014-12-16 17:07:58 -08:00
Sage Weil
d6df6eabab osd: retire meta/infos object entirely
Do not create it; remove it on upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
9d1466cb6e osd: prevent upgrade from older than v7
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
d1973f3ba0 osd/PGLog: drop support for ancient logs
Assume that users have at least upgraded to dumpling first.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
8cb25cbdd6 test/osd/osd-test-helper.sh: enable filestore log
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
6074bdd155 osd: move pg log and info to a per-pg object
Move all PG metadata (info, biginfo, epoch, and log) to a single per-PG
object.  This should make it easier for ObjectStore implementations to
avoid lock contention in the write path.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
8fa94071f9 osd/PGLog: explicitly pass log_oid collection
Do not assume the log_oid is ine the 'meta' collection.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:57 -08:00
Sage Weil
9819221799 osd/PGLog: take ghobject_t instead of hobject_t
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
be86cc739c osd: move PG collection creation into static PG method
This way we properly create the pgmeta object and set the infokey attr on
it.  Break it into two steps, one part before split, one after.

Disable the collection_empty() check in split so that it doesn't trip over
this object.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
5a5a4a8775 osd: don't need to pass object names via ctor
Calculate biginfo on the fly.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
35fcfceff2 osd/PG: define pgmeta_oid member
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:56 -08:00
Sage Weil
4d22adf676 osd: skip pgmeta object on remove_dir
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
e247cd4be4 ceph-objectstore-tool: skip pgmeta object on export
We will (eventually) export the contents of this object explicitly as
PG metadata.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
9ebdb99a5e osd/PGBackend: skip pgmeta objects
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
a56f10af82 osd/osd_types: spg_t::make_pgmeta_oid()
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
033a533a72 common/hobject_t: add pgmeta named ctor and is_pgmeta() check
There will be exactly one of these per PG.  It slots into the flat

  (pool, hash, name...)

namespace but is not a legal rados name (since it is the empty string).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
c765de6b9f common/hobject_t: fix whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:55 -08:00
Sage Weil
95eccd8e95 osd: add PGMETA ondisk feature
Once you start using the pgmeta objects there is no going back.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
18214928d1 os/FileStore: move to v4, trigger DBOjectMap upgrade
From FileStore's perspective, nothing changed from v3 to v4.  However, we
need to pass the do_upgrade flag to DBOjectMap open to trigger it's
upgrade process.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
69e169d96b os/DBObjectMap: new version v2; drop support for upgrading from v1
- Don't support upgrade from v1 anymore
- Add upgrade for v1 -> v2

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
462bad3221 ceph_test_rados: generate mix of small and large xattrs
This ensures that we exercise the large xattr handling paths in the OSD.

(Specifically, we want to trigger DBObjectMap usage on EC objects so that
we can test the fix for #10171).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
456255b728 os/DBObjectMap: include hash in header key for EC objects
We were inadvertantly dropping the hash portion of the header key for EC
objects.  This is usually not noticeable, but turns up quite quickly with
the new pgmeta objects, which share the same name ('') across PGs with
different hash values

Fixes: #10171
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-16 17:07:54 -08:00
Sage Weil
242e28d13a Merge pull request #2530 from majianpeng/fiemap
Fiemap

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-16 16:50:45 -08:00
Sage Weil
ca1b216b47 Merge pull request #2137 from yuyuyu101/pg-sparse-recovery
ReplicatedPG: Make recovery using fiemap and fix perf counter

sage-2014-12-16_08:43:28-powercycle-wip-sage-testing-distro

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 15:15:02 -08:00
Sage Weil
9323ab29cd Merge pull request #3000 from yuyuyu101/wip-10166
Wip 10166: fiemap or FileStore::do_sparse_copy_range bug

sage-2014-12-16_08:43:28-powercycle-wip-sage-testing-distr

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 15:14:20 -08:00
Sage Weil
47f0a52e85 Merge branch 'wyc-quota-v2' of git://github.com/kylinstorage/ceph into wip-fs-quota
Conflicts:
	src/include/ceph_features.h

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
2014-12-16 14:29:39 -08:00
Sage Weil
f8d37a2cc4 Merge remote-tracking branch 'gh/master' into wip-watch-notify
Conflicts:
	PendingReleaseNotes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh During <jdurgin@redhat.com>
2014-12-16 14:02:32 -08:00
Samuel Just
868d7496f5 Merge pull request #3033 from dachary/wip-10201-osd-scrub-repair
osd scrub repair uses ceph_objecstore_tool + shell toolbox

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-16 13:54:11 -08:00
Josh Durgin
9d61dec011 Merge pull request #3181 from ceph/wip-10299
librbd: complete all pending aio ops prior to closing image

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2014-12-16 13:44:21 -08:00
John Spray
9f53eeb88d doc: add cephfs ENOSPC and eviction information
Adding this at this time to give us a sensible place
to talk about the epoch barrier stuff.  The eviction
stuff will probably get simplified once we add a mon-side
eviction command that handles blacklisting and MDS session
eviction in one go.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00
John Spray
a8babcbaa9 client: add 'status' asok
Useful in test/debug for querying OSD epoch
and OSD barrier epoch.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00
John Spray
6fdf890355 client: propagate flush errors to fclose/fsync
Ensure that all calls to _flush get an explicit
context (rather than implicit PutInode used
before), which tags any error onto the Inode object.

Later, read the error from the Inode object in _release_fh
and ensure all callers of _release_fh properly handle
its return code.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00