Commit Graph

34918 Commits

Author SHA1 Message Date
Sage Weil
d7fb7bf5f2 ceph_test_rados_api_tier: fix cache cleanup (ec too)
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-10 12:48:29 -07:00
Sage Weil
ebbe8aab17 ceph_test_rados_api: fix cleanup of cache pool
We can't simply try to delete everything in there because some items may
be whiteouts.  Instead, flush+evict everything, then remove overlay, and
*then* delete what remains.

Fixes: #9055
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-10 12:48:13 -07:00
Sage Weil
1d199fb117 librados/TestCase: inheret cleanup_default_namespace
No need to duplicate this code.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-10 11:41:23 -07:00
Loic Dachary
5808d6a6a5 osd: improve ruleno/ruleset consistency
rename variables to reflect when they hold a ruleset and when they hold
a rule id / ruleno in the hope to avoid bugs in the future.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-10 17:18:30 +02:00
Loic Dachary
3f5d86af72 erasure-code: ErasureCodeIsa::create_ruleset must return a ruleset
http://tracker.ceph.com/issues/9044 Refs: #9044

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-10 17:18:30 +02:00
Loic Dachary
0029a35872 erasure-code: ErasureCodeJerasure::create_ruleset must return a ruleset
CrushWrapper::add_simple_ruleset does not return a ruleset, it returns
a ruleid that must be converted into a ruleset before being returned.

http://tracker.ceph.com/issues/9044 Fixes: #9044

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-10 17:18:30 +02:00
Loic Dachary
04a484a4d5 erasure-code: OSDMonitor::crush_ruleset_create_erasure needs ruleset
When OSDMonitor::crush_ruleset_create_erasure checks the ruleset for
existence, it must convert the ruleid into a ruleset before assigning it
back to the *ruleset parameter.

http://tracker.ceph.com/issues/9044 Fixes: #9044

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-10 17:18:30 +02:00
Loic Dachary
104440ce62 Merge pull request #2201 from dachary/wip-8496-erasure-code-base-class
erasure code base class

Reviewed-by: Andreas-Joachim Peters <Andreas.Joachim.Peters@cern.ch>
2014-08-10 09:51:31 +02:00
Sage Weil
4e8de17927 os/FileStore: dump open fds before asserting
Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-09 13:23:06 -07:00
Kevin Cox
281159bf3b doc: Initial network docs.
Signed-Off-By: Kevin Cox <kevincox@kevincox.ca>
2014-08-09 15:52:38 -04:00
Ma Jianpeng
4f6d513366 os/chain_xattr: Remove all old xattr entry when overwrite the xattr.
Ceph use multiple xattrs to store the value of a single xattr which size
is larger than CHAIN_XATTR_MAX_BLOCK_LEN.
But when overwote the content of xattr in func
chain_setxattr/chain_fsetxattr, we don't know the size of previous
content of the xattr.
So we only try to remove until system return  -ENODATA.

Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
2014-08-09 20:03:14 +08:00
Sage Weil
4d2d4dd56e Merge branch 'next'
Conflicts:
	src/osd/OSD.cc
2014-08-08 23:00:13 -07:00
Greg Farnum
84c9654704 OSD: add require_same_peer_inst(OpRequestRef&,OSDMap&) helper
Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit e99acf9810)
2014-08-08 22:59:25 -07:00
Greg Farnum
4a0c9417c1 OSD: introduce require_self_aliveness(OpRequestRef&,epoch_t) function
Take the self-aliveness checks out of require_same_or_newer_map() and use
the new function for that and for require_up_osd_peer().

Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit e179e9227b)
2014-08-08 22:59:17 -07:00
Greg Farnum
48c9b38414 OSD: use OpRequestRef& for a few require_* functions
Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit eb2f1ea2c3)
2014-08-08 22:59:14 -07:00
Greg Farnum
f86bf1d8be OSD: introduce require_up_osd_peer() function for gating replica ops
This checks both that a Message originates from an OSD, and that the OSD
is up in the given map epoch.
We use it in handle_replica_op so that we don't inadvertently add operations
from down peers, who might or might not know it.

Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit ccd0eec501)
2014-08-08 22:59:09 -07:00
Sage Weil
65706b113a Merge remote-tracking branch 'gh/next' 2014-08-08 20:08:19 -07:00
Sage Weil
790f4c2345 Merge pull request #2228 from ceph/wip-pg-epoch
osd: fix pg epoch floor tracking

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-08 18:17:27 -07:00
Sage Weil
448edb191c Merge pull request #2209 from somnathr/wip-sd-check-pool-existence
RadosClient: Adding pool existence checks for pool create interfaces.

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-08 17:53:54 -07:00
Pavan Rallabhandi
e4b00718ef RadosClient: Enable pool existence check
Pool create interfaces would now refer to updated osdmap
for pool existence, such that an appropriate message is
thrown to the user when run via CLI.

Fixes: #8835
Signed-off-by: Pavan Rallabhandi <pavan.rallabhandi@sandisk.com>
2014-08-08 16:57:56 -07:00
Ilya Dryomov
c34f935284 test_librbd_fsx: also flatten as part of randomize_parent_overlap
With randomize_parent_overlap fsx will randomly truncate base images
after they have been cloned from.  This throws flatten into the mix:
base image will be flattened with 2/16 chance (equal to the chance of
leaving the image intact).

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-08-08 18:53:14 +04:00
Wido den Hollander
214630b15b mds: Also check min/max uid on snap removal
Not only check if the uid is correct when creating a snapshot, but also
verify if the UID is correct when trying to remove a snapshot.

Fixes: #9029
2014-08-08 12:03:56 +02:00
Sage Weil
a52a855f6c osd: fix pg epoch floor tracking
If you call erase() on a multiset it will delete all instances of a value;
we only want to delete one of them.  Fix this by passing an iterator.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-07 17:42:06 -07:00
Samuel Just
de0c7202a0 Merge pull request #2226 from athanatos/wip-8396
Wip 8396

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-07 15:56:18 -07:00
Samuel Just
2120f4bb6c OSD: move waiting_for_pg into the session structures
Each message belongs to a session.  Further, no ordering is implied
between messages which arrived on different sessions.  Breaking the
global waiting_for_pg structure into a per-session structure lets
us avoid the problem of taking a write lock on a global structure
(pg_map_lock) in get_pg_or_queue_for_pg at the cost of some
complexity in updating each session's waiting_for_pg structure when
we receive a new map (due to pg splits) or when we locally create
a pg.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 15:54:38 -07:00
Sage Weil
f5d6818d37 Merge pull request #2227 from athanatos/wip-8625
osd_types: s/stashed/rollback_info_completed and set on create

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-07 15:42:27 -07:00
John Wilkins
71cb4d3c0d doc: Removed Debian reference from Upstart.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-08-07 15:30:43 -07:00
John Wilkins
8357cae027 doc: Incorporated user feedback to clarify upgrade doc.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-08-07 15:29:34 -07:00
John Wilkins
b6bf33cd33 doc: Added configuration discussion at end of gateway install with links to config.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-08-07 14:15:45 -07:00
Samuel Just
11cefcab23 OSD::shutdown: actually drop sessions waiting on map
There might be messages for which we still don't have the
map.  Dispatching waiting won't actually help.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 11:48:54 -07:00
Samuel Just
6aba942170 OSD: clear_session_waiting_on_map in ms_handle_reset
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 11:48:54 -07:00
Samuel Just
11d6e5ad84 OSD: rename session_waiting_for_map_lock to session_waiting_lock
This lock will also protect the waiting_for_pg structures in each
session.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 11:48:54 -07:00
Samuel Just
20fd714a08 OSD: wake_pg_waiters outside of the pgmap write_lock, pg_lock
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 11:48:54 -07:00
Samuel Just
6e7b86a24f OSD: fix wake_pg_waiters revert error in _open_lock_pg
231fe1b685 reintroduced erroneously
this call to wake_pg_waiters.  All _create_lock_pg callers handle
calling wake_pg_waiters after the pg lock has been dropped.

Fixes: #8691
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 11:48:54 -07:00
Samuel Just
4260767973 osd_types: s/stashed/rollback_info_completed and set on create
Originally, this flag indicated that the object had already been stashed and
that therefore recording subsequent changes is unecessary.  We want to set it
on create() as well since operations like [create, writefull] should not need
to stash the object.

Fixes: #8625
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-07 11:47:52 -07:00
Sage Weil
f95e82b134 Merge remote-tracking branch 'gh/wip-filestore-bigxattr' 2014-08-07 11:21:06 -07:00
Alfredo Deza
d0ccb1cbef make ceph-disk use the new init flag for cluster
Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
2014-08-07 11:11:57 -04:00
Alfredo Deza
23b4915fa2 allow passing a --cluster flag to the init script
It will fallback to looking at /etc/ceph/$cluster.conf
otherwise.

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
2014-08-07 10:48:09 -04:00
Wido den Hollander
6bd2b0fc6e mds: Make min/max UID configurable for who is allowed to create a snapshot
This way administrators can configure which UIDs are allowed to create
snapshots on CephFS.

In certain situations it might be desirable that regular users are not allowed
to create snapshots. For example on shared hosting environments.

Fixes: #9029
2014-08-07 15:24:59 +02:00
Wido den Hollander
a5ecf1515e powerdns: Update README with better markdown 2014-08-07 15:12:19 +02:00
Sage Weil
9bbd11497b Merge pull request #2220 from somnathr/wip-lock-leak-fix
RadosClient: Fixing potential lock leaks.

Backport: firefly
Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-06 18:42:18 -07:00
Pavan Rallabhandi
f1aad8bcfc RadosClient: Fixing potential lock leaks.
In lookup_pool and pool_delete, a lock is taken
before invoking wait_for_osdmap, but is not
released for the failure case of the call. Fixing the same.

Fixes: #9022
Signed-off-by: Pavan Rallabhandi <pavan.rallabhandi@sandisk.com>
2014-08-06 18:30:01 -07:00
Sage Weil
26750fcfe8 os/FileStore: force any new xattr into omap on E2BIG
If we have a huge xattr (or many little ones), the _fgetattrs() for the
inline_set will fail with E2BIG.  The conditions later where we decide
whether to clean up the old xattr will then also fail.  We *will* put
the xattr in omap, but the non-omap version isn't cleaned up.

Fix this by setting a flag if we get E2BIG that the inline_set is known
to be incomplete.  In that case, take the conservative step of assuming
the xattr might be present and chain_fremovexattr().  Ignore any error
because it might not be there.

This is clearly harmless in the general case because it won't be there.
If it is, we will hopefully remove enough xattrs that the E2BIG
condition will go away (usually by removing some really big chained
xattr).

See original bug #7779.  With this in place, we can repair objects in
the broken state if we know the rados attr(s) that are responsible.
Usually that is user.rgw.manifset, and a rados get + set of the attr
will repair things.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 17:28:45 -07:00
Yehuda Sadeh
d050cf6c82 Merge pull request #2218 from ceph/wip-rados-xattr
rados: fix get/setxattr commands up

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-06 17:11:44 -07:00
Sage Weil
cc3112efd6 rados: use STD{IN,OUT}_FILENO for magic values
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 17:06:26 -07:00
Sage Weil
e3819b6216 qa/workunits/rados/test_rados_tool: add a few xattr tests
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 17:06:26 -07:00
Sage Weil
645c28a108 rados: optionally read setxattr value from stdin
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 15:31:19 -07:00
Sage Weil
59a715a39c rados: don't add \n to getxattr
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 15:31:15 -07:00
Sage Weil
ea731ae142 qa/workunits/cephtool/test.sh: fix 'ceph df ...' tests
Broken by ee2dbdb0f5 and friends.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 13:16:49 -07:00
Alfredo Deza
bdf5f30010 use cluster name when checking the host
Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
2014-08-06 15:30:30 -04:00