Commit Graph

33907 Commits

Author SHA1 Message Date
John Spray
200d0ae9c6 ceph-disk: Enable creating multiple osds per dev
This is useful for demos and testing.  Enables
creation of lots of OSDs on a single block device
by simply running ceph-disk prepare more than once,
with a --data-size argument set.

Signed-off-by: John Spray <john.spray@inktank.com>
2014-05-23 18:38:21 +01:00
Yehuda Sadeh
2ceb13af34 rgw: check appropriate entity permission on put_metadata
Fixes: #8428
Backport: firefly

Cannot use verify_object_permission() to test acls, as the operation
here might either be on object or on bucket.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-05-23 09:40:17 -07:00
Loic Dachary
ab0db34eaa documentation: adapt PG formula for erasure coded pools
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-05-23 07:25:37 +02:00
Kai Zhang
d3af8fa880 Update architecture.rst:Calculating PG IDs
To Calculate PG ID, if I didn't get it wrong, CRUSH calculates the hash modulo 
the number of PGs instead of OSDs, according to osd/osd_types.cc:963 
ceph_stable_mod(pg.ps(), pg_num, pg_num_mask).

Signed-off-by: Kai Zhang <zakir.exe@gmail.com>
2014-05-22 17:37:16 -07:00
Samuel Just
04f8e0effa Merge pull request #1854 from ceph/wip-java-build
cephfs-java: build against older jni headers

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-05-22 14:34:53 -07:00
Loic Dachary
ddb580b100 Merge pull request #1851 from FlorentFlament/master
documentation: update osd pool default size from 2 to 3.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-05-22 16:19:15 +02:00
Florent Flament
99b9682790 documentation: update osd pool default size from 2 to 3.
Fixes: #7725

Signed-off-by: Florent Flament <florent.flament@cloudwatt.com>
2014-05-22 14:14:56 +02:00
Loic Dachary
4a29c6a167 Merge pull request #1815 from ceph/osd-metadata
osd: include osd_objectstore in metadata reported to mon

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-05-22 10:58:07 +02:00
Gregory Farnum
7b3457d5b8 Merge pull request #1793 from ceph/wip-multimds
Speed up several cross-MDS operations by reducing the number of two-phase commit disk accesses we have to go through.

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-05-21 22:53:14 -07:00
Greg Farnum
4d4b77e5b6 cephfs-java: build against older jni headers
Older versions of the JNI interface expected non-const parameters
to their memory move functions. It's unpleasant, but won't actually
change the memory in question, to do a cast_const in order to satisfy
those older headers. (And even if it *did* modify the memory, that
would be okay given our single user.)

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-05-21 21:41:26 -07:00
Sage Weil
f81c53a716 Merge pull request #1838 from ceph/wip-client-time
fs: include client timestamp in all mds requests

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-05-21 17:29:52 -07:00
Sage Weil
596d41182e Merge pull request #1835 from ceph/wip-8332
HashIndex: in cleanup, interpret missing dir as completed merge

Reviewed-by: Sage Weil <sage@inktank.com>
2014-05-21 13:20:45 -07:00
Sage Weil
5d8442a109 Merge pull request #1834 from ceph/wip-8334
Wip 8334

Reviewed-by: Sage Weil <sage@inktank.com>
2014-05-21 13:19:54 -07:00
Sage Weil
6069ff0899 doc/release-notes: v0.67.9
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-21 13:00:35 -07:00
Sage Weil
df0086c031 Merge pull request #1848 from yuyuyu101/librbd-flush
Avoid extra check for clean object

Backport: firefly, dumpling
Reviewed-by: Sage Weil <sage@inktank.com>
2014-05-21 10:22:35 -07:00
Sage Weil
7784ef178d Merge pull request #1849 from ceph/wip-mon-get-version
mon: set MMonGetVersionReply tid

Reviewed-by: Sage Weil <sage@inktank.com>
2014-05-21 08:40:16 -07:00
Haomai Wang
f51e33bd9c Avoid extra check for clean object
We needn't to check clean object via buffer state, skip the clean object.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-05-21 18:14:49 +08:00
Ilya Dryomov
9235dcb1ef mon: set MMonGetVersionReply tid
Currently we don't set MMonGetVersionReply tid even if the original
MMonGetVersion message had a non-zero tid.  This is bad for the kernel
client, which has the infrastructure in place that relies on tids to
lookup message buffers and contexts.  To kick off transitioning away
from the workaround, set MMonGetVersionReply tid to the tid of the
original MMonGetVersion message.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-05-21 12:20:51 +04:00
Sage Weil
ba53889a33 README: move package dependencies into separate files
Sorted, with a short, simple command to paste.  Avoid the duplication of
the lists.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 21:56:05 -07:00
Sage Weil
77066a2396 README: clean up some whitespace
Remove $ prefix, use tab consistently.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 21:55:18 -07:00
Haomai Wang
c08adbc98f Fix set_alloc_hint op cause KeyValueStore crash problem
Now KeyValueStore doesn't support set_alloc_hit op, the implementation of
_do_transaction need to consider decoding the arguments. Otherwise, the
arguments will be regarded as the next op.

Fix the same problem for MemStore.

Fix #8381

Reported-by: Xinxin Shu <xinxin.shu5040@gmail.com>
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-05-21 12:51:13 +08:00
Sage Weil
e5fb75fc84 Merge pull request #1723 from ctheune/patch-1
Fix formatting of header
2014-05-20 20:22:55 -07:00
John Wilkins
e9eb30c066 Merge pull request #1845 from wdneto/master
doc: Update manual-deployment.rst
2014-05-20 17:40:29 -07:00
John Wilkins
08f72d561b Merge pull request #1828 from nereocystis/faq-fix
doc: fix link to FAQ
2014-05-20 17:38:11 -07:00
John Wilkins
a147b46a75 Merge pull request #1829 from nereocystis/preflight-cleanup
doc: Clean up pre-flight documentation
2014-05-20 17:37:56 -07:00
John Wilkins
7564df85f8 Merge pull request #1833 from nereocystis/admin-node-doc
doc: mention admin-node in common documentation
2014-05-20 17:37:07 -07:00
John Wilkins
aca9b1edfd Merge pull request #1837 from nereocystis/quick-ceph-deploy-cleanup
doc: quick-ceph-deploy cleanup
2014-05-20 17:35:56 -07:00
Sage Weil
ec9a9dac1e Merge pull request #1841 from ceph/wip-8373
Fixes for crush ruleset selection during pool creation (Wip 8373)

Reviewed-by: Sage Weil <sage@inktank.com>
2014-05-20 17:10:39 -07:00
John Wilkins
4523e58120 Merge pull request #1836 from ceph/wip-doc-openstack
doc: Clarifications to Ceph Block Device and OpenStack documentation.
2014-05-20 17:06:43 -07:00
Sage Weil
d04b386bdf init-ceph: if we fail to set an OSD's crush position, continue
If we fail to set the CRUSH position for one OSD, continue on to try
starting others, just as we do when we fail to start the daemon.

Fixes: #8342
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 16:29:23 -07:00
Sage Weil
46f1eb6865 init-ceph: continue loop immediately on daemon start failure
If we fail to start the daemon, skip the post_start and other steps.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 16:28:31 -07:00
Sage Weil
f305b8a3bd Merge pull request #1842 from ceph/wip-buildroot
ceph.spec.in: remove BuildRoot
2014-05-20 15:27:26 -07:00
Sage Weil
7e695318ec Merge pull request #1844 from onlyjob/hppa
Define AO_REQUIRE_CAS (fixes FTBFS on 'hppa')

Backport: firefly
Reviewed-by: Sage Weil <sage@inktank.com>
2014-05-20 15:26:56 -07:00
Aristoteles Neto
c2644b29e8 Update manual-deployment.rst
- When creating the OSD data, specify osd-uuid so that it matches when the osd is first created.
- Modify caps when adding osd auth to match what ceph-deploy does.
2014-05-21 10:20:55 +12:00
Dmitry Smirnov
74218f3d6c Define AO_REQUIRE_CAS (fixes FTBFS on 'hppa')
to fix FTBFS due to undeclared atomic functions.

As reported

    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=748571

by John David Anglin <dave.anglin@bell.net>

~~~~
./include/atomic.h: In member function 'size_t ceph::atomic_t::inc()':
./include/atomic.h:42:36: error: 'AO_fetch_and_add1' was not declared in this scope
       return AO_fetch_and_add1(&val) + 1;
                                    ^
./include/atomic.h: In member function 'size_t ceph::atomic_t::dec()':
./include/atomic.h:45:42: error: 'AO_fetch_and_sub1_write' was not declared in this scope
       return AO_fetch_and_sub1_write(&val) - 1;
                                          ^
./include/atomic.h: In member function 'void ceph::atomic_t::add(size_t)':
./include/atomic.h:48:36: error: 'AO_fetch_and_add' was not declared in this scope
       AO_fetch_and_add(&val, add_me);
                                    ^
./include/atomic.h: In member function 'void ceph::atomic_t::sub(int)':
./include/atomic.h:52:48: error: 'AO_fetch_and_add_write' was not declared in this scope
       AO_fetch_and_add_write(&val, (AO_t)negsub);
                                                ^
./include/atomic.h: In member function 'size_t ceph::atomic_t::dec()':
./include/atomic.h:46:5: warning: control reaches end of non-void function [-Wreturn-type]
     }
     ^
make[5]: *** [cls/user/cls_user_client.o] Error 1
~~~~

Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
2014-05-21 08:18:12 +10:00
John Spray
fb504baed9 mon: Fix check of ruleset id on pool update
This code was using CrushWrapper::rule_exists, which
checks for a *rule* existing, whereas the value being
set is a *ruleset*.

Signed-off-by: John Spray <john.spray@inktank.com>
2014-05-20 23:16:54 +01:00
John Spray
1d9e4ac2e2 mon: Fix default replicated pool ruleset choice
Specifically, in the case where the configured
default ruleset is CEPH_DEFAULT_CRUSH_REPLICATED_RULESET,
instead of assuming ruleset 0 exists, choose the lowest
numbered ruleset.

In the case where an explicit ruleset is passed to
OSDMonitor::prepare_pool_crush_ruleset, verify
that it really exists.

The idea is to eliminate cases where a pool could
exist with its crush ruleset set to something
other than a value ruleset ID.

Fixes: #8373

Signed-off-by: John Spray <john.spray@inktank.com>
2014-05-20 23:16:54 +01:00
Sage Weil
799f76aa3f mds: remove unused Mutation::now
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 15:07:10 -07:00
Sage Weil
8768857061 mds: use mds_stamp for mksnap
Use the server timestamp for the snapshot timestamp.  This could arguably
be the client timestamp, but I think snapshot creation times are a bit
more important to have accurate timestamps on, and this should not be
something that existing client apps will strongly depend on.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 15:07:07 -07:00
Sage Weil
d4bfa39fd3 mds: reset mds_stamp for readdir, rename, link
These ops to complicated work prior to starting the real operation, like
fetching missing directories, or opening remote dirfrags, creating
snaprealms.  Reset the mds timestamp after this slow work has completed.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 15:05:28 -07:00
Sage Weil
a09547bbfc mds: use real timestamp for readdir leases
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 15:05:28 -07:00
Sage Weil
9c86a110f8 mds: use client-provided time stamp for user-visible file metadata
Use the op_stamp from the MDRequest, populated by the MClientRequest when
possible, for setting timestamps on user-visible metadata (like ctime,
mtime).

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 15:05:28 -07:00
Sage Weil
4a6596ec61 mds: do the balancer and propagation timers based on mds_stmap
Use the (more) stable time stamp for doing the balancer's temperature
tracking and associated decay.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 15:05:27 -07:00
Sage Weil
fd1f9bdada mds: do rstat timestamps (rctime, fragstat mtime) in terms of op stamp
Use the op (client) timestamp for the recursive stats, for santity's sake.
Note that since this is monotonically increasing, the danger here is
that we lose track of nested changes due to skewed client clocks.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 14:55:05 -07:00
Sage Weil
29c684442e mds: make sure mds_stamp is set when we journal
This is a catch-all that we are carrying over from before.  It may not
be strictly necessary, but I'm not inclined to check the code for
Mutation users who didn't call acquire_locks().

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 14:52:59 -07:00
Sage Weil
3569e809a4 mds: set mds_stamp on lock acquisition
Normally acquire_locks() will set this for us.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 14:51:35 -07:00
Sage Weil
e4c9c34136 mds: add {mds,op}_stamp to Mutation
With getters, setters.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 14:50:46 -07:00
Sage Weil
401319a152 ceph.spec.in: remove BuildRoot
Deprecated

Fixes: #8143
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-20 13:41:35 -07:00
John Spray
93a5b88fcc tools/cephfs: error handling in EventOutput
Print error instead of hitting assertion if
we e.g. can't create directory.

Signed-off-by: John Spray <john.spray@inktank.com>
2014-05-20 14:43:55 +01:00
John Spray
3207c50507 osdc/Journaler: fix obj count in Journaler:erase
This was off by one (too few) in the case of a
trimmedpos->write_pos range that had length
layout_period+2 and starting position one byte
before a period boundary.

Signed-off-by: John Spray <john.spray@inktank.com>
2014-05-20 14:43:54 +01:00