Commit Graph

37795 Commits

Author SHA1 Message Date
John Wilkins
3450220be5 Merge pull request #3201 from nilamdyuti/wip-doc-ceph
doc: Changes format style in ceph to improve readability as html.

Reviewed-by: John Wilkins <jowilkin@redhat.com>
2014-12-18 15:00:37 -08:00
Sage Weil
5439d9ee70 Merge pull request #3057 from ceph/wip-crush-straw
crush: fix straw bucket scaler bugs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2014-12-18 14:13:33 -08:00
Sage Weil
2a1bd761e9 .gitmodules: update ceph-object-corpus auth repo to github
It's annoying having ceph.com for only this repo, and it's not that big.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-18 14:09:21 -08:00
Sage Weil
e3f370f246 Merge pull request #3101 from yuyuyu101/wip-10147
Messenger Unit Tests

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-18 14:05:16 -08:00
Gregory Farnum
cc504694bf Merge pull request #3205 from ceph/wip-caps-init
messages/MClientCaps: init peer.flags

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2014-12-18 11:36:49 -08:00
Samuel Just
a2112fd072 Merge pull request #3123 from linuxbox2/wip-claim-3
Wip claim 3

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-18 11:23:23 -08:00
Samuel Just
2474a7854c Merge pull request #3111 from yuyuyu101/wip-10193
WBThrottle: make bytes/ios/inode_wb's perf counter effective

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-18 11:21:28 -08:00
Samuel Just
0a6d5aef16 Merge pull request #3133 from XinzeChi/wip-2
OSD: FileJournal: call writeq_cond.Signal if necessary in submit_entry

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-18 11:20:24 -08:00
Samuel Just
a9aaea7709 Merge pull request #3132 from mslovy/optimazation_collection_map
os: IndexManager:: optimize map<coll_t, CollectionIndex* > col_indices to unordered_map

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-18 11:19:34 -08:00
Loic Dachary
916a211aac Merge pull request #3204 from ceph/wip-osdmap-leak
osd: fix leaked OSDMap

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-18 19:32:41 +01:00
Haomai Wang
1f9c087421 AsyncConnection: Fix time event is called after AsyncMessenger destruction
AsyncConnection uses time event to handle async event partially, but it's not
very effective so here use dispatch_event_external instead.

And if client try to connect to server after a period, it may be call
AsyncConnection::process which will reference to AsyncMessenger. Since
AsyncMessenger doesn't use reference count, it will result in segment
fault. Now we record time event id and delete these registered time events
when stopping connection.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-18 23:36:27 +08:00
Haomai Wang
20ea086722 PipeConnection: Avoid deadlock when calling is_connected
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-18 22:51:33 +08:00
Haomai Wang
9783a5cb64 test/msgr/test_msgr: Fix potential unsafe cond wakeup and wrap check
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-18 22:50:57 +08:00
Sage Weil
2cd9dc0804 messages/MClientCaps: init peer.flags
*** CID 1258789:  Uninitialized scalar field  (UNINIT_CTOR)
/messages/MClientCaps.h: 132 in MClientCaps::MClientCaps(int, inodeno_t,
inodeno_t, unsigned long, int, unsigned int)()
126         head.ino = ino;
127         head.realm = realm;
128         head.cap_id = id;
129         head.migrate_seq = mseq;
130         peer.cap_id = 0;
131         inline_version = 0;
>>>     CID 1258789:  Uninitialized scalar field  (UNINIT_CTOR)
>>>     Non-static class member field "peer.flags" is not initialized in this
constructor nor in any functions that it calls.
132       }
133     private:
134       ~MClientCaps() {}
135
136     public:
137       const char *get_type_name() const { return "Cfcap";}

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-18 06:29:40 -08:00
Sage Weil
679652ab36 osd: fix leaked OSDMap
*** CID 1258787:  Resource leak  (RESOURCE_LEAK)
/osd/OSD.cc: 6118 in OSD::handle_osd_map(MOSDMap *)()
6112                    << dendl;
6113            clog->warn() << "failed to encode map e" << e << " with expecte
crc\n";
6114            MMonGetOSDMap *req = new MMonGetOSDMap;
6115            req->request_full(e, last);
6116            monc->send_mon_message(req);
6117            last = e - 1;
>>>     CID 1258787:  Resource leak  (RESOURCE_LEAK)
>>>     Variable "o" going out of scope leaks the storage it points to.
6118            break;
6119           }
6120
6121
6122           hobject_t fulloid = get_osdmap_pobject_name(e);
6123           t.write(META_COLL, fulloid, 0, fbl.length(), fbl);

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-18 06:24:49 -08:00
Haomai Wang
001ea29386 Messenger: Create an Messenger implementation by name.
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-18 21:51:12 +08:00
Yan, Zheng
2d76bc2d98 Merge pull request #3202 from ceph/wip-fix-asok
mds: fix asok on rank 0
2014-12-18 20:22:40 +08:00
John Spray
3a2cb7148a mds: fix asok on rank 0
Broken by:
1b2b34431 (MDS: do not allow invocation of most commands on an inactive MDS)

MDS::whoami is a rank, so check for inactive is <0, not <=0

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-18 12:16:13 +00:00
Nilamdyuti Goswami
8de9a0f437 doc: Adds updated man page for ceph under man/
Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
2014-12-18 17:13:27 +05:30
Nilamdyuti Goswami
8b79617306 doc: Changes format style in ceph to improve readability as html.
Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>
2014-12-18 17:11:22 +05:30
Wido den Hollander
883e1aea96 doc: Show how new data pools can be added to CephFS for file layouts 2014-12-18 10:59:31 +01:00
Yan, Zheng
bfa4acf26e Merge pull request #3198 from ceph/wip-10335
MDS: do not allow invocation of most commands on an inactive MDS
2014-12-18 10:32:31 +08:00
Greg Farnum
1b2b344313 MDS: do not allow invocation of most commands on an inactive MDS
They just cause all kinds of trouble!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-12-17 15:32:23 -08:00
Gregory Farnum
be7b2f8a30 Merge pull request #2836 from ceph/wip-7317
#7317 ENOSPC and OSD epoch barrier handling

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2014-12-17 14:12:51 -08:00
Loic Dachary
65be519b03 Merge pull request #3197 from ktdreyer/rpm-spec-comment
ceph.spec.in: quote %files macro in comment

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-17 22:00:53 +01:00
Ken Dreyer
69fa5329ce ceph.spec.in: quote %files macro in comment
Prior to this commit, RPM would expand the %files macro that was present
in the comment.

Use a double percent sign to quote the macro so that RPM will not expand
it.

This fixes an rpmlint warning, "W: macro-in-comment %files"

More information from rpmlint's "-I" (help) command:

  $ rpmlint -I macro-in-comment
  macro-in-comment:
  There is a unescaped macro after a shell style comment in the
  specfile.  Macros are expanded everywhere, so check if it can cause a
  problem in this case and escape the macro with another leading % if
  appropriate.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2014-12-17 13:06:58 -07:00
Loic Dachary
38768da1b6 Merge pull request #3171 from tchaikov/refactor-rados-class
python-rados: refactor class Rados a little bit

Reviewed-by: Josh During <jdurgin@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-17 19:32:34 +01:00
Kefu Chai
378ebb79d0 python-rados: refactor class Rados a little bit
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2014-12-18 02:05:32 +08:00
Wido den Hollander
0c9d55d404 pybind/cephfs: Fix setxattr function. Pass value as character pointer
This passes the value of the xattr correctly and makes it work.
2014-12-17 14:01:44 +01:00
Loic Dachary
9b73c76209 Merge pull request #3190 from dachary/wip-test-helpers
tests: ignore xmlstarlet extra empty lines
2014-12-17 09:00:42 +01:00
Loic Dachary
023a8aca8c tests: ignore xmlstarlet extra empty lines
With xmlstarlet 1.3.0 (which is on Ubuntu precise)

xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and
contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n

will output an extra empty line which does not show on xmlstarlet
1.5.0 (which is on Ubuntu trusty and Fedora 20).

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2014-12-17 08:02:17 +01:00
Sage Weil
242e28d13a Merge pull request #2530 from majianpeng/fiemap
Fiemap

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-16 16:50:45 -08:00
Sage Weil
ca1b216b47 Merge pull request #2137 from yuyuyu101/pg-sparse-recovery
ReplicatedPG: Make recovery using fiemap and fix perf counter

sage-2014-12-16_08:43:28-powercycle-wip-sage-testing-distro

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 15:15:02 -08:00
Sage Weil
9323ab29cd Merge pull request #3000 from yuyuyu101/wip-10166
Wip 10166: fiemap or FileStore::do_sparse_copy_range bug

sage-2014-12-16_08:43:28-powercycle-wip-sage-testing-distr

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-16 15:14:20 -08:00
Sage Weil
f8d37a2cc4 Merge remote-tracking branch 'gh/master' into wip-watch-notify
Conflicts:
	PendingReleaseNotes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh During <jdurgin@redhat.com>
2014-12-16 14:02:32 -08:00
Samuel Just
868d7496f5 Merge pull request #3033 from dachary/wip-10201-osd-scrub-repair
osd scrub repair uses ceph_objecstore_tool + shell toolbox

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-16 13:54:11 -08:00
Josh Durgin
9d61dec011 Merge pull request #3181 from ceph/wip-10299
librbd: complete all pending aio ops prior to closing image

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2014-12-16 13:44:21 -08:00
John Spray
9f53eeb88d doc: add cephfs ENOSPC and eviction information
Adding this at this time to give us a sensible place
to talk about the epoch barrier stuff.  The eviction
stuff will probably get simplified once we add a mon-side
eviction command that handles blacklisting and MDS session
eviction in one go.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00
John Spray
a8babcbaa9 client: add 'status' asok
Useful in test/debug for querying OSD epoch
and OSD barrier epoch.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00
John Spray
6fdf890355 client: propagate flush errors to fclose/fsync
Ensure that all calls to _flush get an explicit
context (rather than implicit PutInode used
before), which tags any error onto the Inode object.

Later, read the error from the Inode object in _release_fh
and ensure all callers of _release_fh properly handle
its return code.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00
John Spray
a8611ace59 client: handle ENOSPC in _flush
Previously we would avoid issuing writes
from _write, but the same condition was
not applied to _flush.  This was okay
when we were blocking on ENOSPC, but now
we're cancelling ops on ENOSPC so need
to make sure we don't issue any new ones.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:55:24 +00:00
John Spray
cdf679be3c mds: add 'osdmap barrier' admin command
This is for use with client eviction.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
fcc64f3849 mds: set epoch barrier on transition to active
To handle case where MDSs restart after experiencing
a barrier-inducing operation: rather than persisting
OSD barrier somewhere, just have the MDSs always barrier
on latest OSD epoch at startup.

Effect is that after restart, MDS cap issues will
be delayed in (compliant) clients until the client
sees the latest OSD map.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
40503ffbd7 mds: implement osd epoch barrier on caps
Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
2d993c4b2b messages: always encode barrier
Instead of truncating on INLINE_DATA, send
some blank fields there for clients that
don't support it (the kclient) in order
that we can include the following
epoch_barrier field.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
68ba7f5838 messages: add osd_epoch_barrier to cap msgs
Extension to client-server protocol to allow clients
to release capabilities conditional on the receiver
having a particular OSD map, and the MDS to issue
caps conditional on the user having a particular
OSD map.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
fc7d62a17c mds: return ENOSPC on write ops while osds full
Allow removals and read-only ops, prevent others.  This
is a soft policy aimed at reducing the likelihood of a
"full" (mon_osd_full_ratio) OSD becoming physically full
(where it is unable to accept journal writes from MDS
even though it does not respect the FULL flag)

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
09287fd666 osdc: add public Objecter::osdmap_full_flag
For the (now numerous) places where callers
would like to check this without doing a verbose
lock/unlock cycle.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:51:49 +00:00
John Spray
bab3d3d414 osdc: remove spurious _maybe_request_map return value
This was always zero: make the function return void, and
remove all the places we asserted on its return value.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:45:58 +00:00
John Spray
cf3101af8f osdc: implement Objecter::set_epoch_barrier
For use in MDS and Client when they want to wait
for a particular OSDMap epoch before using caps
that depend on a prior blacklist or op cancellation.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-16 20:45:58 +00:00