Commit Graph

27986 Commits

Author SHA1 Message Date
Joao Eduardo Luis
ad1392f681 mon: Monitor: StoreConverter: clearer debug message on 'needs_conversion()'
The previous debug message outputted the function's name, as often our
functions do.  This was however a source of bewilderment, as users would
see those in logs and think their stores would need conversion.  Changing
this message is trivial enough and it will make ceph users happier log
readers.

Backport: cuttlefish
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-16 10:33:03 -07:00
Joao Eduardo Luis
e752c40c23 mon: Monitor: StoreConverter: sanitize 'store' pointer on init
We are supposed to have umount'ed the store and set the pointer to NULL.
We should not tolerate any other case on init().

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-16 10:32:14 -07:00
Joao Eduardo Luis
036e6739a4 mon: Monitor: do not reopen MonitorDBStore during conversion
We already open the store on ceph_mon.cc, before we start the conversion.
Given we are unable to reproduce this every time a conversion is triggered,
we are led to believe that this causes a race in leveldb that will lead
to 'store.db/LOCK' being locked upon the open this patch removes.

Regardless, reopening the db here is pointless as we already did it when
we reach Monitor::StoreConverter::convert().

Fixes: #5640
Backport: cuttlefish

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-16 10:31:46 -07:00
Gregory Farnum
38691e7f95 Merge pull request #438 from yehudasa/wip-rgw-next
Fix an issue with bucket placements and with listing on new installations.

Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-16 09:33:52 -07:00
Danny Al-Gaaf
d873c292b8 rgw/rgw_rest_s3.cc: fix dereferencing after deallocation
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 17:06:46 +02:00
Danny Al-Gaaf
e45a9bb16f rgw/rgw_op.cc: reduce scope of local 'ret' integer variable
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 17:06:40 +02:00
Danny Al-Gaaf
c484150b44 rgw/rgw_rados.cc: use static_cast instead of C-Style cast
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 17:05:54 +02:00
Danny Al-Gaaf
cc49d3fed0 rgw/rgw_rest.cc: fix malloc memory leak
Free with malloc allocated memory in error case before return.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 14:20:11 +02:00
Danny Al-Gaaf
7a2e8a6f19 rgw/rgw_bucket.cc: return result of function calls
Return the result of rgw_(un)link_bucket()/ from
RGWBucketMetadataHandler::put() to signal errors correctly to
the function caller.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 14:20:01 +02:00
Danny Al-Gaaf
7769b01dc8 rgw_bucket.cc: don't return bool if return value is int
Fix minor style issue: if the function return value is an integer
don't return bool values.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 14:19:46 +02:00
Danny Al-Gaaf
64b512c056 rgw/rgw_bucket.cc: remove unused local variable
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 14:18:51 +02:00
Danny Al-Gaaf
a937a12d4e rgw_admin.cc: fix same expression on both sides of '||'
Remove twice listed checks, get checks in alphabetical order,
added comment above the checks to point out to keep it in
alphabetical order to avoid double checks.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 14:18:46 +02:00
Danny Al-Gaaf
5ea4c5e1ca cls_rgw_client.cc: reduce scope of local variable
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-16 14:18:42 +02:00
Yehuda Sadeh
408014ee46 rgw: handle ENOENT when listing bucket metadata entries
Just return success (with an empty list)

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 18:43:56 -07:00
Yehuda Sadeh
eef4458279 rgw: fix bucket placement assignment
When we set bucket.instance meta, we need to set
the correct bucket placement to the bucket (according to
the specific placement rule). However, it might be that
bucket placement was never configured and we just go by
the defaults, using the old legacy pools selection.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 15:49:42 -07:00
Samuel Just
39e5a2a406 OSD: add config option for peering_wq batch size
Large peering_wq batch sizes may excessively delay
peering messages resulting in unreasonably long
peering.  This may speed up peering.

Backport: cuttlefish
Related: #5084
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-15 15:15:17 -07:00
Sage Weil
b46930c96c mon: make report pure json
Put the crc in the status string and drop the header and footer.  If users
want to capture it,

ceph report 2>&1 > foo.txt

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 14:29:14 -07:00
Sage Weil
40a8bbdc53 Merge remote-tracking branch 'gh/wip-mon-report' into next 2013-07-15 14:23:40 -07:00
Sage Weil
daf7672309 ceph: drop --threshold hack for 'pg dump_stuck'
We can live with the incompatibility here; the hack is currently
not working anyway (see #5623).

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-07-15 14:05:21 -07:00
Sage Weil
4282971d47 msg/Pipe: be a bit more explicit about encoding outgoing messages
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:48:07 -07:00
Sage Weil
314cf046b0 messages/MClientReconnect: clear data when encoding
The MClientReconnect puts everything in the data payload portion of
the message and nothing in the front portion.  That means that if the
message is resent (socket failure or something), the messenger thinks it
hasn't been encoded yet (front empty) and reencodes, which means
everything gets added (again) to the data portion.

Decoding keep decoding until it runs out of data, so the second copy
means we decode garbage snap realms, leading to the crash in bug

Clearing data each time around resolves the problem, although it does
mean we do the encoding work multiple times.  We could alternatively
(or also) stick some data in the front portion of the payload
(ignored), but that changes the wire protocol and I would rather not
do that.

Fixes: #4565
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-15 13:47:39 -07:00
Sage Weil
6a524c358c Merge pull request #436 from ceph/wip-mon-fixes
Wip mon fixes

Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-15 13:46:43 -07:00
Sage Weil
34f76bd915 mon: set forwarded message recv stamp
Set it to the stamp of the MForward that carried us.  One could argue
we really want the original receive stamp on the origin, but that is
not available to us, and this is better than nothing.

In particular, this gives 'ceph log ...' commands a timestamp when they
are forwarded via a peon.  The stamp is still between when the request
is sent and when it is committed/acked, so all is well from the
client's perspective.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:54 -07:00
Sage Weil
eac559f474 mon: drop win_election() _reset() kludge and strengthen assertions
This is only there for the benefit of win_standalone_election(), but it
doesn't need it, it clutters the code, and weakens our assertions.

Now the only win_election() callers are win_standalone_election() (which
is a single path that just did _reset()) and from the elector.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:54 -07:00
Sage Weil
c67d50b86b mon: set peon state to electing if other mons call an election
Previously we would call mon->reset() and set various flags (like
exited_quorum timestamp), but the state would remain PEON.  Make an
explicit join_election() callback and set the state there, and add
asserts in reset() (renamed to be private) so that we ensure all
callers are well-behaved.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:53 -07:00
Sage Weil
3ea984cd6a mon: once sync full is chosen, make sure we don't change our mind
It is possible for a sequence like:

 - probe
 - first probe reply has paxos trim that indicates a full sync is
   needed
 - start sync
 - clear store
 - something happens that makes us abort and bootstrap (e.g., the
   provider mon restarts
 - probe
 - first probe reply has older paxos trim bound and we call an election
 - on election completion, we crash because we have no data.

Non-determinism of the probe decision aside, we need to ensure that
the info we share during probe (fc, lc) is accurate, and that once we
clear the store we know we *must* do a full sync.

Fixes: #5621
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:53 -07:00
Sage Weil
1fc85acdb8 mon/PaxosService: consolidate resetting in restart()
We had duplicated code in election_finished() and restart(), and it was
incomplete.  Put it all in restart() only (the mon should have called
restart() long before the election finishes).  Note that we cannot
assert as much in election_finished() because another service may have
just cross-proposed.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:53 -07:00
Sage Weil
7666c33a1d mon/PaxosService: assert not proposing in propose_pending
Drop the useless active check after the assert, too.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:53 -07:00
Sage Weil
c711203c0d mon/Paxos: separate proposal commit from the end of the round
Each commit should match with exactly one proposal; finish it when we
actually commit it and make sensible asserts.

The old finish_proposal() turns into finish_round(), and performs
generic checks and cleanup associated with the transition from
updating -> active.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:53 -07:00
Sage Weil
5c31010795 mon/Paxos: make all handle_accept paths go via out label
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 13:42:53 -07:00
Sage Weil
2bf95e5a1c Merge branch 'next' 2013-07-15 13:20:32 -07:00
Sage Weil
f1ce8d7c95 mon: fix scrub vs paxos race: refresh on commit, not round completion
Consider:

 - paxos starts a commit N+1
 - a majority of the peers ack it
  - paxos::commit() writes N+1 it to disk
  - tells peers to commit
 - peers commit N+1, *and* refresh_from_paxos(), and generate N+1 full map
 - leader does _scrub on N+1, without latest full osdmap
 - peers do _scrub on N+1, with latest full osdmap
 - leader finishes paxos gather, does refresh_from_paxos()
 -> scrub fails.

Fix this by doing the refresh_from_paxos() at commit time and not when
the paxos round finishes.  We move the refresh out of finish_proposal
and into its own helper, and update all callers accordingly.  This
keeps on-disk state more tightly in sync with in-memory state and
avoids the need for a e.g., kludgey workaround in the scrub code.

We also simplify the bootstrap checks a bit by doing so immediately
and relying on the normal bootstrap paxos reset paths to clean up
any waiters.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 12:54:56 -07:00
Yehuda Sadeh
e996e9bee2 Merge pull request #437 from kri5/wip-fix-typo-rgw
rgw: Fix typo in rgw_user.cc

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 12:20:15 -07:00
Yehuda Sadeh
1b5258155a Merge remote-tracking branch 'origin/wip-rgw-warnings' into next
Conflicts:
	src/test/test_rgw_admin_log.cc
	src/test/test_rgw_admin_meta.cc
	src/test/test_rgw_admin_opstate.cc

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 11:17:03 -07:00
Yehuda Sadeh
a722fb713e rgw: fix bucket instance json encoding
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 10:37:28 -07:00
Yehuda Sadeh
346d9f42bc rgw_admin: fix gc list encoding
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 10:37:28 -07:00
Yehuda Sadeh
791d51eb36 Merge pull request #434 from gregsfortytwo/next
test_rgw: fix a number of unsigned/signed comparison warnings

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-15 10:24:18 -07:00
John Wilkins
55ff523ef2 doc: Fixed link in Calxeda repo instruction.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-07-15 10:05:28 -07:00
Sage Weil
aa60f940ec mon: once sync full is chosen, make sure we don't change our mind
It is possible for a sequence like:

 - probe
 - first probe reply has paxos trim that indicates a full sync is
   needed
 - start sync
 - clear store
 - something happens that makes us abort and bootstrap (e.g., the
   provider mon restarts
 - probe
 - first probe reply has older paxos trim bound and we call an election
 - on election completion, we crash because we have no data.

Non-determinism of the probe decision aside, we need to ensure that
the info we share during probe (fc, lc) is accurate, and that once we
clear the store we know we *must* do a full sync.

Fixes: #5621
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-15 10:02:47 -07:00
Sage Weil
6d3e13f0e1 rgw: fix more warnings
test/test_rgw_admin_opstate.cc: In member function 'int admin_log::test_helper::extract_input(int, char**)':
warning: test/test_rgw_admin_opstate.cc:129:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_opstate.cc:131:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_opstate.cc:133:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_opstate.cc:135:24: comparison between signed and unsigned integer expressions [-Wsign-compare]

test/test_rgw_admin_log.cc: In member function 'int admin_log::test_helper::extract_input(int, char**)':
warning: test/test_rgw_admin_log.cc:132:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_log.cc:134:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_log.cc:136:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_log.cc:138:24: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-15 09:58:08 -07:00
Greg Farnum
1dfd06dfad test_rgw: fix a number of unsigned/signed comparison warnings
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-15 08:43:42 -07:00
Christophe Courtaut
01ec4f129b rgw: Fix typo in rgw_user.cc
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-07-15 13:47:58 +02:00
Sage Weil
cf9571c13c test_rgw_admin_meta: fix warnings
test/test_rgw_admin_meta.cc: In member function 'int admin_meta::test_helper::extract_input(int, char**)':
warning: test/test_rgw_admin_meta.cc:126:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_meta.cc:128:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_meta.cc:130:24: comparison between signed and unsigned integer expressions [-Wsign-compare]
warning: test/test_rgw_admin_meta.cc:132:24: comparison between signed and unsigned integer expressions [-Wsign-compare]

on arm7l

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-14 16:37:45 -07:00
Sage Weil
bc3088c92f cls_rgw: fix warning
cls/rgw/cls_rgw.cc: In function 'int get_obj_vals(cls_method_context_t, const string&, const string&, int, std::map, ceph::buffer::list>*)':
warning: cls/rgw/cls_rgw.cc:175:28: narrowing conversion of '129' from 'int' to 'char' inside { } is ill-formed in C++11 [-Wnarrowing]

on arm7l

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-14 16:36:21 -07:00
Noah Watkins
d277edd24a Merge branch 'wip-hadoop-doc'
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-07-14 16:32:28 -07:00
Noah Watkins
6ef48c7ec2 doc: update Hadoop docs with plugin download
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-07-14 16:31:33 -07:00
Noah Watkins
a805958f89 doc: document new hadoop config options
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-07-14 16:31:33 -07:00
Sage Weil
9baa66801a Makefile: cls_rgw needs cls_rgw_types linked in now too
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-14 16:24:46 -07:00
Sage Weil
6b7d9f2b3e mon: include some (basic) auth info in report
Nothing privileged!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-14 16:20:54 -07:00
Sage Weil
56c36fa914 mon: include paxos info in report
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-14 16:16:55 -07:00