Commit Graph

26508 Commits

Author SHA1 Message Date
Dan Mick
24a270a1ae librados, rados.py: add rados_create2/init2
librados clients, particularly the ceph tool, need to be able
to specify a full 'name'; rados_create enforced 'client.<param>'
with no workaround.  New interface.  Python Rados().__init__ selects
appropriate create function depending on whether name or id is
supplied.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-04 13:13:02 -07:00
Dan Mick
ee0913c2e6 Merge remote-tracking branch 'origin/master' into wip-ceph-cli
Conflicts:
	src/mon/OSDMonitor.cc
	src/osd/OSD.cc
	src/osd/ReplicatedPG.cc
2013-06-04 13:12:08 -07:00
Sage Weil
21a1c4d39c mon: fix POOL_OP_AUID_CHANGE
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 21:07:17 -07:00
Dan Mick
18f2f19473 test/librados: remove "set_auid" pool tests
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:17 -07:00
Dan Mick
51a887a24c Add ceph_test_rados_api_cmd to the ceph-test package
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:17 -07:00
Dan Mick
fbfdd51a4d ceph: backward-compatibility hack: blank line before JSON output
Many JSON commands (osd dump, et. al.) used to print a status
line first before the actual output; this has been fixed, but there
are scripts/tools/etc. that expect it.  A simple compatibility hack
is to output a blank line, which won't confuse properly-written
JSON parsers, but will allow the tools-with-workarounds to continue
to work.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:17 -07:00
Dan Mick
f6f4f8f7f3 librados cmd tests: mon_status comes to stdout, not stderr
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:17 -07:00
Dan Mick
6cec9ccc6b librados: fix unsigned vs. signed warning
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:16 -07:00
Dan Mick
5bd93b3a89 ceph: refactor help so it can be called after cluster connection
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:16 -07:00
Sage Weil
d4bfce7ca0 mon: better debugging of caps messages/values
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 21:07:16 -07:00
Sage Weil
0bec7e139c mon/MonCap: fix setting of text in parse()
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 21:07:16 -07:00
Sage Weil
5e90e6c87d mon/MonCap: set text in set_allow_all()
This will let the cap encode/decode to the same value.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 21:07:16 -07:00
Dan Mick
8c674800d2 ceph: various cleanups
- make base class valid() do useful work
 - remove valid from CephPoolname; pool need not exist for create
 - add --user as alias for --id
 - remove vestige of special --keyring handing
 - be sure childargs is an empty list rather than None
 - remove -- from childargs if present (to stop interpreting -- args)
 - handle connection timeout cleanly

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:16 -07:00
Dan Mick
a4b824cd81 PGMonitor: fix pg dump and dump_stuck
dump needed to default to format plain, not json
dump_stuck was not doing format = plain correctly (coredumping)

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:07:16 -07:00
Dan Mick
be1a512402 cmdparse, mon: add cmd_vartype_stringify for _allowed_command
cmd_vartype are not all strings, and need a type-variant function
to turn them into strings for authorization against caps.  Use
boost::apply_visitor to get this behavior.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:06:34 -07:00
Dan Mick
f1021c9b35 cmdparse: add handle_bad_get(), ceph:: qualifiers, cmdmap_t
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 21:05:12 -07:00
Dan Mick
456f96f957 ceph: remove global declarations, declare verbose and cluster near top
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:58 -07:00
Dan Mick
df34cff767 ceph: use conf_parse_argv to deal with nonspecial Ceph args
This means parsing argv only once now, so pass arg lists to
new_style_command().  A little cleanup/commenting too.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:58 -07:00
Dan Mick
065b29cdd9 librados, rados.py: add conf_parse_argv_remainder()
New parsing function to extract any known arguments from a vector
and return any unknowns; useful for ceph CLI to allow librados
first dibs on arguments so it doesn't have to reproduce the
argument recognition

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:41 -07:00
Dan Mick
73a1b21692 ceph: make ceph -w output 'ceph status' first
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:41 -07:00
Sage Weil
dc73581a07 mon: include cluster fsid/uuid in status output
Implements: #5147
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 17:30:41 -07:00
Sage Weil
74f5ea44a6 mon/MonCap: allow osd, mds to write to cluster log
Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 17:30:41 -07:00
Dan Mick
9809371855 ceph: move admin-daemon command before connect()
Allow admin-daemon commands to work even if we can't connect()

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:41 -07:00
Dan Mick
8dd3618315 ceph: finish changing bestsigs to bestcmds
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:41 -07:00
Dan Mick
b0469a14cf cephtool/test.sh: fix "expect failure"
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:30:41 -07:00
Dan Mick
22660bdecd MDSMonitor, cmdparse: increase resiliency of bad cmd_getval()
MDSMonitor: check for and handle bad maxmds get
cmdparse.h: Use gcc demangler to print bad boost::variant typenames,
add backtrace in case of bad boost::variant get

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-03 17:24:39 -07:00
Sage Weil
1cb7dbd94a messages/MClientCapRelease: fix string output
Use the vector size, not what is in the header, which is not yet filled in
for outgoing messages.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-06-03 13:45:01 -07:00
Sage Weil
f7c1944029 Merge pull request #337 from dalgaaf/wip-da-CID-medium.v2
Fix several medium impact CID issues V2

Reviewed-by: Sage Weil <sage@inktank.com>
2013-06-02 15:52:23 -07:00
Sage Weil
44d1513880 Merge remote-tracking branch 'gh/next' 2013-06-02 14:48:04 -07:00
Sage Weil
d7e2ab1451 mon: fix uninitialized fields in MMonHealth
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
2013-05-31 21:16:54 -07:00
Sage Weil
f1ccb2d808 mon: start lease timer from peon_init()
In the scenario:

 - leader wins, peons lose
 - leader sees it is too far behind on paxos and bootstraps
 - leader tries to sync with someone, waits for a quorum of the others
 - peons sit around forever waiting

The problem is that they never time out because paxos never issues a lease,
which is the normal timeout that lets them detect a leader failure.

Avoid this by starting the lease timeout as soon as we lose the election.
The timeout callback just does a bootstrap and does not rely on any other
state.

I see one possible danger here: there may be some "normal" cases where the
leader takes a long time to issue its first lease that we currently
tolerate, but won't with this new check in place.  I hope that raising
the lease interval/timeout or reducing the allowed paxos drift will make
that a non-issue.  If it is problematic, we will need a separate explicit
"i am alive" from the leader while it is getting ready to issue the lease
to prevent a live-lock.

Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-05-31 17:09:19 -07:00
Sage Weil
fb3cd0c2a8 mon: discard messages from disconnected clients
If the client is not connected, discard the message.  They will
reconnect and resend anyway, so there is no point in processing it
twice (now and later).

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-05-31 17:05:17 -07:00
Sage Weil
6b8e74f064 mon/Paxos: adjust trimming defaults up; rename options
- trim more at a time (by an order of magnitude)
- rename fields to paxos_trim_{min,max}; only trim when there are min items
  that are trimmable, and trim at most max items at a time.
- adjust the paxos_service_trim_{min,max} values up by a factor of 2.

Since we are compacting every time we trim, adjusting these up mean less
frequent compactions and less overall work for the monitor.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-05-31 17:05:03 -07:00
Samuel Just
a03ccf17b0 OSD: *inodes_hard_limit must be less than the fd limit
Also add a comment explaining that.

Fixes: #5224
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-05-31 15:13:24 -07:00
Samuel Just
e12a9c4606 OSD: tell them they died if they don't exist as well
OSDMap::get_down_at() asserts that the osd exists.

Fixes: #5223
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-31 15:10:53 -07:00
Sage Weil
b2e490413a Merge branch 'wip-osd-leaks' into next
Reviewed-by: David Zafman <david.zafman@inktank.com>
2013-05-31 14:48:51 -07:00
Sage Weil
cec8379800 osd: fix msg leak on shutdown in ms_dispatch
Reported-by: David Zafman <david.zafman@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2013-05-31 14:48:33 -07:00
Sage Weil
9865bb460b osd: reset heartbeat peers during shutdown
This fixes a leak of the Connection's and related structures.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-05-31 14:48:33 -07:00
Sage Weil
923683ff73 mon/MonClient: fix leak of MMonGetVersionReply
Signed-off-by: Sage Weil <sage@inktank.com>
2013-05-31 14:48:32 -07:00
Sage Weil
222059ec28 osd: fix leak of MOSDMarkMeDown
Signed-off-by: Sage Weil <sage@inktank.com>
2013-05-31 14:48:32 -07:00
Sage Weil
0f246a3a90 Merge pull request #338 from alram/next
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-31 12:47:24 -07:00
Alexandre Marangone
851619ab66 upstart: handle upper case in cluster name and id
Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
2013-05-31 12:33:11 -07:00
John Wilkins
54d230529b doc: Added Java example for setting protocol to HTTP.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-31 11:15:20 -07:00
John Wilkins
145c480807 doc: Text of diagram for osd_throttles.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-31 11:14:29 -07:00
John Wilkins
8d70c091d9 doc: Omitted text diagram, and used literal include to text file.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-31 11:14:04 -07:00
Samuel Just
e6ad9da03a PGLog: only add entry to caller_ops in add() if reqid_is_indexed()
Fixes: #5216
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-31 11:09:50 -07:00
Samuel Just
db85c52d4c PG: don't write out pg map epoch every handle_activate_map
We don't actually need to write out the pg map epoch on every
activate_map as long as:
a) the osd does not trim past the oldest pg map persisted
b) the pg does update the persisted map epoch from time
to time.

To that end, we now keep a reference to the last map persisted.
The OSD already does not trim past the oldest live OSDMapRef.
Second, handle_activate_map will trim if the difference between
the current map and the last_persisted_map is large enough.

Fixes: #4731
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit 2c5a9f0e178843e7ed514708bab137def840ab89)

Conflicts:

	src/common/config_opts.h
	src/osd/PG.cc
	- last_persisted_osdmap_ref gets set in the non-static
	  PG::write_info

Conflicts:

	src/osd/PG.cc
2013-05-31 10:39:10 -07:00
Danny Al-Gaaf
5aa0b1292d .gitignore: add directory from coverity tools
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-31 19:15:23 +02:00
Danny Al-Gaaf
cfdd87cac9 mds/Server.cc: fix dereference after null check
CID 716927 (#1 of 1): Dereference after null check (FORWARD_NULL)
  var_deref_model: Passing null pointer "diri->snaprealm" to function
  "SnapRealm::resolve_snapname(std::string const &

Make sure not to dereference diri->snaprealm.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-31 19:15:23 +02:00
Danny Al-Gaaf
e5555582bf mds/Server.cc: fix dereference after null check
CID 716926 (#1 of 1): Dereference after null check (FORWARD_NULL)
  var_deref_model: Passing "mdr" to function
  "Server::apply_allocated_inos(MDRequest *)", which dereferences
  null "mdr->session".

Add assert for 'mdr' and assert for session in apply_allocated_inos().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-31 19:15:23 +02:00