Commit Graph

27868 Commits

Author SHA1 Message Date
Dan Mick
e4d063b082 ceph-rest-api manpage
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
07590ae865 cephtool/test.sh: sync status no longer supported
da0aff28ab removed 'ceph sync status';
test script needs to stop trying it

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
e66d011cc8 cephtool/test.sh cleanup
1) don't pass subshell to expect_false
2) osd pause/unpause don't take arguments (apply to all OSDs)

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
6cafb0e3e0 ceph-rest-api tests
Patterned after cephtool/test.sh, with some deeper validation of
output format and contents (because structured output is easier
to validate).

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
4cb0e06266 Add 'ceph-rest-api'
ceph-rest-api is a Python WSGI module for accessing the Ceph cluster.
It supports most of the commands supported by the ceph CLI,
appropriately translated to HTTP GET/PUT requests.  It is not a
truly RESTful interface.

Not supported at this moment: "tell", "pg <pgid>", and "daemon"
commands.

Configuration options are specified in ceph.conf, specified with
-c/--conf or obtained from $CEPH_CONF, /etc/ceph/ceph.conf,
~/.ceph/ceph.conf, or ./ceph.conf.

-n/--name specifies the client name, used for the cluster
authentication key and for the ceph.conf section name (default
is client.restapi).

        restapi keyring = <keyring file>
        restapi public addr = listenIP:port (default 0.0.0.0:5000)
        restapi base url = <base path> (default /api/v0.1)
        restapi log level = (error, warning, info, debug)
        restapi log file = (default /var/log/ceph/<clientname>.log)

Primitive human-level command discovery is supported; GET from
BASEURL (say, http://localhost:5000/api/v0.1) will show an HTML
table of all commands and arguments, method supported, and help strings.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
45dc57316a OSDMonitor: return failure for 'osd lost' without safety switch
If user doesn't supply --yes-i-really-mean-it, fail the operation

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
c87f55b5a8 CompatSet, MDSMap: formatted dumps must avoid numeric tags
Tags made of numbers only confuse XML parsers.  Avoid them.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
e75db7138f mon: OSDMonitor: formatter output on 'osd map'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
17595f3414 mon: OSDMonitor: osd pool get: move to preproc, add formatted output
Move 'osd pool get' handling to preprocess_command()
It's a read operation; there's no need for it to be in prepare_command().
Also, add formatted output.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 20:58:51 -07:00
Dan Mick
a15d93276e mon: formatted output for 'fsid'
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 20:58:50 -07:00
Dan Mick
04fbc523cf mon: Monitor: 'health detail --format=json' should only output JSON
Fixes: #5526

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 20:58:50 -07:00
Dan Mick
300f42b876 mon: Monitor: support multiple formatters on some status functions
Commands such as 'mon_status', 'quorum_status', 'sync_status' and
'sync_force' didn't support other formatter besides json.  Regardless of
'--format=foo' being specified, they would always output in json.

This commit changes that behavior, allowing a format to be passed.  These
functions do not output in plain-text however.  Plain-text will default
to 'json' -- the reason: the information they provide are better outputted
in a structured fashion, and I was too lazy to come up with a plain-text
version that could be at least as good.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-10 20:58:42 -07:00
Sage Weil
c6d1dba63a test/encoding/import-genearted: fix
Dest dir was wrong, cp shortcut check was wrong.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-10 20:47:08 -07:00
Sage Weil
3cf8ae01de mds: make rename_rollback::drec not assert on unexpected d_type
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-10 20:40:24 -07:00
Sage Weil
846382b330 mds: fix rename_rollback::drec cctor
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-10 20:33:15 -07:00
Sage Weil
db68975e78 cls/rgw: init op in rgw_cls_obj_complete_op ctor
Turned up by the encoding/decoding unit tests.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-10 20:25:12 -07:00
Dan Mick
db71a38063 mon: add formatted output for several commands:
pg stat
    pg map
    osd stat
    osd lspools
    osd blacklist ls
    osd create
    osd getmaxosd
    mds compat show

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
f8ee4a7439 auth: cephx: KeyServer: list secrets to formatter or plaintext
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
e76b05b937 mon: AuthMonitor: improve error reporting for import
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
b18554a6a5 mon: AuthMonitor: formatted output to 'auth get-or-create[-key]'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
7da0320c56 mon: AuthMonitor: dump formatted and plaintext key on 'get-key'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
7049efca50 mon: AuthMonitor: early out on bad entity name
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
83bcde34c9 mon,auth: AuthMonitor, KeyRing: add Formatter-dumps of auth info
Signed-off-by: Dan Mick <dan.mick@inktank.com>

auth: KeyRing: encode_formatted() receives a label as first argument

Also, this allows us to standardize formatted output on the AuthMonitor,
so that all output starts with a section 'auth'.  Other subsystems using
the KeyRing class, can specify whatever section they prefer.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 19:02:31 -07:00
Dan Mick
92d5eb08b4 auth: Crypto: add encode_formatted/plaintext() functions
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 19:02:30 -07:00
Dan Mick
2e2bc6d023 ceph_argparse.py cleanup: simplify some code, error on unused args
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:30 -07:00
Dan Mick
a5621625e1 ceph CLI: valid() no longer returns bool, but just exception
The type validation's valid() method was using a combination of
return code and exception to really indicate the same thing;
simplify by only raising on validation error, and change callers
to cope.  validate_one() follows suit.

Also, allow validate() to be called with args that are dicts
(for REST support) rather than bare words.  Rules: 'name':'value'
must both match descriptor's name and validate (through valid() for
the value.  If value is '', it's assumed to be the same as name,
(one can pass, for example, "detail" as one parameter to
REST, but it will still show up as {'detail':''} here).

Tweak validate()'s algorithm a bit in the process, and make
validate_command() exit the bestcmds loop immediately on first
full validation.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:30 -07:00
Dan Mick
0f2ebc6ee3 ceph.in, ceph_argparse.py: move descsort() to ceph_argparse
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:30 -07:00
Dan Mick
bfa2284d21 MonCommands: add new fields: modulename, perms, availability
To help optimize the REST API, we need to know whether the commands
are read (GET) or write (PUT/POST).  However, we also could use that
same info for permission/caps checking.  Add modulename/perms as
the required caps for each command to drive both needs.

The availability field is to control whether a command is displayed/
advertised through the CLI or REST interfaces; some commands aren't
really useful for REST, and we may want to invent REST-only commands;
also, this gives us a way to deprecate commands quickly and leave the
code, should that be desirable.  Make the CLI display only commands
marked with the 'CLI' marker.

Also stop renaming 'help' to 'helptext' in the client.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:30 -07:00
Dan Mick
c9b54d7569 common: Formatter: add flush to bufferlist
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-10 19:02:30 -07:00
Dan Mick
989be66cf8 rados.py: allow long-form names of 'warn' and 'err' as watch levels
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 19:02:30 -07:00
Yehuda Sadeh
7f67848060 Merge pull request #420 from gregsfortytwo/master
radosgw-admin help command naming and text improvements

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-10 18:42:18 -07:00
Samuel Just
1ad2127ff5 PGMonitor: fix stat comparison in prepare_pg_stats
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-10 18:33:51 -07:00
Sandon Van Ness
180c7b4567 Get device-by-path by looking for it instead of assuming 3rd entry.
On some systems (virtual machines so far) the device-by-path entry
from udevadm is not always in the same spot so instead actually
look for the right output instead of blindy assuming that its a
specific field in the output.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Gary Lowell  <gary.lowell@inktank.com>
2013-07-10 15:06:25 -07:00
Sandon Van Ness
836e35f8ed Get device-by-path by looking for it instead of assuming 3rd entry.
On some systems (virtual machines so far) the device-by-path entry
from udevadm is not always in the same spot so instead actually
look for the right output instead of blindy assuming that its a
specific field in the output.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Gary Lowell  <gary.lowell@inktank.com>
2013-07-10 14:55:52 -07:00
Gary Lowell
348acecb8d Merge branch 'wip-5492-gl' into next
Re-instate fix for #5492 with corrections.

Signed-off-by: Gary Lowell  <gary.lowell@inktank.com>
2013-07-10 14:28:35 -07:00
Gary Lowell
a120d81df6 Makefile.am: fix ceph_sbindir
This reinstates the fix for the ceph_sbindir from commit
352f362567, with corrections
from Danny's review commits pull request #389.
Fixes: #5492

Reported-by: Denis Kaganovich <mahatma@eu.by>
Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Gary Lowell  <gary.lowell@inktank.com>
2013-07-10 13:44:02 -07:00
Josh Durgin
5deeb7b0c5 Merge branch 'next' 2013-07-10 13:27:58 -07:00
Josh Durgin
dcb42ca36c test_rados: don't use assert_greater
It's only present in very recent versions of nose, and doesn't provide
much beyond a simple assert.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-07-10 13:25:56 -07:00
Johannes Erdfelt
d8fe439260 Python Ioctx.get_xattrs does not take xattr_name
Fixes bug 5528

Since it returns all xattrs, it does not take an xattr_name. Also,
add unit tests for the Object xattrs methods.

Signed-off-by: Johannes Erdfelt <johannes@erdfelt.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-07-10 13:14:55 -07:00
Sage Weil
bc291d3fc3 osd: fix warning
From 653e04a794

Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-10 11:32:34 -07:00
Sage Weil
653e04a794 osd: limit number of inc osdmaps send to peers, clients
We should not send an unbounded number of inc maps to our peers or clients.
In particular, if a peer is not contacted for a while, we may think they
have a very old map (say, 10000 epochs ago) and send thousands of inc maps
when the distribution shifts and we need to peer.

Note that if we do not send enough maps, the peers will make do by
requesting the map from somewhere else (currently the mon).  Regardless
of the source, however, we must limit the amount that we speculatively
share as it usually is not needed.

Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-07-10 11:02:08 -07:00
Sage Weil
5c8a563463 mon: do not populate MMonCommand paxos version field
The field is not used or useful since the monitor does not even look
at it (in Monitor::handle_command()).  Avoid populating it and the
subsequent confusion for poor developers.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-10 10:19:38 -07:00
Sage Weil
b36338be43 messages/MPGStats: do not set paxos version to osdmap epoch
The PaxosServiceMessage version field is meant for client-coordinated
ordering of messages when switching between monitors (and is rarely
used).  Do not fill it with the osdmap epoch lest it be compared to a
pgmap version, which may cause the mon to (near) indefinitely put it on
a wait queue until the pgmap version catches up.

Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-10 10:19:38 -07:00
Greg Farnum
997c6f4e95 rgw-admin: add some of the newer flags and specify their use
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-10 10:09:35 -07:00
Greg Farnum
2ea6c4cc70 rgw-admin: use 'get' instead of 'show' or info for geo stuff
These all output data that you can put back in via 'set', so they're
a 'get' rather than something else.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-10 10:09:35 -07:00
Sage Weil
c124aa09ee Merge pull request #419 from dalgaaf/wip-da-fix-build-rados-classes
ceph.spec.in: add missing %{_libdir}/rados-classes/libcls_* files
2013-07-10 09:34:40 -07:00
Sage Weil
d36ec78ba1 qa/workunits/cls/test_cls_rgw: add new rgw class tests
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-10 09:30:19 -07:00
Sage Weil
6ddaa26e14 Merge pull request #418 from dalgaaf/wip-da-fix-build-ceph-test
Fix build and packaging of ceph-test package

Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-10 09:21:05 -07:00
Sage Weil
bee4c72f57 Merge pull request #417 from dalgaaf/wip-da-fix-CRYPTO_LIBS
Makefile.am: fix build, use $(CRYPTO_LIBS)

Reviewed-by: Sage Weil <sage@Inktank.com>
2013-07-10 09:20:03 -07:00
Danny Al-Gaaf
2e79f73bd3 ceph.spec.in: add missing %{_libdir}/rados-classes/libcls_* files
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-10 18:12:05 +02:00