Commit Graph

28066 Commits

Author SHA1 Message Date
Sage Weil
323bdaa257 mon/MonCap: mds needs to subscribe to the osdmap
Usually it can get it from the OSD, but it assumes the mon will also share
and doesn't behave when it does not.

Fixes: #5767
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 16:58:22 -07:00
Sage Weil
14a3e2ddce remove unused fiemap code
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 16:58:14 -07:00
Sage Weil
6faf8b680d PendingReleaseNotes: note on 'ceph tell <pgid> ...'
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 16:21:09 -07:00
Sage Weil
0757c60b66 Merge remote-tracking branch 'gh/wip-tell-unified' into next
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 16:12:19 -07:00
Dan Mick
aa00ace1d8 ceph_rest_api.py: cleanup, more docstrings, unused vars
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 16:11:12 -07:00
Dan Mick
d75b6ea1a4 ceph_argparse.py: make find_cmd_target handle tell <pgid>
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 16:11:12 -07:00
Dan Mick
8985e1c9e8 ceph_argparse, mon: make "tell <pgid>" work (duplicating "pg <pgid>")
It's a wad of special cases, but it implements "tell <pgid>" such that
it has the same effect as "pg <pgid>".

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 16:11:12 -07:00
Dan Mick
7b42deef38 ceph_rest_api.py: obtain and handle tell <osd-or-pgid> commands
Contact an OSD that's up to get a list of the commands, and use
them to add to the URL map.

Special treatment throughout for these commands:

* hack the help signature dump
* keep a 'flavor' per command to allow special handler() processing
* strip off 'tell/<target>' when constructing command
* allow multiple dicts with the same url
  (the parameters and get/put methods can change)
* because of above, method must be validated in handler()
* validate the given OSD
* calculate target for command (mon, osd, pg)

Unrelated: make method_dict into global METHOD_DICT

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 16:11:03 -07:00
Sage Weil
6ac8aed040 mon/PGMonitor: reset in-core PGMap if on-disk format changes
We might have a sequence like:

- start mon, load pgmap 100
- sync
 - including a format upgrade at say v 150
- refresh
 - see format_version==1, and try read pgmap:101 as new format

This simply clears our in-memory state if we see that the format has
changed.  That will make update_from_paxos reload the latest and prevent
it from walking through the old and useless inc updates.

Note: this does not affect the auth monitor because we unconditionally
load the latest map in update_from_paxos on upgrade.  Also, the upgrade
there wasn't a format change--just a translation of cap strings from the
old to new style.

Fixes: #5764
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-07-26 15:25:30 -07:00
Sage Weil
c36c280a78 Merge remote-tracking branch 'gh/next'
Conflicts:
	src/pybind/ceph_argparse.py
2013-07-26 15:14:02 -07:00
Sage Weil
10959404b3 Merge pull request #470 from dalgaaf/wip-da-CID-3.v2
Fix several CID issues - v2

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-26 15:08:56 -07:00
Danny Al-Gaaf
de51aac8ac rgw/rgw_metadata.cc: delete md_log (RGWMetadataLog) in destructor
Call delete on md_log in the destructor.

CID 1054826 (#1 of 1): Resource leak in object (CTOR_DTOR_LEAK)
 1. alloc_new: Allocating memory by calling "new RGWMetadataLog(_cct, _store)".
 2. var_assign: Assigning: "this->md_log" = "new RGWMetadataLog(_cct, _store)".
 3. ctor_dtor_leak: The constructor allocates field "md_log" of
    "RGWMetadataManager" but the destructor and whatever functions it calls
    do not free it.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-26 23:56:10 +02:00
Danny Al-Gaaf
a45e296c04 rgw/rgw_rest_log.cc: free 'handle' to prevent memory leak
Call complete_list_entries() at end of RGWOp_MDLog_List::execute()
to free the 'handle'.

CID 1054834 (#1-2 of 2): Resource leak (RESOURCE_LEAK)
  leaked_storage: Variable "handle" going out of scope leaks the
  storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-26 23:55:57 +02:00
Danny Al-Gaaf
94cefc7765 test_cls_log.cc: fix resource leak, delete 'rop'
Fixes for:

CID 1054838 (#14 of 14): Resource leak (RESOURCE_LEAK)
  leaked_storage: Variable "rop" going out of scope leaks the
  storage it points to.

CID 1054836 (#5 of 5): Resource leak (RESOURCE_LEAK)
  leaked_storage: Variable "rop" going out of scope leaks the
  storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-26 23:55:53 +02:00
Danny Al-Gaaf
87f8e8fe9c test_cls_log.cc: remove empty lines
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-26 23:55:49 +02:00
Danny Al-Gaaf
44f43ff8e9 test_cls_statelog.cc: fix memory leak, delete 'rop'
CID 1054840 (#3 of 3): Resource leak (RESOURCE_LEAK)
  leaked_storage: Variable "rop" going out of scope leaks
  the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-26 23:55:44 +02:00
Danny Al-Gaaf
143b843ed1 test_cls_version.cc: close some memory leaks
Close some memory leaks caused by calling new() without
freeing the memory already assigned to 'rop'.

CID 1054848 (#1-6 of 6): Resource leak (RESOURCE_LEAK)
  overwrite_var: Overwriting "rop" in "rop = new_rop()"
  leaks the storage that "rop" points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-07-26 23:55:41 +02:00
Sage Weil
a08998e8dc Merge pull request #468 from cernceph/condrestart
add condrestart to the sysvinit script

Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 14:55:30 -07:00
Sage Weil
792280cf06 Merge pull request #467 from dalgaaf/wip-da-next-SCA-cppcheck-1
Fix typical smaller issues from SCA

Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 14:54:50 -07:00
Sage Weil
a90a2b42db upstart: stop ceph-create-keys when the monitor stops
This avoids lingering ceph-create-keys tasks.

Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 14:07:02 -07:00
Sage Weil
6f996223fb osd: make open classes on start optional
Some might want the old behavior.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-26 14:02:07 -07:00
Sage Weil
c24e652d8c osd: load all classes on startup
This avoid creating a wide window between when ceph-osd is started and
when a request arrives needing a class and it is loaded.  In particular,
upgrading the packages in that window may cause linkage errors (if the
class API has changed, for example).

Fixes: #5752
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-07-26 13:59:45 -07:00
Dan Mick
bcbb807c01 PG: add formatted output to pg <pgid> query, list_missing
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:09 -07:00
Dan Mick
6e6ceffa1b OSD: "tell <osd.n>" version, bench, dump_pg_recovery_stats: formatted
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:09 -07:00
Dan Mick
c9fcda88a7 OSD: provide newer command descs with module/perm/avail
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:09 -07:00
Dan Mick
cd16d73fda ceph.in: clean up help, fix partial matching on all help
Remove --help-all; too much effort for low benefit

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:09 -07:00
Dan Mick
cb38762bca ceph.in: admin_socket() now validates command and passes format
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:09 -07:00
Dan Mick
67eb7de42f ceph_argparse.py, ceph.in: validate_command: stop handling format
We were passing the entire parsed_args Namespace just to look at
and tack on 'format', and that's kinda silly; do it in the callers
instead.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:09 -07:00
Dan Mick
ba6ca5829a In general, flush in caller of dump worker rather than worker
This allows easier refactoring of workers (no dual flushes when code
changes).

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-26 13:51:00 -07:00
Samuel Just
c562b72e70 FileStore: fix fd leak in _check_global_replay_guard
Bug introduced in f3f92fe210.

Fixes: #5766
Backport: cuttlefish
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-26 13:46:26 -07:00
Dan van der Ster
1e991edefe add condrestart to the sysvinit script
We need to be able to condrestart all the ceph services on a
machine, so that we don't restart daemons that are supposed to be
stopped (e.g. broken disks).

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2013-07-26 22:20:52 +02:00
Greg Farnum
2a1aea7c7f Merge branch 'wip-rgw-versionchecks' into next
Conflicts:
	src/rgw/rgw_rest_metadata.h

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com
2013-07-26 13:03:01 -07:00
Dan Mick
c7c4c23e5f Formatter, admin_socket: make default formatter be json-pretty
If not given, default to json-pretty; if given but not equal to one
of the formatter choices, return NULL as before.  Remove defaulting
code in admin_socket.cc in favor of this.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 22:18:09 -07:00
Dan Mick
3f598e8c65 AdminSocket users: use generic formatting
All call() routines get a format parameter; all places where
JSONFormatter was created get a new_formatter() instead.
'plain' formatting is unsupported, and help is forced to be
'json-pretty' as it was.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 22:18:09 -07:00
Dan Mick
4aeb73a5e6 ceph_rest_api.py: reversed test for failed request
A failed request has a response code *outside* the range (200, 300],
not within it.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 21:42:19 -07:00
Dan Mick
47d0937ccb rest/test.py: earlier versions of requests.py don't quote ' ' in params
requests 0.12.1 handles queryparams in the URL with embedded
spaces; requests 0.8.2 does not.  Avoid the issue by quoting the
URL into expect().

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 21:41:32 -07:00
Samuel Just
6951d2345a OSD: tolerate holes in stored maps
We may have holes in stored maps during init_splits_between
and advance_pg.  In either case, we should simply skip the
missing maps.

Fixes: #5677
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-25 20:37:49 -07:00
Dan Mick
fbf74d987d ceph_rest_api.py: return error in nonformatted mode
When a nonformatted request is made, currently the only text in the
response is the (probably empty) response buffer.  Add the statusmsg
as well, where the error is likely to be explained.  This lets
the http client get a clue what happened.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 20:03:22 -07:00
Dan Mick
8b3d7a1b5e ceph_rest_api.py: actually remove the trailing / on baseurl
Lurking bug since version 1...

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-25 18:06:41 -07:00
Sage Weil
dfabc61c2c mon/MonCap: match param for entity (not name)
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 17:58:47 -07:00
Sage Weil
ca8ac9502b mon: translate caps=[k1,v1,k2,v2] into [caps_k1=v1, caps_k2=v2]
The command matching needs key/value pairs.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-07-25 17:58:35 -07:00
Sage Weil
05b6c7e864 mon/Paxos: share uncommitted value when leader is/was behind
If the leader has and older lc than we do, and we are sharing states to
bring them up to date, we still want to also share our uncommitted value.
This particular case was broken by b26b7f6e, which was only contemplating
the case where the leader was ahead of us or at the same point as us, but
not the case where the leader was behind.  Note that the call to
share_state() a few lines up will bring them fully up to date, so
after they receive and store_state() for this message they will be at the
same lc as we are.

Fixes: #5750
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-07-25 17:01:20 -07:00
Greg Farnum
063c71f0ff rgw: expose the version of synced items to the poster
To support this, we add an optional out argument to
RGWMetadatManager::put() and fill in the read_version. When the
function returns, that contains whatever the current on-disk version
of the object is (either what already existed or what we just wrote).

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-25 16:03:59 -07:00
Greg Farnum
81b62b5c71 rgw: return the update status on sync requests
Add new STATUS_APPLIED, then specify the RGWX_UPDATE_STATUS header
based on that return code when doing metadata puts.
Add a send_response() function to RGWOp_Metadata_Put in order to
support sending back our new headers. Move the translation from
STATUS_NO_APPLY from set_req_state_err() to this function, so we
can turn different sync results into failures if necessary elsewhere.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-25 16:03:54 -07:00
Greg Farnum
8ffc4ca914 rgw: add sync state parsing code and REST interface
Specify the param "sync-type" as one of "always", "update-by-version",
"update-by-timestamp". It defaults to always.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-25 16:00:17 -07:00
Greg Farnum
18eabd3322 rgw: pass the sync mode into the RGWMetadataManager::put() calls.
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-25 16:00:17 -07:00
Greg Farnum
4f9855e470 rgw: add preliminary support for sync update policies on metadata sync
We want to be able to conditionally apply new updates:
1) if we already have a newer version than the sync is applying for some
reason (replay of logs?), we don't want to go back in time.
2) If both zones were active at the same time, then we'd like to be
able to do a merge based on timestamps.

In order to support this, we add a sync_type flag to the implementations of
RGWMetadataHandler::put, and then check the version or the mtime of the
incoming put to what we have on disk, and refuse the update if needed.
We return the 204 NoContent success code when refusing sync; for the
moment the conversion is automatic but we're going to pull it out in
the next couple commits.

This commit does not complete the feature: we don't provide an interface
for specifying a different sync protocol.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-07-25 15:59:24 -07:00
Sage Weil
176aa39466 remove old push_to_kclient.pl
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-25 15:24:52 -07:00
Alfredo Deza
ebb9aceb79 ceph-disk: use new dumpling-style osd caps if we can, or fall back to old-style ones
Signed-off-by: Alfredo Deza <alfredo@deza.pe>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-25 14:30:00 -07:00
David Zafman
b46fb62df3 osd: Don't put functional code inside of an assert
Fix assert in OSDService::get_map()

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-07-25 14:01:54 -07:00