Commit Graph

35619 Commits

Author SHA1 Message Date
Erik Logtenberg
6c264f2204 ceph.spec.in: add ceph-libs-compat
Added a ceph-libs-compat package in accordance with Fedora packaging
guidelines [1], to handle the recent package split more gracefully.
In Fedora this is necessary because there are already other packages
depending on ceph-libs, that need to be adjusted to depend on the new
split packages instead. In the mean time, ceph-libs-compat prevents
breakage.

[1] http://fedoraproject.org/wiki/Upgrade_paths_%E2%80%94_renaming_or_splitting_packages

Signed-off-by: Erik Logtenberg <erik@logtenberg.eu>
2014-08-27 12:56:07 -07:00
Yehuda Sadeh
62d74b30ef civetweb: update submodule
Fixes: #9208

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-27 12:27:30 -07:00
Yehuda Sadeh
11acb7097c rgw: convert header field underscores into dashes
Fixes: 9206
Backport: firefly

Certain web servers filter out underscores in the header field name.
Convert them into dashes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-27 12:27:29 -07:00
Joao Eduardo Luis
06ec5ef593 test: common: test_config: whitelist '$channel' meta-variable
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:12:15 +01:00
Joao Eduardo Luis
8a9831f938 osd/mds/monc: Adjust for LogClient/LogChannel usage
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:12:05 +01:00
Joao Eduardo Luis
bb9d4ca3f5 mon: LogMonitor: debug update_from_paxos
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
8f9e105d14 mon: LogMonitor: expand meta variables at time-of-call
Instead of expanding it when we updated config options, as that would be
nothing but trouble.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
86b215391e mon: LogMonitor: observe config changes
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
b97b06e17f mon: LogMonitor: write log messages on a per-channel basis
Instead of dumping everything to the same log file, let us this way
allow a user to specify per-channel log files, levels and syslog
facilities.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
8b5e0cfe08 mon: Monitor's clog/audit_clog need to accessed as pointers
:1,$s/clog\./clog->/

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
68ef681402 mon: Monitor: observe log client config changes
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
3ee9061718 mon: Monitor: propagate return for functions called for admin socket
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
301fc0cefd mon: Monitor: log administrative actions via admin socket
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
43075bf11c mon: Monitor: log every administrative action in an 'audit log'
Fixes: #7988

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
b7ce633455 common: LogEntry: if 'channel' is empty on decode assign default
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:02:04 +01:00
Joao Eduardo Luis
7ef1273a51 common: LogClient: allow specifying facility for LogClient
Instead of allowing only one LogClient, we will now allow any daemon to
have any number of LogClients.  They may either all log to the default
facility and level, or they may see their facility and level specified
upon creation (via a new constructor).

This patch also changes 'handle_log_ack' in such a way that the LogClient
will handle all acks with bearing the LogClient's facility, or will
otherwise simply ignore them.  The function will return true whenever the
message has been handled or false if that was not the case.  It will fall
on the caller the responsibility of deciding whether the message will be
passed to other LogClients or not, and when it is to be release.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 19:01:54 +01:00
Joao Eduardo Luis
688582494a common: str_map: have 'get_str_map' only handling plain-text
'get_str_map()' used to handle both JSON and plain-text.  In fact it
would try parsing the map as JSON on a first try and then fallback to
plain-text if it failed.  Altough useful this would pose a big issue
when we attempted to parse some values, tha we knew to be plain-text,
that had some meaning in JSON -- e.g., 'false' or 'true'.  In such case
the JSON parser would spit out an error, stating it had been able to
parse the JSON but didn't expected the type, which in fairness is
acceptable.

In its stead we now have two functions: 'get_str_map()' will only handle
plain-text, whereas 'get_json_str_map()' will keep the previous
behavior, attempting to parse a JSON string and falling back to
'get_str_map()' should it fail.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:55:02 +01:00
Yehuda Sadeh
9181114d6f rgw: clear bufferlist if write_data() successful
Fixes: #9201
Backport: firefly

We sometimes need to call RGWPutObjProcessor::handle_data() again,
so that we send the pending data. However, we failed to clear the buffer
that was already sent, thus it was resent. This triggers when using non
default pool alignments.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-27 10:29:41 -07:00
Joao Eduardo Luis
b1af4bd0a1 common: str_map: add helper methods to get values from maps
Both methods obtain values for keys from a given map.  Main distinction
is that one method will return a default value if key is not present and
if the default value is specified (i.e., not NULL), while the other
method will return the value of a fallback key if the key is not
present.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:21:47 +01:00
Joao Eduardo Luis
15e9fc45be common: LogClient: delegate message reference release to caller
Caller decides whether to release the message's reference.  It may want
to use the same message on another LogClient, if we decide not to handle
it.

Currently we always handle the message and thus always return true.  This
will not be as such later in the patch series.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:21:47 +01:00
Joao Eduardo Luis
59dac4275e common: LogEntry: refactor how we output to syslog
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:21:47 +01:00
Joao Eduardo Luis
8a2bfcd88e common: LogEntry: add 'channel' field
We now introduce the concept of 'channel', analogous to syslog
facilities, for log entries.  This will, shortly, allow a LogClient
to send messages to more than just the default syslog facility and log
file, also allowing multiple LogClients and having a way to associate
a LogEntry with its rightful owner.

We also add the same field to the MLogAck message so that a LogClient
waiting on a given ack is able to recognize the MLogAck as its own.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:21:47 +01:00
Joao Eduardo Luis
3178ca9eba common: LogEntry: add clog_type_to_string()
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:21:47 +01:00
Joao Eduardo Luis
0d2a6c70a6 common: LogEntry: change field name from 'type' to 'prio' (aka priority)
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-27 18:21:47 +01:00
Sage Weil
f25bca3136 systemd: ceph-mds and ceph-mon need networking too
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-27 09:22:59 -07:00
Sage Weil
9c9b160241 Merge pull request #2306 from mgarces/patch-1
needs network or it will not start if enabled

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-27 09:22:18 -07:00
Sage Weil
d36eaa9b59 Merge pull request #2313 from dachary/wip-buffer
common: remove spurious uint32_t in buffer.c

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-27 09:17:51 -07:00
Sébastien Han
c6f4b42f93 sample.ceph.conf: some updates
This commit adds:
- a section for radosgw
- fix the replica count value (default is 3 since firefly)
- more OSDs options for performance and CRUSH

Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
2014-08-27 18:06:22 +02:00
Yehuda Sadeh
fdfe77f9e4 Merge pull request #1950 from xylv/wip-8583
rgw: admin ops create user API can not determine existing user

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-27 08:56:25 -07:00
Sage Weil
2b13de16c5 osd/PG: fix crash from second backfill reservation rejection
If we get more than one reservation rejection we should ignore them; when
we got the first we already sent out cancellations.  More importantly, we
should not crash.

Fixes: #8863
Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-27 06:19:12 -07:00
Sage Weil
5a63215846 mon/OSDMonitor: make reweight_by_* output less misleading
Sometimes we increase weights.  State simply which OSDs are reweighted
(not overloaded).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-27 05:39:22 -07:00
Zhiqiang Wang
146e60d714 osd: Remove extra call to release_op_ctx_locks in eval_repop. It is
called in remove_repop.

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
2014-08-27 16:20:31 +08:00
Sage Weil
3bfbc141df test/mon/mkfs.sh: fix test
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-26 21:18:17 -07:00
Sage Weil
68b942e358 Merge pull request #2197 from ceph/wip-mon-empty-store
ceph_mon: check for existing mon store before opening db

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Pavan Rallabhandi <pavanrallabhandis@gmail.com>
2014-08-26 18:05:30 -07:00
Sage Weil
e9f7311b58 Merge pull request #2312 from ceph/wip-config-diff
config diff: obtain diff between current config and default config

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-26 18:03:21 -07:00
Sage Weil
c2f21c0420 osd: fix osd_tp shutdown
We need to clear the queue, not just drain the currently executing jobs.

Fixes: #9218
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-26 17:43:10 -07:00
Sage Weil
0a0c10f0ef Merge remote-tracking branch 'gh/next'
Conflicts:
	src/os/CollectionIndex.h
2014-08-26 17:33:59 -07:00
Samuel Just
8346e10755 PG: mark_log_for_rewrite on resurrection
Fixes: #8777
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-26 16:53:08 -07:00
Samuel Just
cb4c5e83a3 OSD::session_notify_pg_create: requeue at the start of the queue
Introduced: 2120f4bb6c
Fixes: #9205
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-26 16:39:40 -07:00
Joao Eduardo Luis
3c847c5a03 common: ceph_context: add admin socket command 'config diff'
Outputs a diff between the current config and what the daemon believes
to be its default config.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-26 23:31:37 +01:00
Joao Eduardo Luis
ef51160b4f common: config: let us obtain a diff between current and default config
It's mildly annoying when trying to figure out what has been changed on
a running system's config options and having to rely on whatever is set
on ceph.conf and the admin's memory of what has been injected.

With this we can simply ask the daemon for the diff between what would be
its default and what is its current config.

Current form will output extraneous information that was not directly
supplied by the user though, such as 'host' 'fsid' and 'daemonize', as
well as defaults we may rewrite ourselves (leveldb tunables on the monitor
for instance).  Nonetheless, it's way better than the alternative and
considering it should be used solely for debug purposes I think we can
get away with it.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-26 23:31:37 +01:00
Sage Weil
ac4b456254 Merge pull request #2334 from dachary/wip-9235-uninitialized-variables
tests: histogram prevent re-use of local variables
2014-08-26 14:43:51 -07:00
Loic Dachary
4b8b25ecd1 tests: histogram prevent re-use of local variables
By moving the tests to separate functions.

http://tracker.ceph.com/issues/9235 Fixes: #9235

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-08-26 23:23:17 +02:00
Sage Weil
722a3b9f83 Merge pull request #2333 from dachary/wip-9235-uninitialized-variables
tests: histogram prevent re-use of local variables

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-26 14:15:35 -07:00
Samuel Just
a525bf3190 ReplicatedPG:start_flush send a second delete
Suppose we start with the following in the cache pool:

30:[29,21,20,15,10,4]:[22(21), 15(15,10), 4(4)]+head

The object doesn't exist at 29 or 20.

First, we flush 4 leaving the backing pool with:

3:[]+head

Then, we begin to flush 15 with a delete with snapc 4:[4] leaving the
backing pool with:

4:[4]:[4(4)]

Then, we finish flushing 15 with snapc 9:[4] with leaving the backing
pool with:

9:[4]:[4(4)]+head

Next, snaps 10 and 15 are removed causing clone 10 to be removed leaving
the cache with:

30:[29,21,20,4]:[22(21),4(4)]+head

We next begin to flush 22 by sending a delete with snapc 4(4) since
prev_snapc is 4 <---------- here is the bug

The backing pool ignores this request since 4 < 9 (ORDERSNAP) leaving it
with:

9:[4]:[4(4)]

Then, we complete flushing 22 with snapc 19:[4] leaving the backing pool
with:

19:[4]:[4(4)]+head

Then, we begin to flush head by deleting with snapc 22:[21,20,4] leaving
the backing pool with:

22[21,20,4]:[22(21,20), 4(4)]

Finally, we flush head leaving the backing pool with:

30:[29,21,20,4]:[22(21*,20*),4(4)]+head

When we go to flush clone 22, all we know is that 22 is dirty, has snaps
[21], and 4 is clean. As part of flushing 22, we need to do two things:
1) Ensure that the current head is cloned as cloneid 4 with snaps [4] by
sending a delete at snapc 4:[4].
2) Flush the data at snap sequence < 21 by sending a copyfrom with snapc
20:[20,4].

Unfortunately, it is possible that 1, 1&2, or 1 and part of the flush
process for some other now non-existent clone have already been
performed.  Because of that, between 1) and 2), we need to send
a second delete ensuring that the object does not exist at 20.

Fixes: #9054
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-26 14:13:57 -07:00
Loic Dachary
ee02293ad2 tests: histogram prevent re-use of local variables
By moving the test to a separate function.

http://tracker.ceph.com/issues/9235 Fixes: #9235

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-08-26 22:05:06 +02:00
Samuel Just
66c7439ea0 ReplicatedPG::start_flush: remove superfluous loop
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-26 12:54:24 -07:00
rootfs
0416b884fb update license for libcephfs
Signed-off-by: rootfs <hchen@redhat.com>
2014-08-26 15:52:37 -04:00
Samuel Just
e81723a3d4 Merge pull request #2330 from ceph/wip-9211
osd/OSDMap: encode blacklist in deterministic order

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-26 12:24:05 -07:00
Samuel Just
e6da732365 PG: recover from each osd at most once
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-26 12:05:42 -07:00