Commit Graph

35027 Commits

Author SHA1 Message Date
Yehuda Sadeh
00c677b4b6 rgw: use a separate callback for civetweb access log
Access log is separate from the regular civetweb logging. Also, changed
the log level for the regular logging as it's used mostly for error
messages.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-22 10:05:53 -07:00
Yehuda Sadeh
850242cad7 rgw: separate civetweb log from rgw log
The civetweb log now is independent from the rgw log.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-22 10:05:53 -07:00
Sage Weil
ce6e9a916b doc/release-notes: v0.84
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-18 11:57:59 -07:00
Sage Weil
a59bc86594 Merge pull request #2280 from ceph/wip-fs-docs
doc: add notes on using "ceph fs new"

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-18 10:04:41 -07:00
john
b016f84682 doc: add notes on using "ceph fs new"
Signed-off-by: John Spray <john.spray@redhat.com>
2014-08-18 17:47:31 +01:00
Sage Weil
055be68cf8 qa/workunits/rbd/qemu-iotests: touch common.env
This seems to be necessary on trusty.

Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-17 20:54:28 -07:00
Sage Weil
313e60b360 Merge pull request #2010 from ceph/wip-misplaced
osd: track misplaced objects separately from degraded objects

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-17 20:49:05 -07:00
Sage Weil
5045c5cb4c qa/workunits/rest/test.py: use rbd instead of data pool for size tests
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 22:07:56 -07:00
Sage Weil
3279f3e737 qa/workunits/rest/test.py: do snap test on our data2/3 pool
This way it works when a 'data' pool doesn't already exist.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 22:07:56 -07:00
Sage Weil
6d7a229c14 qa/workunits/rest/test.py: fix rd_kb -> rd_bytes
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 22:07:56 -07:00
Sage Weil
284647f350 Merge pull request #2272 from ceph/wip-8621
Wip 8621

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-16 22:04:13 -07:00
Sage Weil
0e07f7f045 osd: fix theoretical use-after-free of OSDMap
In practice, the map will remain pinned for a while, but this
will make coverity happy.

*** CID 1231685:  Use after free  (USE_AFTER_FREE)
/osd/OSD.cc: 6223 in OSD::handle_osd_map(MOSDMap *)()
6217
6218           if (o->test_flag(CEPH_OSDMAP_FULL))
6219            last_marked_full = e;
6220           pinned_maps.push_back(add_map(o));
6221
6222           bufferlist fbl;
>>>     CID 1231685:  Use after free  (USE_AFTER_FREE)
>>>     Calling "encode" dereferences freed pointer "o".
6223           o->encode(fbl);
6224
6225           hobject_t fulloid = get_osdmap_pobject_name(e);
6226           t.write(coll_t::META_COLL, fulloid, 0, fbl.length(), fbl);
6227           pin_map_bl(e, fbl);
6228           continue;

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 14:51:31 -07:00
Sage Weil
44a0e3766a Merge pull request #2259 from ceph/wip-9039
Wip 9039

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-16 13:41:41 -07:00
Sage Weil
904a5f1c31 vstart.sh: make filestore fd cache size smaller
I hit the fd limit on a vstart cluster with the default 128; reduce this
to 16.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:19:46 -07:00
Sage Weil
932e478783 mon: track stuck undersized
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:19:46 -07:00
Sage Weil
190dc2f38f mon: track pgs that get stuck degraded
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:19:46 -07:00
Sage Weil
5168907fe2 osd: track last_fullsized in pg_stat_t
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:19:46 -07:00
Sage Weil
dbc3f65046 osd: track last_undegraded pg stat
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:19:46 -07:00
Sage Weil
1907066fee osd/osd_types: add last_undegraded, last_undersized to pg_stat_t
Keep track of the last time the PG was known to not be degraded or
undersized.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:19:45 -07:00
Sage Weil
6d6767d34c osd/PG: track PG_STATE_UNDERSIZED separately from DEGRADED
DEGRADED means there are objects without complete reduncancy; also check
for needs_recovery().

UNDERSIZED means acting set is too small.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:18:54 -07:00
Sage Weil
b037e47a36 osd: add PG_STATE_UNDERSIZED
This is a distinct concept from degraded.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:18:54 -07:00
Sage Weil
6c0a213436 osd/PG: account for misplaces separately than degraded
A degraded object does not have enough replicas or shards, while a
misplaced object is not stored in the correct place.  Account for them
separately.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:18:54 -07:00
Sage Weil
988d7419e7 librados: approximate legacy 'degraded' value
The librados API returns a degraded count and no misplaced count.  Sum them
to approximate the old behavior.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:18:53 -07:00
Sage Weil
d734d7fcac mon: warn about misplaced objects, just like degraded
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:18:53 -07:00
Sage Weil
a3149994e8 osd: num_objects_misplaced
Signed-off-by: Sage Weil <sage@inktank.com>
2014-08-16 13:18:53 -07:00
Sage Weil
34fe7a8214 Merge pull request #2217 from ceph/wip-problem-osds
mon: 'ceph osd blocked-by' for histogram of peers OSDs are waiting for

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-16 13:15:10 -07:00
Sage Weil
14614e013f qa/workunits/rest/test.py: fix 'df' test to use total_used_bytes
This changed back in ee2dbdb0f5

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:06:02 -07:00
Sage Weil
ee9e1eadab Merge pull request #2271 from ceph/wip-9053
paxos: fix problem with disjoint quorum members

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-08-16 09:18:19 -07:00
Yehuda Sadeh
c83c90ca07 rgw: update civetweb submodule
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-08-15 21:37:50 -07:00
Alfredo Deza
a14a700acc Merge pull request #2270 from ceph/wip-init-ceph
init-ceph: don't use bashism

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2014-08-15 19:42:59 -04:00
Sage Weil
0d6d1aa7e0 init-ceph: don't use bashism
-z STRING
              the length of STRING is zero

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-15 16:41:43 -07:00
Alfredo Deza
dc611e864b Merge pull request #2247 from ceph/wip-ceph-disk
ceph-disk: fix various dmcrypt bugs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2014-08-15 19:40:15 -04:00
Loic Dachary
082db05c81 Merge pull request #2269 from ceph/wip-osd-mon-feature
osd: fix mon feature requirement

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-08-16 00:19:59 +02:00
Sage Weil
1d0c66ae3a Merge remote-tracking branch 'gh/next' 2014-08-15 15:01:23 -07:00
Boris Ranto
7df67a544f Fix -Wno-format and -Werror=format-security options clash
This causes build failure in latest fedora builds, ceph_test_librbd_fsx adds -Wno-format cflag but the default AM_CFLAGS already contain -Werror=format-security, in previous releases, this was tolerated but in the latest fedora rawhide it no longer is, ceph_test_librbd_fsx builds fine without -Wno-format on x86_64 so there is likely no need for the flag anymore

Signed-off-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-15 15:00:59 -07:00
Sage Weil
ae0b9f1776 osd: fix feature requirement for mons
These features should be set on the client_messenger, not
cluster_messenger.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-15 14:29:11 -07:00
Sage Weil
d9e96b1708 Merge pull request #2268 from ceph/wip-9119
Wip 9119

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-15 14:11:10 -07:00
Samuel Just
0db3e51165 ReplicatedPG::maybe_handle_cache: do not forward RWORDERED reads
Even with READFORWARD, we can't forward RWORDERED reads.

Fixes: #9119
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-15 14:04:20 -07:00
Samuel Just
5040413054 ReplicatedPG::cancel_copy: clear cop->obc
Otherwise, an objecter callback might still be hanging
onto this reference until after the flush.

Fixes: #8894
Introduced: 589b639af7
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-08-15 14:04:20 -07:00
Sage Weil
eb589428dd Merge pull request #2264 from ceph/wip-crush-features
do not require crush features for rules that aren't being used

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-08-15 13:55:36 -07:00
Sage Weil
2f0e2951d7 unittest_osdmap: test EC rule and pool features
TODO: tiering feature bits.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-15 13:54:11 -07:00
Sage Weil
e4d238bbcf Merge pull request #2266 from kevincox/removewirehsark
Remove Old Wireshark Dissectors

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-15 13:41:15 -07:00
Samuel Just
cab479367a Merge pull request #2070 from somnathr/wip-sd-filestore-optimization
Wip sd filestore optimization

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-15 13:37:54 -07:00
Kevin Cox
0b2761036f Remove Old Wireshark Dissectors
Remove the two old Wireshark plugins.  They do not build and are
superseded by the dissector which is inside Wireshark.

Signed-Off-By: Kevin Cox <kevincox@kevincox.ca>
2014-08-15 15:27:13 -04:00
Sage Weil
16dadb86e0 osd: only require crush features for rules that are actually used
Often there will be a CRUSH rule present for erasure coding that uses the
new CRUSH steps or indep mode.  If these rules are not referenced by any
pool, we do not need clients to support the mapping behavior.  This is true
because the encoding has not changed; only the expected CRUSH output.

Fixes: #8963
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-15 08:55:27 -07:00
Sage Weil
1d95486780 crush: add is_v[23]_rule(ruleid) methods
Add methods to check if a *specific* rule uses v2 or v3 features.  Refactor
the existing checks to use these.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-15 08:55:27 -07:00
Loic Dachary
cb4c564933 Merge pull request #2213 from dachary/wip-9025-chunk-remapping
erasure-code: chunk remapping

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-15 12:43:03 +02:00
Sage Weil
c54f1e4d66 mon/Paxos: share state and verify contiguity early in collect phase
We verify peons are contiguous and share new paxos states to catch peons
up at the end of the round.  Do this each time we (potentially) get new
states via a collect message.  This will allow peons to be pulled forward
and remain contiguous when they otherwise would not have been able to.
For example, if

  mon.0 (leader)  20..30
  mon.1 (peon)    15..25
  mon.2 (peon)    28..40

If we got mon.1 first and then mon.2 second, we would store the new txns
and then boot mon.1 out at the end because 15..25 is not contiguous with
28..40.  However, with this change, we share 26..30 to mon.1 when we get
the collect, and then 31..40 when we get mon.2's collect, pulling them
both into the final quorum.

It also breaks the 'catch-up' work into smaller pieces, which ought to
smooth out latency a bit.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-14 16:56:14 -07:00
Sage Weil
3e5ce5f0dc mon/Paxos: verify all new peons are still contiguous at end of round
During the collect phase we verify that each peon has overlapping or
contiguous versions as us (and can therefore be caught up with some
series of transactions).  However, we *also* assimilate any new states we
get from those peers, and that may move our own first_committed forward
in time.  This means that an early responder might have originally been
contiguous, but a later one moved us forward, and when the round finished
they were not contiguous any more.  This leads to a crash on the peon
when they get our first begin message.

For example:

 - we have 10..20
 - first peon has 5..15
   - ok!
 - second peon has 18..30
   - we apply this state
 - we are now 18..30
 - we finish the round
   - send commit to first peon (empty.. we aren't contiguous)
   - send no commit to second peon (we match)
 - we send a begin for state 31
   - first peon crashes (it's lc is still 15)

Prevent this by checking at the end of the round if we are still
contiguous.  If not, bootstrap.  This is similar to the check we do above,
but reverse to make sure *we* aren't too far ahead of *them*.

Fixes: #9053
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-14 16:55:58 -07:00
Loic Dachary
5c2d2320c0 erasure-code: remap chunks if not sequential
If the remap vector is not empty, use it to figure out the sequence of
data chunks.

http://tracker.ceph.com/issues/9025 Fixes: #9025

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-15 01:07:22 +02:00