Commit Graph

36052 Commits

Author SHA1 Message Date
David Zafman
7973280a48 osd: Remove unused PG functions queue_notify(), queue_info(), queue_log()
Signed-off-by: David Zafman <dzafman@redhat.com>
2014-09-24 17:55:28 -07:00
Josh Durgin
7a39e7cbe6 Merge remote-tracking branch 'origin/giant' 2014-09-24 15:27:02 -07:00
Sage Weil
c5906eca2f Merge pull request #2567 from dachary/wip-6697-strncmp-vs-memcmp
tests: use memcmp to compare binary buffers

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-24 07:30:37 -07:00
Loic Dachary
2cd9b5f969 tests: use memcmp to compare binary buffers
instead of strncmp because it will stop at the first \0

http://tracker.ceph.com/issues/6697 Fixes: #6697

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-24 16:00:44 +02:00
Loic Dachary
468d245a02 Merge pull request #2506 from dachary/wip-9304-unintended-implicit-ruleset
erasure-code: pool create must not always create a ruleset

Reviewed-by: João Eduardo Luís <joao@redhat.com>
2014-09-24 13:35:55 +02:00
John Spray
b8e6a6b180 Merge remote-tracking branch 'origin/giant' 2014-09-24 11:40:52 +01:00
Sage Weil
f711819df5 Merge pull request #2561 from athanatos/wip-9293
Wip 9293

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-23 11:40:13 -07:00
Loic Dachary
5fab7db3db Merge pull request #2414 from dachary/wip-cephtool-test
tests: make qa/workunits/cephtool/test.sh more robust

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-09-23 19:01:35 +02:00
John Wilkins
2c7ada49f8 Merge pull request #2417 from dachary/wip-placement-group
doc: revise placement group number guide.

Reviewed-by: John Wilkins <jowilkin@redhat.com>
2014-09-23 09:30:01 -07:00
Loic Dachary
34e665867e Merge pull request #2557 from ceph/wip-mon-fix-checks
ceph-mon: check fs stats just before preforking

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-09-23 17:59:02 +02:00
Loic Dachary
9c825ec2c4 documentation: revise placement group number guide
When a cluster has few OSDs (less than 50) propose a preselection of
values: as long as the number of placement groups is not too small nor
too large, it won't make much of a difference anyway.

Users of small clusters tend to blindly apply the (OSD*100)/(pool size)
formula and worry about chosing a wrong value because they do not
understand the tradeoffs. The preselection will hopefully save them from
this uncertainty.

Add an explanation of how placement groups relate to OSDs, CRUSH and
pools to help understand the tradeoffs. Explain the
tradeoffs (durability, distribution and resource usages) with examples.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Reviewed-by: Gerben Meijer <infernix@gmail.com>
Reviewed-by: Laurent Guerby <laurent@guerby.net>
2014-09-23 16:54:50 +02:00
Joao Eduardo Luis
7f71c11666 ceph-mon: check fs stats just before preforking
Otherwise statfs may fail if mkfs hasn't been run yet or if the monitor
data directory does not exist.  There are checks to account for the mon
data dir not existing and we should wait for them to clear before we go
ahead and check the fs stats.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-23 14:05:48 +01:00
Loic Dachary
9d3fbe92a8 Merge pull request #2551 from dachary/wip-9343-erasure-code-feature
erasure code feature

Reviewed-by: João Eduardo Luís <joao@redhat.com>
2014-09-23 13:37:27 +02:00
Loic Dachary
9687150cea erasure-code: isa/lrc plugin feature
There are two new plugins (isa and lrc). When upgrading a cluster, there
must be a protection against the following scenario:

  * the mon are upgraded but not the osd
  * a new pool is created using plugin isa
  * the osd fail to load the isa plugin because they have not been
    upgraded

A feature bit is added : PLUGINS_V2. The monitor will only agree to
create an erasure code profile for the isa or lrc plugin if all OSDs
supports PLUGINS_V2. Once such an erasure code profile is stored in the
OSDMap, an OSD can only boot if it supports the PLUGINS_V2 feature,
which means it is able to load the isa and lrc plugins.

The monitors will only activate the PLUGINS_V2 feature if all monitors
in the quorum support it. It protects against the following scenario:

  * the leader is upgraded the peons are not upgraded
  * the leader creates a pool with plugin=lrc because all OSD have
    the PLUGINS_V2 feature
  * the leader goes down and a non upgraded peon becomes the leader
  * an old OSD tries to join the cluster
  * the new leader will let the OSD boot because it does not contain
    the logic that would excluded it
  * the old OSD will fail when required to load the plugin lrc

This is going to be needed each time new plugins are added, which is
impractical. A more generic plugin upgrade support should be added
instead, as described in http://tracker.ceph.com/issues/7291.

http://tracker.ceph.com/issues/9343 Refs: #9343

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-23 13:34:52 +02:00
Sage Weil
7354165c1f Merge pull request #2538 from ceph/wip-mon-data-space-die
mon: die if 'mon data' fs has critically low available disk space & fix logging issues

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-22 19:16:18 -07:00
Joao Eduardo Luis
89fceb3c36 mon: Monitor: log RO commands on 'debug' level, RWX on 'info'
Fixes: #9455

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
2c5b12d909 mon: Monitor: use MonCommand::requires_perm() when checking perms
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
bb55862093 mon: Monitor.h: add 'requires_perm()' function to MonCommand struct
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
f1b814e515 mon: Monitor: log RO admin socket commands on 'debug' level
Reduces the noise caused by read-only operations via the admin socket.
RW commands are still logged at 'info' level.

Fixes: #9455

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
282bac79b4 mon: LogMonitor: adjust debug messages output levels
Reduce the noise.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
9686044a23 mon: LogMonitor: add debug message upon logging to a channel's file
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
3760bc1b46 mon: LogMonitor: appropriately expand channel meta variables
We must only expand the log file's channel meta variables upon requiring
a channel's log file.  As we may have a 'default' channel that will
cover all channels, we must wait to expand channels as they come in and
do so if they haven't yet been expanded.  Expanding the 'log_file' in
place would have the unfortunate side effect of expanding, say,

default=/tmp/whatever.$channel.log

to

default=/tmp/whatever.default.log

which would not be what we wanted upon receiving a message that should
go into channel 'foo' -- assuming we specified no such channel in the
options, channel 'foo' should go into '/tmp/whatever.foo.log'.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
6c378aebcb common: LogEntry: if channel is missing, default to "cluster"
Keeps backward compatibility when there are entities that do not know
what a channel is.  This way we ensure that those messages are logged as
they were expected to be before channels were introduced: to the cluster
log.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
2da1a2914a ceph_mon: check available storage space for mon data dir on start
error out if available storage space is below 'mon data avail crit'

Fixes: #9502

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
9996d44698 mon: DataHealthService: use get_fs_stats() instead
and relieve the DataStats struct from clutter by using
ceph_data_stats_t instead of multiple fields.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:29 +01:00
Joao Eduardo Luis
3d74230d1c common: util: add get_fs_stats() function
simplifies the task of obtaining available/used disk space, as well as
used available percentage.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-09-22 17:36:28 +01:00
Loic Dachary
f421d5cc35 documentation: comment the CompatSet data members
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-22 16:31:00 +02:00
Samuel Just
544b8c7ffb ReplicatedPG::on_removal: clear rollback info
Fixes: #9293
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-09-21 10:19:51 -07:00
Samuel Just
781f05c90e Revert "PG::init: clear rollback info for backfill as well"
Log here is already empty, need to rollback the log when we start
removal in on_removal().

This reverts commit 1e69ff2f3c.
2014-09-21 10:10:02 -07:00
Loic Dachary
3cf38bbb70 Merge pull request #2547 from wkennington/master
Cleanup boost optionals for boost 1.56

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-09-21 13:22:08 +02:00
William A. Kennington III
a53ead14c1 osd: Cleanup boost optionals
Signed-off-by: William A. Kennington III <william@wkennington.com>
2014-09-21 01:20:46 -07:00
Loic Dachary
86bdd044e4 Merge pull request #2545 from dachary/wip-9547-python-rados-truncate
python radio aio_read must not truncate on \000

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-09-21 09:38:11 +02:00
Yan, Zheng
e8ac9d866e Merge pull request #2546 from ceph/wip-log-msgs
mds: remove spurious logging
2014-09-21 09:26:29 +08:00
John Spray
28b7b93367 mds: remove spurious logging
This is from 4f3b8032d, it should not have been
included.  Harmless but noisy.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-09-21 00:55:06 +01:00
Loic Dachary
226c0c7ac6 test: check python rados aio_read with buffers containing null
http://tracker.ceph.com/issues/9547 Refs: #9547

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-20 13:08:52 +02:00
Mohammad Salehe
8bda44ff37 pybind: Fix aio_read handling of string buffer
Read data may contain \0, and buf.value interprerts them as string terminator.

Signed-off-by: Mohammad Salehe <salehe+dev@gmail.com>
2014-09-20 12:43:27 +02:00
Yan, Zheng
7fe6e10544 Merge pull request #2431 from ceph/wip-6613
Wip 9178
2014-09-20 11:51:36 +08:00
Yan, Zheng
2a2711daf8 Merge pull request #2542 from ceph/wip-9539
Filer: add lock to protect struct PurgeRange
2014-09-20 06:25:19 +08:00
Yan, Zheng
9d9c8c73ad Filer: add lock to protect strcut PurgeRange
Fix: #9539
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2014-09-20 06:09:09 +08:00
Sage Weil
77b9f4a562 Merge pull request #2529 from dachary/wip-doc-troubleshooting
documentation: fix bugous osd stop/start example
2014-09-19 14:40:47 -07:00
Sage Weil
927f14b942 Merge pull request #2532 from dachary/wip-doc-tiering
documentation: tiering typo
2014-09-19 14:40:24 -07:00
Sage Weil
859bf308ae Merge pull request #2534 from dachary/wip-9537-erasure-code-assert
erasure-code: fix assert overflow

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-19 14:40:05 -07:00
Sage Weil
bdb1f0f087 Merge remote-tracking branch 'gh/wip-9301'
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-09-19 14:27:19 -07:00
Jason Dillaman
b47fdd400e rbd: Use a rolling average to compute RBD write throughput
Replace the cumulative average with a rolling average
to better expose variations within IOS/sec and bytes/sec.

Fixes: #9374
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-09-19 13:53:43 -07:00
Samuel Just
437db44b46 Merge remote-tracking branch 'wonzhq/req-pending' into giant 2014-09-19 13:42:16 -07:00
Samuel Just
184773d67a Merge remote-tracking branch 'somnathr/wip-optracker-optimization' into giant 2014-09-19 13:34:44 -07:00
Danny Al-Gaaf
f3acae418d rgw_main.cc: add missing virtual destructor for RGWRequest
CID 1160858 (#1 of 1): Non-virtual destructor (VIRTUAL_DTOR)
 nonvirtual_dtor: Class RGWLoadGenRequest has a destructor
 and a pointer to it is upcast to class RGWRequest which doesn't
 have a virtual destructor.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit b82ceda777)
2014-09-19 11:35:36 -07:00
Danny Al-Gaaf
eeb74a1c02 os/GenericObjectMap.cc: pass big parameter by reference
CID 1188142 (#1 of 1): Big parameter passed by value (PASS_BY_VALUE)
 pass_by_value: Passing parameter header of type
 GenericObjectMap::_Header (size 176 bytes) by value.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 13b8c92a29)
2014-09-19 11:35:31 -07:00
Danny Al-Gaaf
1f134bb06f mds/Beacon.*: fix UNINIT_CTOR cases
CID 1238905 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member want_state is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member last_send is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ff6148324a)
2014-09-19 11:35:02 -07:00
Danny Al-Gaaf
f2a7d62c23 ErasureCodeLrc.h: fix UNINIT_CTOR
Fix Coverity issue, preinit with 0:

uninit_member: Non-static class member chunk_count is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ea02dc37dc)
2014-09-19 11:34:59 -07:00