Commit Graph

31330 Commits

Author SHA1 Message Date
Loic Dachary
f019c906b6 erasure-code: implement example create_ruleset
And the associated unit tests.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-04 08:06:25 +01:00
Loic Dachary
6ca9b24693 erasure-code: add crush ruleset creation API
Because only the erasure code plugin knows enough to create a ruleset
that is best suited for a given set of parameters.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-04 08:06:25 +01:00
Loic Dachary
2dca7371c9 erasure-code: the plugin is in a convenience library
So that it can be used by mon without linking with libosd

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-04 08:06:25 +01:00
Sage Weil
a8fcffb9ad Merge remote-tracking branch 'gh/next' 2014-02-03 22:20:47 -08:00
Sage Weil
e47e49d23a doc/release-notes: v0.77 draft notes
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-03 21:55:45 -08:00
Sage Weil
2c504ea122 doc/release-notes: v0.76
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-03 21:40:59 -08:00
Sage Weil
60ca6f699b client: fix warnings
client/Client.cc: In member function 'int Client::_read(Fh*, int64_t, uint64_t, ceph::bufferlist*)':
warning: client/Client.cc:5893:27: comparison between signed and unsigned integer expressions [-Wsign-compare]
client/Client.cc: In member function 'int Client::_write(Fh*, int64_t, uint64_t, const char*)':
warning: client/Client.cc:6235:30: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-03 21:12:41 -08:00
Josh Durgin
1829d2c9fd librados: add timeout to wait_for_osdmap()
This is used by several pool operations independent of the objecter,
including rados_ioctx_create() to look up the pool id in the first
osdmap.

Unfortunately we can't just rely on WaitInterval returning ETIMEDOUT,
since it may also get interrupted by a signal, so we can't avoid
keeping track of time explicitly here.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-03 19:00:20 -08:00
Josh Durgin
0dcceff137 conf: add options for librados timeouts
These will be implemented in subsequent patches.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-03 18:57:33 -08:00
Sage Weil
a23a2c8f01 os/KeyValueStore: fix warning
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-03 17:50:32 -08:00
Sage Weil
8e30db8f2a rest: add a few rest api tests
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-03 17:50:32 -08:00
Sage Weil
eb9ffd5a79 mon: use 'mds set inline_data ...' for enable/disable of inline data
This makes the management interface a bit more consistent.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-03 17:50:32 -08:00
Sage Weil
408b0c8e75 mon: fix 'mds set allow_new_snaps'
We had already added this as a flag (set/unset) when I generalized the
'mds set_max_mds' to be 'ceph mds set <var> <val>'.  Switch the snaps
flag to be a key/value to with true/false (similar to the hashpspool
pool flag) since there are fewer users and the var/val approach is more
general.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-03 17:50:18 -08:00
Sage Weil
33338c5ceb Merge pull request #1179 from dachary/wip-force-proposal
mon: do not force proposal when no osds

Backport: emperor, dumpling
Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-03 16:37:00 -08:00
Loic Dachary
e5ed1b2998 mon: do not force proposal when no osds
If there are no OSDs, there is no need to propose to paxos. It does not
hurt on a production cluster but it matters when running tests because
it effectively alway ignores --paxos-propose-interval.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-04 01:25:15 +01:00
Gregory Farnum
9f4edb611e Merge pull request #1151 from ceph/wip-performance-configs
Performance improvements and config options

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-03 15:06:12 -08:00
Ken Dreyer
f9b9f52420 Merge branch 'next' 2014-02-03 22:39:26 +00:00
Ken Dreyer
3b990136bf v0.76 2014-02-03 18:26:25 +00:00
Sage Weil
7ff2b541c2 client: use 64-bit value in sync read eof logic
The file size can jump to a value that is very much larger than our current
position (for example, it could be a disk image file that gets a sparse
write at a large offset).  Use a 64-bit value so that 'some' doesn't
overflow.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: John Spray <john.spray@inktank.com>
2014-02-03 08:54:14 -08:00
Sage Weil
e4ff4720d5 Merge remote-tracking branch 'gh/next'
Conflicts:
	src/mon/OSDMonitor.cc
	src/osd/OSDMap.h
2014-02-02 09:40:11 -08:00
Sage Weil
29eac1d14e Merge remote-tracking branch 'gh/wip-inline'
Passed fs suite, sage-2014-02-01_22:18:10-fs-wip-inline-testing-basic-plana,
modulo a snap test error in the suite.

Reviewed-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-02 09:16:06 -08:00
Sage Weil
9c3a4d8af6 Merge pull request #1165 from mo22/client_fuse_multithreading
client: ceph-fuse use fuse_session_loop_mt to allow multithreaded operat...

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-01 21:00:41 -08:00
Somnath Roy
684e5c4576 Pipe, cephx: Message signing under config option
The config option was present earlier but the option is checked
not in the beginning, so, it was doing some stuff. Now the config
option guard is at the very beginning of the function.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-31 15:54:40 -08:00
Somnath Roy
5fde828dac cmp.h: boost tuple comparison is replaced by regular comparison
Profiler was showing this boost tuple comparison is expensive.
So, it is replaced by regular comparison.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-31 15:54:40 -08:00
Greg Farnum
8e19488859 TrackedOp: optionally disable the actual tracking operations
To avoid op contention on global locks, optionally disable the
op tracking. Create an "osd_op_tracker" config to control it, then
in the OpTracker constructor set a bool. If it's set, the OpTracker
doesn't actually maintain its lists and avoids taking any locks. We
maintain the within-op tracking for now since it shouldn't contend,
but we can turn that off later on if we like.

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-31 15:54:40 -08:00
Somnath Roy
98ae0592ed Throttle: Turn off throttle based on max bytes
If max throttle bytes is 0, throttle should not be doing anything.
This check is introduced in the beginning of each throttle function

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-31 15:54:40 -08:00
Greg Farnum
03f7f771d6 Throttler: optionally disable use of perfcounters
These can be expensive enough that we want to disable them. We're already
mostly set up for it to be optional, so just plug in a config option and
move a timestamp read under the "if (logger)" guard to set it up!

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-31 15:54:40 -08:00
Samuel Just
ee48c871d6 common/shared_cache.hpp: compact to a single lookup where possible
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-31 15:54:40 -08:00
Samuel Just
27b5f2b873 common/shared_cache.hpp: avoid list::size()
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-01-31 15:54:40 -08:00
John Wilkins
b717e11b52 Merge pull request #1174 from alram/master
doc: rgw: el6 documentation fixes
2014-01-31 14:20:37 -08:00
Alexandre Marangone
ee4cfda151 doc: rgw: el6 documentation fixes
- fix a couple of typo for repo configuration and service restart
- the ceph package must be installed on RPM distro since the init
script relies on ceph-conf
- Note on radosgw service name for RPM distro

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>
2014-01-31 13:55:55 -08:00
David Zafman
dffe6019c3 Merge pull request #1162 from ceph/wip-5997
Fixes: #5997

 Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-01-31 12:35:56 -08:00
David Zafman
48fbccece5 osd: Change some be_compare_scrub_objects() args to const
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-31 11:00:22 -08:00
David Zafman
ce1ea619f6 osd: Change be_scan_list() arg to const
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-31 11:00:22 -08:00
David Zafman
e1bfed52f9 common: buffer::ptr::cmp() is a const function
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-31 11:00:22 -08:00
David Zafman
34eb549cd4 osd: Move the rest of scrubbing routines to the backend
Move enum scrub_error_type to osd_types.h
Move PG::_compare_scrub_objects to ReplicatedBackend::be_compare_scrub_objects
Move PG::_select_auth_object to ReplicatedBackend::be_select_auth_object
Move PG::_compare_scrubmaps to ReplicatedBackend::be_compare_scrubmaps

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-31 11:00:22 -08:00
David Zafman
f9128e89a3 osd: Move PG::_scan_list() to backend as ReplicatedBackend::be_scan_list()
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-31 11:00:22 -08:00
David Zafman
37447e758e osd: Add scrub_supported() backend interface
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-31 11:00:22 -08:00
Sage Weil
560f5f1f88 OSDMap: fix deepish_copy_from
Start with a shallow copy!

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d0f13f5414)

Conflicts:

	src/osd/OSDMap.h
2014-01-31 07:57:20 -08:00
Sage Weil
d5080799c8 OSDMonitor: use deepish_copy_from for remove_down_pg_temp
This is a backport of 368852f6c0.

Make a deep copy of the OSDMap to avoid clobbering the in-memory copy with
the call to apply_incremental.

Fixes: #7060
Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-31 07:57:04 -08:00
Sage Weil
61914d82bf OSDMap: deepish_copy_from()
Make a deep(ish) copy of another OSDMap.  Unfortunatley we can't make the
compiler-generated copy operator/constructors private until c++11.  :(

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit bd54b9841b)
2014-01-31 07:57:01 -08:00
Sage Weil
802692ed8e os/KeyValueStore: fix warning
./os/KeyValueStore.h: In member function ‘std::string KeyValueStore::strip_object_key(uint64_t)’:
warning: ./os/KeyValueStore.h:173:31: format ‘%ld’ expects argument of type ‘long int’, but argument 4 has type ‘uint64_t {aka long long unsigned int}’ [-Wformat=]

Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-31 07:19:10 -08:00
Sage Weil
f8316f1a1a Merge branch 'wip-inline' of git://github.com/kylinstorage/ceph
Conflicts:
	src/include/ceph_features.h
2014-01-31 07:00:49 -08:00
Sage Weil
3a53d6deae Merge pull request #1171 from ceph/wip-osdmap-features
mon: encode full osdmap with same feature bits as the incremental

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-01-30 21:01:16 -08:00
Josh Durgin
abcc17bf3f Merge pull request #1169 from dachary/wip-ceph-disk
Reviewed-by: Sage Weil <sage.weil@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-01-30 16:04:09 -08:00
Josh Durgin
3665815738 Merge remote-tracking branch 'origin/next'
Conflicts:
	src/test/ceph-disk.sh
2014-01-30 15:40:09 -08:00
Sage Weil
0389f1d62d mon/OSDMonitor: encode full OSDMap with same feature bits as the Incremental
Each monitor is independently encoding the full OSDMap and storing it in
its local store.  Sometime this happens when we do not have a valid value
for quorum_features (for example, it can happen during sync).

Instead, use the feature bits the Incremental was encoded with for the full
OSDMap so that they always match.

Note that this conveniently the *only* place in the mon where we encode
the full OSDMap, so we're capturing all paths.  Yay!

Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-30 15:13:05 -08:00
Sage Weil
b9208b4774 OSDMap: note encoding features in Incremental encoding
The monitor will need to know what features the incremental was encoded
with so that it can encode the OSDMap using the same bits.  Introduce a
member that is set during decode.  During encode, encoding the value passed
in by the caller.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-30 15:09:58 -08:00
Josh Durgin
ab649919f2 Merge pull request #1170 from ceph/wip-pybind-enverr
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-01-30 13:10:37 -08:00
Ilya Dryomov
e4dd1beae5 pybind: improve EnvironmentError output
Follow 70c7c9e600 ("pybind/rados.py: improve error output") and
improve EnvironmentError output for librbd and libcephfs too.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-01-30 23:04:12 +02:00