Commit Graph

28765 Commits

Author SHA1 Message Date
Loic Dachary
c16b54fd53 mon: get_command_descriptions support program
The get_command_descriptions function is not designed to be tested in
C++ because all the validation happens in pybind/ceph_argparse.py. The
get_command_descriptions program is designed to be used by python unit
tests as a mean to get a JSON dump of the content of mon/MonCommands.h

      get_command_descriptions --all

      {"cmd000":{"sig":["pg","stat"],"help": ... "avail":"cli,rest"}}

It also provides a way to reproduce and keep track of past errors
( typos etc. ) to ensure the python validation keeps catching it.

      get_command_descriptions --pull585

Add /get_command_descriptions to .gitignore so that

      git ls-files --exclude-standard --others

does not see it which is required for
f018d220f2/build-ceph.sh (L73)

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-23 23:46:43 +02:00
Loic Dachary
e0b5697387 mon: convenience function to convert commands to json
The get_command_descriptions is added to Monitor.h and contains
the code previously inlined in Monitor::handle_command to implement
the get_command_descriptions command. It is intended for tests to
convert command descriptions into json, including error cases.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-23 23:46:43 +02:00
Loic Dachary
9c9d7e6a32 autotools: set noinst_PROGRAMS
to be used by unit test support programs that do not need to be installed

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-23 23:46:43 +02:00
Loic Dachary
ee33ff81f8 autotools: group test scripts in check_SCRIPTS
The check_SCRIPTS is added in Makefile-env.am to list the tests that do
not require compilation. The scripts listed in check-local and in the
TESTS variable use check_SCRIPTS instead.

The PYTHONPATH environment variable is added to Makefile-env.am and
includes the pybind directory so that python unit tests can load the
libraries from sources.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Roald J. van Loon <roaldvanloon@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-23 23:46:43 +02:00
Loic Dachary
0eb0c3e0f3 Merge pull request #626 from ceph/wip-usage
common/config: include --cluster in default usage message
2013-09-23 14:25:15 -07:00
Sage Weil
104277c836 Merge pull request #617 from dachary/wip-lfnindex
os: fix TestLFNIndex test warnings

Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-23 14:11:36 -07:00
Sage Weil
157754b3a0 common/config: include --cluster in default usage message
Clean it up a bit too.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-23 14:10:36 -07:00
Sage Weil
679f520781 Merge pull request #624 from ceph/automake-fix-common
make: build common/secret only on linux

Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-23 14:07:22 -07:00
Noah Watkins
27fb44b358 make: build common/secret only on linux
libkeyutils is only available on linux. before the automake refactoring
secret.c was compiled into rbd and ceph_mount targets which are linux
only targets. secret.c was moved to libcommon during the refactoring,
but the conditional compilation was lost.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-09-23 13:07:12 -07:00
Sage Weil
9adb73ae41 Merge pull request #618 from dachary/wip-jerasure-minimum-to-decode
ErasureCode: minimum_to_decode unit tests and optimization

Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-23 10:53:54 -07:00
Sage Weil
5bc4610855 Merge pull request #605 from dachary/wip-erasure-code-plugin-test
erasure code plugin test
2013-09-23 10:27:47 -07:00
Sage Weil
10f291838e Merge pull request #603 from dachary/wip-erasure-code-example
erasure code example cleanup
2013-09-23 10:26:48 -07:00
Sage Weil
b683005c21 Merge pull request #607 from ceph/prctl-getname-test
code_env: use feature test for PR_GET_NAME support

Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-23 10:14:07 -07:00
Sage Weil
6b745c9491 Merge pull request #616 from ceph/automake-flags
automake: fixup am_common_flags to avoid re-definitions

Reviewed-by: Roald J. van Loon <roaldvanloon@gmail.com>
2013-09-23 10:12:16 -07:00
Gregory Farnum
2299c4dbf2 Merge pull request #614 from ceph/wip-rados-model
Prevent copy_from against yourself, and fix issues with the testing framework around it

Reviewed-by: Greg Farnum <greg@inktank.com>
2013-09-23 09:55:32 -07:00
Sage Weil
10363daddb Merge remote-tracking branch 'gh/next' 2013-09-23 09:50:10 -07:00
Wido den Hollander
991139bc0d doc: When bootstrapping mon set the correct caps for client.admin
Otherwise client.admin will be privilege-less and the admin can't
access the cluster.
2013-09-23 13:52:51 +02:00
Loic Dachary
9dfac9edfe ErasureCode: minimum_to_decode unit tests and optimization
The minimum_to_decode function simply returns the available chunks if no
recovery is necessary.

Add unit tests covering all minimum_to_decode situations:

* trying to read nothing
* read a chunk if none are available
* reading a subset of the available chunks
* read a missing chunk if there is less than k chunks available.
* reading when a chunk must be recovered

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-22 11:02:39 +02:00
Loic Dachary
9442cfaa39 os: fix TestLFNIndex test warnings
Do not ignore system return value. Wrap emacs compile line to 80
characters.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-22 10:46:23 +02:00
Noah Watkins
0a439745c4 automake: add per-target AM_CPPFLAGS
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-09-21 10:41:17 -07:00
Noah Watkins
11d8d75595 makefile-env: separate cppflags and cflags usage
AM_COMMON_FLAGS included both CPP and C[XX] flags. This can cause
re-definition errors like this:

make[4]: Nothing to be done for `all-am'.
CXX      cls/version/cls_version_client.o
<command-line>: warning: "__STDC_FORMAT_MACROS" redefined

because the macro definition appears in both AM_CXXFLAGS and
AM_CPPFLAGS.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-09-21 10:24:04 -07:00
João Eduardo Luís
3c64784e3f Merge pull request #612 from ceph/wip-6361
perfglue/heapprofiler: expect cmd name when handling command instead of 'heap <cmd>'

This was broken by the cli rework.

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-09-21 04:40:55 -07:00
Sage Weil
2b75abbd85 ceph_test_rados: fix COPY_FROM completion
Fix the copy_from operation to not remove the objects from the in_use list
until after the entire operation is complete.  In particular, the racing
read was completing and removing the dest oid from the in-use list before
the copy-from completed.  This keeps the model in sync with what the OSD
is actually doing.

If another new read started up, it would grab the previous value from the
model and expect to see that, but would instead see the updated value.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-20 21:06:09 -07:00
Sage Weil
935eb22833 ceph_test_rados: fix seq_num, improve error output
Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-20 20:49:55 -07:00
Sage Weil
4d0ee0b55b ceph_test_rados: limit max_in_flight to objects/2
In particular, we need to allow all in-flight ops to be COPY_FROM.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-20 20:44:36 -07:00
Sage Weil
303bc68e97 ceph_test_rados: do not COPY_FROM self
Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-20 20:43:12 -07:00
Sage Weil
59ee51aff7 osd/ReplicatedPG: handle COPY_FROM self
Return EINVAL if we try to COPY_FROM ourselves.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-20 20:43:00 -07:00
Dan Mick
08a97ae45f Merge pull request #592 from ceph/wip-6287
mon/OSDMonitor: make busy creating pgs message more explicit

Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-09-20 15:44:34 -07:00
Dan Mick
2d089e4644 Merge pull request #608 from dachary/wip-6357
mon: fix inverted test in osd pool create

Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-09-20 15:41:09 -07:00
Dan Mick
5cb7b2985e mon: fix wrong arg to "instructed to" status message
Fixes: #6293
Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 16ebb25f7c)
2013-09-20 15:27:28 -07:00
Yehuda Sadeh
9e98620e43 rgw: destroy get_obj handle in copy_obj()
Fixes: #6176
Backport: dumpling
We take different code paths in copy_obj, make sure we close the handle
when we exit the function. Move the call to finish_get_obj() out of
copy_obj_data() as we don't create the handle there, so that should
makes code less confusing and less prone to errors.
Also, note that RGWRados::get_obj() also calls finish_get_obj(). For
everything to work in concert we need to pass a pointer to the handle
and not the handle itself. Therefore we needed to also change the call
to copy_obj_data().

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-09-20 14:07:35 -07:00
John Wilkins
556956f449 Merge pull request #613 from ceph/wip-6356
note that ceph-deploy should not be called with sudo on certain situations
2013-09-20 13:08:06 -07:00
Alfredo Deza
c28dd12c09 note that ceph-deploy should not be called with sudo on certain situations
Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
2013-09-20 16:02:28 -04:00
Joao Eduardo Luis
b1eeaddd5f qa: workunits: cephtool: check if 'heap' commands are parseable
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2013-09-20 17:51:50 +01:00
Joao Eduardo Luis
296f2d0db3 osd: OSD: add 'heap' command to known osd commands array
Must have been forgotten during the cli rework.

Backport: dumpling

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2013-09-20 17:51:42 +01:00
Joao Eduardo Luis
238fe272c6 mds: MDS: pass only heap profiler commands instead of the whole cmd vector
The heap profiler doesn't care, nor should it, what our command name is.
It only cares about the commands it handles.

Backport: dumpling

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2013-09-20 17:51:21 +01:00
Joao Eduardo Luis
c98b910d49 perfglue/heap_profiler.cc: expect args as first element on cmd vector
We used to pass 'heap' as the first element of the cmd vector when
handling commands.  We haven't been doing so for a while now, so we
needed to fix this.

Not expecting 'heap' also makes sense, considering that what we need to
know when we reach this function is what command we should handle, and
we should not care what the caller calls us when handling his business.

Fixes: #6361
Backport: dumpling

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2013-09-20 17:51:10 +01:00
Samuel Just
e112a8348a FileStore: only remove the omap entries if nlink == 1
Fixes: #6359
Introduced in 17c5d765d7c7573f875f6b3ba66e3b6813110a06
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-20 09:30:06 -07:00
Yehuda Sadeh
7c1d2ded8f lru_map: don't use list::size()
replace list::size() with map::size(), which should have
a constant time complexity.

Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-09-20 08:56:22 -07:00
Yehuda Sadeh
532e41a998 common/lru_map: rename tokens to entries
This code was originally used in a token cache, now
as a generic infrastructure rename token fields.

Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-09-20 08:55:46 -07:00
Yehuda Sadeh
ca984e321e bufferlist: don't use list::size()
Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-09-20 08:55:12 -07:00
Sage Weil
04de6a839f Merge pull request #590 from ceph/wip-6284
osdc/ObjectCacher: finish contexts after dropping object reference

Tested: ubuntu@teuthology:/a/sage-2013-09-19_19:40:56-fs-wip-6284-testing-basic-plana
Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-20 08:46:29 -07:00
Sage Weil
0681971e62 Makefile: add extra cls DENCODER_DEPS in the cls makefile, not rgw
They belong next to the class.  Also, if rgw isn't built, ceph-dencoder
will fail to link.

Fixes: #6352
Tested-by: Haomai Wang <haomaiwang@gmail.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-19 21:03:36 -07:00
David Zafman
91dcdec18e Merge pull request #609 from dmick/wip-6293
mon: fix wrong arg to "instructed to" status message

Reviewed-by: David Zafman <david.zafman@inktank.com>
2013-09-19 18:06:27 -07:00
Dan Mick
16ebb25f7c mon: fix wrong arg to "instructed to" status message
Fixes: #6293
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-09-19 16:05:01 -07:00
Loic Dachary
bcc1680ad2 mon: fix inverted test in osd pool create
When using the properties key=value only, the test was inverted
and an attempt to obtain a substring at index string::npos throws
an exception.

Add variations of osd pool create to qa/workunits/mon/pool_ops.sh
to assert the problem has been fixed and all code paths are used.

http://tracker.ceph.com/issues/6357 fixes #6357

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-20 00:33:13 +02:00
Noah Watkins
f3718c29bb code_env: use feature test for PR_GET_NAME support
Function `get_process_name` has platform specific dependencies. Check
for Linux prctl function and correct command flag.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-09-19 15:00:31 -07:00
Yehuda Sadeh
08fe028bad rgw: use bufferlist::append() instead of bufferlist::push_back()
push_back() expects char *, whereas append can append a single char.
Appending a NULL char to push_back is cast as a NULL pointer which is
bad.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-09-19 13:03:11 -07:00
Gary Lowell
5216ed134b Merge pull request #604 from ceph/wip-buck-centos-core
Wip buck centos core
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
2013-09-19 13:01:00 -07:00
Sage Weil
fd6646f256 Makefile: fix unittest_arch
Also broken by makefile refactor.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-09-19 12:52:53 -07:00