average_util should be calculated from the same set of nodes that
dump_item considers when calculating the stddev.
While here, remove unnecessary check for kb > 0:
get_bucket_utilization returns false if kb <= 0.
Also, update get_bucket_utilization accordingly to the preferred
convention to use pointers and not references for output arguments.
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
Add a test to exercise rbd unmap. This was inspired by and is mostly
for rbd unmap <image-name> (aka spec), but rbd unmap <device> is tested
as well. Any significant change to krbd.cc or corresponding rbd cli
should be tested in the following three configurations at least:
- kernel < 3.14
- kernel >= 3.14, rbd.single_major=N
- kernel >= 3.14, rbd.single_major=Y
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Make use of krbd_unmap_by_spec() and allow unmapping by <image-name>
(i.e. pool/image@snap spec) with a bunch of careful adjustments to the
fragile command args/options parsing process to make --pool, --image
and --snap options work as expected for rbd unmap <image-name> case.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Make unmapping by (pool, image, snap) spec possible. For specs mapped
multiple times unmap one device per krbd_unmap_by_spec() call.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Current client code relies on cap message to trim unlinked inode
from cache. This method is too sensitive to message ordering. If
client receives the cap message while directory inode still contains
dentries that are referenced by unsafe requests, the inode can't
get trimmed.
The fix is, when rmdir/rename request finishes, try trimming the
unlinked inode again.
Fixes: #11339
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Not all programs that should be built during 'make check' should
be executed as tests. A new variable, check_TESTPROGRAMS, was
introduced to specify which programs should be executed as a test
during a 'make check'.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
When printing a non-existing key, ceph-authtool exits with a success
value:
[root@ip-172-31-3-178 ~]# ceph-authtool /etc/ceph/ceph.mon.keyring -p -n
client.doesntexist ; echo $?
entity client.doesntexist not found
0
Expected result: 1
Signed-off-by: Gerhard Muntingh <gerhard@qux.nl>
Fix for:
CID 1254380 (#1 of 1): Uninitialized scalar variable (UNINIT)
uninit_use_in_call: Using uninitialized value sa. Field
sa.sin_zero is uninitialized when calling connect
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use memset to fill integer VLAs.
Fix for:
CID 1219649 (#1 of 1): Uninitialized scalar variable (UNINIT)
uninit_use: Using uninitialized value primary_count[primary]
CID 1219648 (#1 of 1): Uninitialized scalar variable (UNINIT)
uninit_use: Using uninitialized value first_count[osds[0UL]]
CID 1219647 (#1 of 1): Uninitialized scalar variable (UNINIT)
uninit_use: Using uninitialized value count[osds[i]]
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Init variable with NULL to make sure it crashes in error
case (would be a bug) if NDEBUG disable the assert() call.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
This is caused by assert(). If assert is disabled we run into use
of uninitialized variables. Return -EINVAL in case assert() is
disabled.
Fix for:
[src/client/Client.cc:7791]: (error) Uninitialized variable: lock_state
[src/client/Client.cc:7829]: (error) Uninitialized variable: lock_type
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
This is an issue in case non-cpp code. Move the declaration of
inodeno_t and vinodeno_t structs into __cplusplus block where
they are needed.
Fix for cppcheck issue:
[src/include/cephfs/libcephfs.h:89] -> [src/include/cephfs/libcephfs.h:74]:
The struct 'inodeno_t' forward declaration is unnecessary. Type struct is
already declared earlier.
[src/include/cephfs/libcephfs.h:93] -> [src/include/cephfs/libcephfs.h:83]:
The struct 'vinodeno_t' forward declaration is unnecessary. Type struct is
already declared earlier.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
All places COMMIT_SNAP_ITEM is used the variables are
'unsigned long long'. Change the format qualifier to reflect
this and to fix:
[src/os/FileStore.cc:1341]: (warning) %lld in format string (no. 1)
requires 'long long *' but the argument type is 'unsigned long long *'.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use sizeof(char) instead of sizeof('\0').
Fix for:
[src/test/objectstore/chain_xattr.cc:164]: (warning) Suspicious
usage of 'sizeof' with a numeric constant as parameter.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Remove http_ret from child class since there is already an integer
with the same name in the parent/base class.
Fix for:
[src/rgw/rgw_rest_log.h:79] -> [src/rgw/rgw_rest.h:275]: (warning) The
class 'RGWOp_MDLog_List' defines member variable with name 'http_ret'
also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:99] -> [src/rgw/rgw_rest.h:275]: (warning) The
class 'RGWOp_MDLog_Info' defines member variable with name 'http_ret'
also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:182] -> [src/rgw/rgw_rest.h:275]: (warning) The
class 'RGWOp_DATALog_List' defines member variable with name 'http_ret'
also defined in its parent class 'RGWRESTOp'.
[src/rgw/rgw_rest_log.h:202] -> [src/rgw/rgw_rest.h:275]: (warning) The
class 'RGWOp_DATALog_Info' defines member variable with name 'http_ret'
also defined in its parent class 'RGWRESTOp'.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
otherwise the 'ceph' default value won't be catched by "main_prepare()",
this breaks TEST_corrupt_and_repair_replicated. seems the behaviour
was changed in python 2.7.9, so we move the default value to where
the '--cluster' argument is added instead of setting it in the
main argument parser.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The patch simply moves the ceph-dencoder build from server part of the
Makefiles to client part of the Makefiles.
Refs: #10691
Signed-off-by: Boris Ranto <branto@redhat.com>
The patch adds all the mds sources to DENCODER_SOURCES to allow a
dencoder client build. The patch also splits the Makefile.am file to
better accomodate the change.
Refs: #10691
Signed-off-by: Boris Ranto <branto@redhat.com>
Dencoder is built if ENABLE_CLIENT is set. However, the rgw/Makefile.am
populated DENCODER_SOURCES only if WITH_RADOSGW was set. The patch fixes
this and populates DENCODER_SOURES if ENABLE_CLIENT is set.
Signed-off-by: Boris Ranto <branto@redhat.com>
The patch adds disabled perfglue stubs to DENCODER sources in order to
avoid tcmalloc-enabled ceph-dencoder builds.
Refs: #10691
Signed-off-by: Boris Ranto <branto@redhat.com>
For CEPH_OSD_CMPXATTR_MODE_U64, OSD expects the u64 to be encoded
as string in object's xattr
Fixes: #11356
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Fixes: #11256
Multiple concurrent requests might be sent using the same tag, need the
entry map to be able to hold multiple entries.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Fixes: #11256
Backport: firefly, hammer
Beforehand we were reusing the object's tag, which is problematic as
this tag is used for bucket index updates, and we might be clobbering a
racing update (like object removal).
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
When the cluster is stuck (wait_for_clean times out), display ceph
report to stderr for debugging purposes.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
This will allow clients to detect that the object map is no longer
invalid and to reload the object map from disk.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Allow the object map rebuild process to build the full object map
in memory and save it to disk with one operation.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
If a client of librados_test_stub modified a bufferlist
retrieved via a read call, the client will actually be
changing the contents of the file. Therefore, read calls
should deep-copy the contents of the buffer::ptrs.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Rebuild will need to be able to update/resize the object maps
for image snapshots. This was previously not permitted.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
When multiple IO updates are occurring within the same object, this
will allow only a single object map update request to be sent to the
OSD.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Acquiring the lock on a missing object map will create an
empty object. Treat the empty object as a non-existant
object map to support rebuilding corrupt object maps while
holding the exclusive lock.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
This is needed to allow an atomic compare and update operation
from the rebuild object map utility.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
It will verify the object map is properly sized, verify
the existence of each object within the image (snapshot),
and clear the invalid object map flag once complete.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
If the object map is corrupt on-disk or too small for the image,
correct these basic issues as soon as possible. The object map
is still flagged as invalid, but there will be less required repair
work if future IO is able to properly update the object map.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
resize, flatten, and rebuild object map now use the same
bootstrap code for sending the request to the remote lock owner
or executing the request locally.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
The request will be sent to the client which owns the exclusive
lock to rebuild the object map for the current image HEAD.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
If an object map update fails, the object map will be flagged as
invalid. However, if a subsequent update failure occurs, the error
code will propagate back to the caller.
Fixes: #11369
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
If a client of librados_test_stub modified a bufferlist
retrieved via a read call, the client will actually be
changing the contents of the file. Therefore, read calls
should deep-copy the contents of the buffer::ptrs.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Upon an watch error notification from librados, pending requests
should be canceled and in-flight IO should be flushed prior to
unlocking the image.
Fixes: #11363
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
The refresh_seq is incremented in notify_change when calling
notify_async_complete after the locker owner completes the resize
request.
Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
If the item weight is 0 we don't want to divide; instead draw a minimal
value.
Fixes: #11357
Reported-by: Yann Dupont <yd@objoo.org>
Tested-by: Yann Dupont <yd@objoo.org>
Signed-off-by: Sage Weil <sage@redhat.com>
Override the RBD default image format back to version 1
to ensure tests properly cover the old format.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
osd: fix PG::all_unfound_are_queried_or_lost for non-existent osds
Reviewed-by: Kefu Chai <tchaikov@gmail.com
Reviewed-by: Samuel Just <sjust@redhat.com>
Fix#: 2862
Changes to some of the common files for command line parsing
Change to ceph_argparse.cc
-------------------------
Added function ceph_arg_value_type()
Given an input it will determine
i) If that input is an option or not
ii) If input is numeric in nature or not.
It will set the flag bool_option and bool_numeric appropriately.
This function is called by ceph_argparse_witharg() to figure out if
the input parameter to those functions are numeric in nature and not
an option. If the input parameter to ceph_argparse_witharg()
happens to be an option then it implies that user didn't supply
value to the option.
Changes to strol.cc
-------------------
Changes to strict_strtoll() and strict_strtol()
Both these functions reponsibility is to convert the string to long or to int.
I felt it may be not be good for it to display error message within this function,
rather caller of this function who has better understanding of the function's purpose
can display the error message.
Made change in this function to just create a generic error message,Its the
caller of this function decides what to do with this message.
Signed-off-by: Rajesh Nambiar <rajesh.n@msystechnologies.com>
Fixes: #2862
Changes related to rbd file
Changes to rbd.cc
-----------------
Change 1: line# 2744 to 2747
If the option is --order then do the check of its value if its less
than 12 or greaterthan 25 then throw error. Correct value of --order
is 12 to 25.
Change 2: Removal of validation from line# 3205 to 3209
Since the check for correct value of --order is done before hence the
check here is not needed.
Signed-off-by: Rajesh Nambiar <rajesh.n@msystechnologies.com>
When handling a proxied snap_create operation, the client which
invoked the snap_create should send the header update notification
to avoid a possible race condition where snap_create completes but
the client doesn't see the new snapshot (since it didn't yet receive
the notification).
Fixes: #11342
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Moved all parent overlap computation to within AioRequest so that
callers don't need to independently compute the overlap. Also
removed the need to pass the snap_id for write operations since
it can only be CEPH_NOSNAP.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
