If we encouter an arg that is not a named flag/arg, and the next
item in the command description is non-positional, then raise an
'unexpected argument' exception.
Signed-off-by: Sage Weil <sage@newdream.net>
Once we have an Optional[bool], we can always transition to non-positional,
since we never have a non-optional bool. Same goes for the 'format'
arg.
Signed-off-by: Sage Weil <sage@newdream.net>
If an arg is non-positional, always show it as
[--arg-name <value>]
(All non-positional args are optional.)
Signed-off-by: Sage Weil <sage@newdream.net>
This commit majorly consists of the RabbitMQ task which is a required and supported endpoint in bucket notification tests.
And some related changes in the AMQP tests. Major changes are:
1. Addition of RabbitMQ task
2. Documentation update for the steps to execute AMQP tests
3. Addition of attributes to the tests
4. Tox dependency removal from kafka.py
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
mgr/dashboard: API Version changes do not apply to pre-defined methods (list, create etc.)
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Avl allocator mode was returning unexpected ENOSPC in first-fit mode if all size-
matching available extents were unaligned but applying the alignment made all of
them shorter than required. Since no lookup retry with smaller size -
ENOSPC is returned.
Additionally we should proceed with a lookup in best-fit mode even when
original size has been truncated to match the avail size.
(force_range_size_alloc==true)
Fixes: https://tracker.ceph.com/issues/50656
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
This fixes a scenario where mgrs continually crash while attempting to apply large increases to pg_num/pgp_num. The max step size (estmax) for each incremental update to the pgp_num is calculated as a percentage of the pg_num, which permits the possibility for the max step size (estmax) to be greater than the current pgp_num when the increase is large; this causes an integer underflow when the max step size is subtracted from the pgp_num in order to calculate the next step size with std::clamp. The integer underflow causes hi < lo in args passed to std::clamp, which causes a failed assertion, SIGABRT, and ultimately crashing mgr.
Fixes: https://tracker.ceph.com/issues/47738
Signed-off-by: Cory Snyder <csnyder@iland.com>
The `FuturizedStore` interface imposes the `get_attr()`
takes the `name` parameter as `std::string_view`, and
thus burdens implementations with extending the life-
time of the data the instance refers to.
Unfortunately, `AlienStore` is unaware that prolonging
the life of a `std::string_view` instance doesn't prolong
the data memory it points to. This problem has manifested
in the following use-after-free detected at Sepia:
```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-26_12:20:26-rados-master-distro-basic-smithi/6136929$ less ./remote/smithi194/log/ceph-osd.7.log.gz
...
DEBUG 2021-05-26 20:24:54,077 [shard 0] osd - do_osd_ops_execute: object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head - handling op
call
DEBUG 2021-05-26 20:24:54,077 [shard 0] osd - handling op call on object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - calling method lock.lock, num_read=0, num_write=0
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - handling op getxattr on object 14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head
DEBUG 2021-05-26 20:24:54,078 [shard 0] osd - getxattr on obj=14:55e1a5b4:test-rados-api-smithi067-38889-2::foo:head for attr=_lock.TestLockPP1
DEBUG 2021-05-26 20:24:54,078 [shard 0] bluestore - get_attr
=================================================================
==34068==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030001851d0 at pc 0x7f824d6a5b27 bp 0x7f822b4201c0 sp 0x7f822b41f968
READ of size 17 at 0x6030001851d0 thread T28 (alien-store-tp)
...
#0 0x7f824d6a5b26 (/lib64/libasan.so.5+0x40b26)
#1 0x55e2cbb2e00b (/usr/bin/ceph-osd+0x2b6dc00b)
#2 0x55e2d31f086e (/usr/bin/ceph-osd+0x32d9e86e)
#3 0x55e2d3467607 in crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) (/usr/bin/ceph-osd+0x33015607)
#4 0x55e2d346b14a (/usr/bin/ceph-osd+0x3301914a)
#5 0x7f8249d32ba2 (/lib64/libstdc++.so.6+0xc2ba2)
#6 0x7f824a00d149 in start_thread (/lib64/libpthread.so.0+0x8149)
#7 0x7f82486edf22 in clone (/lib64/libc.so.6+0xfcf22)
0x6030001851d0 is located 0 bytes inside of 31-byte region [0x6030001851d0,0x6030001851ef)
freed by thread T0 here:
#0 0x7f824d757688 in operator delete(void*) (/lib64/libasan.so.5+0xf2688)
previously allocated by thread T0 here:
#0 0x7f824d7567b0 in operator new(unsigned long) (/lib64/libasan.so.5+0xf17b0)
Thread T28 (alien-store-tp) created by T0 here:
#0 0x7f824d6b7ea3 in __interceptor_pthread_create (/lib64/libasan.so.5+0x52ea3)
SUMMARY: AddressSanitizer: heap-use-after-free (/lib64/libasan.so.5+0x40b26)
Shadow bytes around the buggy address:
0x0c06800289e0: fd fd fd fa fa fa fd fd fd fa fa fa 00 00 00 fa
0x0c06800289f0: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
0x0c0680028a00: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
0x0c0680028a10: fd fd fd fa fa fa fd fd fd fa fa fa fd fd fd fa
0x0c0680028a20: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
=>0x0c0680028a30: fd fd fa fa fd fd fd fd fa fa[fd]fd fd fd fa fa
0x0c0680028a40: fd fd fd fd fa fa fd fd fd fd fa fa 00 00 00 07
0x0c0680028a50: fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa fd fd
0x0c0680028a60: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
0x0c0680028a70: 00 00 00 00 fa fa fd fd fd fd fa fa fd fd fd fd
0x0c0680028a80: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==34068==ABORTING
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
An RBD might be partially deleted if the deletion
process has been started but was interrupted. In
this case return the RBD as part of the RBD list
and mark it as partially deleted.
Fixes: https://tracker.ceph.com/issues/48603
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
lba btree root leaf is empty after osd reboot, because SegmentStateTracker's states are wrong.
and that is caused by tracker->do_write not finished then seastore closed.
in transaction manager read_extent, can't read extent.
ceph_assert(0 == "Should be impossible");
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
As comment in header says client_op might be null, we need to check it
first before accessing client_op->osd_parent_span.
Fixes: #51030
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
This PR intends to remove the run-promtool-unittests.sh script as CMakeLists.txt handles the promtool execution
(also adding the description to run these tests in Readme.md)
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Methods like list(), create(), get() etc doesn't get applied the version.Also for the endpoints that get the version changed, the docs and the request header has still the version v1.0+ in them. So with the version reduced it gives 415 error when trying to make the request. This PR fixes this issue.
Fixes: https://tracker.ceph.com/issues/50855
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
and a simple REPL client allowing developer to peek and poke the
selftest module. if this turns out to be useful, we can promote this
method into a dedicated mix-in class, so other module can use it if
developer wants to test it manually.
Signed-off-by: Kefu Chai <kchai@redhat.com>
there is chance stop() and umount() methods get called even if start()
is not called in the error handling path. in that case, just make these
methods no-op. to ensure that OSD behaves in that case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
thread pool is not needed until AlienStore::start(). with this change,
we are able to tell if the AlienStore is actually started or not in
AlienStore::stop().
as seastar::sharded<Service> start a service in two phases:
1. construct the shard instances
2. actually start them
and it stops a service in a single shot, which both stops the services
and destructs the service instance(s).
so we have to implement a proper stop() method for services whose
start() might not be called after its instance is created by
seastar::sharded<Service>::start() in case of error handling or if
we just don't want to call start().
to ensure we can skip the steps to clean up the stuff created by
start(), we need to have a flag in the sharded service, because
AlienStore is a member variable of OSD, and when we do mkfs, AlienStore
is not start()'ed, and as explained above, we have to call OSD::stop()
to ensure OSD instance is destructed properly. but OSD::stop()
calls store->umount() and store->stop() unconditionally. these methods
in AlienStore rely on a functional thread pool.
fortunately, we don't need to call these methods if the store is never
mounted or started. in a case of failed "mkfs", store is not mounted at
all but the store and osd instances are created.
so, in this change, thread pool is created in AlienStore::start(), and
we will use it to tell if AlienStore is started or not in the following
change which makes the related method no-op if AlienStore is not started
yet.
also, postpone the creation of `store` until in AlienStore::start(), so
we don't need to destroy it in the dtor of AlienStore. otherwise,
BlueStore::~BlueStore() would need to reference resources which are only
available in alien threads, but when OSD::~OSD() is called, we are in
seastar's reactor.
Signed-off-by: Kefu Chai <kchai@redhat.com>
otherwise the sharded_service's dtor complains if we destruct it without
stopping it first, like:
FATAL: startup failed: std::system_error (error crimson::net:3, negotiation failure)
crimson-osd: ../src/seastar/include/seastar/core/sharded.hh:523: seastar::sharded<T>::~sharded() [with Service = crimson::osd::OSD]: Assertion `_instances.empty()' failed.
Aborting on shard 0.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* use seastar::app_template::run() instead of
seastar::app_template::run_deprecated() for returning int,
instead of returning `void`. so the application can return
int explicitly in the continuation passed to run(). more
readable this way.
* wrap the all the block in run() in a giant try-catch block,
so the exceptions thrown by the startup code can be captured
and handled.
* do not capture the exceptions individually, in the try-catch
block anymore. the outer catch block takes care of them.
this change improves the error handling when crimson-osd launches.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Since it is possible there is no podman process running when launching
vstart, use 'command -v' instead of 'pgrep -f'.
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
with the recent support for async rbd operations from pacific+ when an
older client(non async support) goes on upgrade, and simultaneously
interacts with a newer client which expects the requests to be async,
experiences hang; considering the return code for request completion to
be acknowledgement for async request, which then keeps waiting for
another acknowledgement of request completion.
this if happens should be a rare only when lockowner is an old client
and should be deferred if compatibility issues arises.
see also: 541230475d3b25ab18c4eb9bc5011060462594a6(octopus)
Signed-off-by: Deepika <dupadhya@redhat.com>
As there is no inherent ordering, there may be multiple removable
images past the unremovable image. On top of that, removing a clone
may make its parent removable so perform an additional pass if any
image gets removed.
Fixes: https://tracker.ceph.com/issues/51021
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
