Commit Graph

124650 Commits

Author SHA1 Message Date
Deepika Upadhyay
80e82686eb ceph.spec, cmake, debian: use thrift 0.13+ from distro pkg
the change to build and ship libthift was added when we didn't have 0.13.0
version shipped via distro pkgs, now that centos 8 and F34 supports req.
version, we do not need to build and ship it with jaeger library.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 10:59:45 +05:30
Deepika Upadhyay
e92b4745ba ceph.spec: enable WITH_JAEGER=ON if --with-jaeger passed
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 10:58:12 +05:30
Deepika Upadhyay
00f44782af cmake, ceph.spec, debian: use yaml-cpp >= 0.6
* since focal and centos both have yaml-cpp 0.6 available, which dropped
having boost as it's dependency, moving to 0.6 seems a good upgrade.

* cmake: delete Buildyaml, since distro suppilies v0.6 this is not needed

This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
  least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 10:58:08 +05:30
Deepika Upadhyay
9581902ac5 vstart.sh: use quay to pull jaeger instead of docker
In servers like the ones available in sepia labs, users might hit rate
limiting for docker pull, it is better to use quay image to avoid this
issue.

https://blog.container-solutions.com/dealing-with-docker-hub-rate-limiting

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 10:56:44 +05:30
Deepika Upadhyay
7aa3c2c3e6 debian/control: update yaml-cpp version to 0.6
This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
  least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)
```
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-19 21:43:17 +00:00
Deepika
52fc62520f debian, install-deps.sh: use Build-Profiles to optionally build debian libjaeger
library

* use pkg.ceph.jaeger for debian optional pkgs
remove mangling needing install-deps, instead use, buildProfileSpec feature
introduced for debian.  https://wiki.debian.org/BuildProfileSpec * check and

* set extraopts in debian/rules using pkg.ceph.jaeger
see: https://github.com/ceph/ceph/pull/38783#discussion_r662995612

* cleanup libjaeger.install mangling from CMakeLists

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-19 21:01:42 +00:00
Deepika Upadhyay
4e0b4da1a2 cmake: update build<lib>.cmake for supporting ninja
* affects cmake builds for: yaml-cpp, thrift, jaeger, opentracing external
projects

changes done:
* artificat BUILD_BYPRODUCT which tells ninja which library will be generated
after the build(needed for dependent build libs)
* use cmake_command if `make` not found so that we use default generator, cmake
will identify and use ninja directly in this case.
* minor reorder of boost path in jaeger build external projet

fixes: https://tracker.ceph.com/issues/51029

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-06 14:22:28 +00:00
Deepika
bd0ca9dcb8 install-deps.sh: add missing with_jaeger option for ceph.spec mangling
Signed-off-by: Deepika <dupadhya@redhat.com>
2021-07-05 14:14:22 +00:00
Kefu Chai
9abfca0e59
Merge pull request #42144 from cyx1231st/wip-fix-seastore-cache2
crimson/os/seastore/cache: fix retiring mutation-pending extents

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
2021-07-02 16:56:24 +08:00
Yingxin Cheng
f0c0040274 crimson/os/seastore/cache: fix retiring mutation-pending extents
Mark the retiring mutation-pending extent as INVALID, and add it's
prior-instance to retired-set in order to populate transaction
invalidation correctly.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-02 13:09:38 +08:00
Samuel Just
d82a565f0e
Merge pull request #42143 from athanatos/sjust/wip-cm-omap-int-future
crimson/os/seastore: convert omap and collection_manager to interruptible future

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-01 20:34:04 -07:00
Kefu Chai
b992221342
Merge pull request #42138 from rzarzynski/wip-crimson-alienstore-indent
crimson/os: fix indentation in alien_store.cc.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-02 08:16:23 +08:00
Kefu Chai
26a0b38dc4
Merge pull request #42128 from tchaikov/wip-crimson-asock-cleanup
crimson/asock: cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-02 08:11:12 +08:00
Samuel Just
f61eed8579 crimson/os/seastore/omap_manager: convert to use interruptible future
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-07-01 23:59:41 +00:00
Samuel Just
43eb9902e2 crimson/os/seastore/collection_manager: convert to use interruptible_future
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-07-01 23:59:41 +00:00
Samuel Just
cd78b9770a test/crimson/seastore: clean up transaction_manager based tests
Permits using either vanilla TransactionManager and
InterruptedTransactionManager, updates users to use submit_transaction
helpers.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-07-01 23:59:38 +00:00
Yuval Lifshitz
6c0c924e5a
Merge pull request #41980 from yuvalif/wip-yuval-fix-51320
rgw/notification: support version-id for all event types
2021-07-01 23:43:10 +03:00
Ali Maredia
1ca7682159
Merge pull request #42127 from ivancich/wip-little-rgw-cleanup
rgw: a small set of clean-ups

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-07-01 14:59:29 -04:00
Yuval Lifshitz
092d8fa756
Merge pull request #42012 from yuvalif/wip-yuval-fix-50039
rgw/notifications: support v4 auth for topics and notifications
2021-07-01 20:15:58 +03:00
Yuval Lifshitz
a2b8ba13f6 rgw/notification: support version-id for all event types
including: object copy, multipart upload, delete marker on versioned bucket

Fixes: https://tracker.ceph.com/issues/51320

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-07-01 20:13:25 +03:00
zdover23
f4154b1c7b
Merge pull request #42134 from zdover23/wip-doc-cephadm-operations-2021-07-01
doc/cephadm: improve "Ceph Daemon Logs" (1 of x)

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-02 03:12:22 +10:00
Ali Maredia
0bc9b868e2
Merge pull request #42023 from TRYTOBE8TME/wip-rgw-notification-tests
rgw: Modification in the usage of force-branch

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-07-01 12:44:20 -04:00
Yuval Lifshitz
1c13283f9b
Merge pull request #41945 from yuvalif/wip-yuval-fix-51261
rgw/notifications: support metadata filter in CompleteMultipartUpload and Copy events
2021-07-01 19:19:03 +03:00
Ernesto Puerta
1be0cc0da9
Merge pull request #41953 from SMIL-Infra/fix-dashboard-osd-count
mgr/dashboard: fix OSD out count

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-07-01 17:26:32 +02:00
Ernesto Puerta
4e03486235
Merge pull request #42116 from votdev/issue_51443_migrate_userdb
mgr/dashboard: User database migration has been cut out

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2021-07-01 17:23:03 +02:00
Radoslaw Zarzynski
5e1463d499 crimson/os: fix indentation in alien_store.cc.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-01 14:47:01 +00:00
Sage Weil
5c9a3962b7 Merge PR #42107 into master
* refs/pull/42107/head:
	mgr/ActivePyModules: streamline mgr_ips a bit
	mgr/ActivePyModules: don't take unnecessary lock
	pybind/mgr/mgr_module: use get("mgr_ips") for standby get_mgr_ip()
	mgr: add get() for standby modules

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2021-07-01 09:11:09 -04:00
Zac Dover
09b1dd0bb5 doc/cephadm: improve "Ceph Daemon Logs" (1 of x)
This PR turned out to be a 3-in-1:

(1) improves syntax and formatting of "Logging to stdout"
(2) improves syntax and formatting of "Logging to files"
(3) replaces all carets with tildes in 3rd-level section
    headers in operations.rst (./build-doc was crying
    about inconsistency when I fed it tildes, but tildes
    and not carets are the RST standard according to
    https://docutils.sourceforge.io/ \
    docs/user/rst/quickstart.html#sections
    so the carets had to go.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-07-01 22:43:10 +10:00
Kefu Chai
e2cc12ea0f
Merge pull request #42130 from tchaikov/wip-doc-install
doc/install/get-packages: s/el7/el8/ and cleanups

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-01 17:59:38 +08:00
Kefu Chai
166d961360
Merge pull request #42108 from zdover23/wip-doc-upgrading-ceph-cancelling-the-upgrade-2021-06-30
doc/cephadm: improve "Canceling an Upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-01 17:50:34 +08:00
Kefu Chai
3a41134592 doc/install/get-packages: do not mention additional 3rd party packages
since we are able to build and run on ubuntu focal without extra
dependencies of 3rd party packages, there is no need to mention them.
actually all of the removed packages are available in ubuntu focal.
some of them have different package names though.

the same applies to el8.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-01 16:38:56 +08:00
Kefu Chai
71e6adac35 doc/install/get-packages: s/el7/el8/
since we don't build for el7 anymore. replace all occurrences of el7 with
el8.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-01 16:38:56 +08:00
Kalpesh Pandya
9bb68a55ca rgw: Modification in the usage of force-branch
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-07-01 13:54:51 +05:30
Kefu Chai
e023bf4a54 crimson/admin: let AdminSocket::register_command() return void
instead of returning a seastar::future<>, it can just return void,
as it does not involve asynchronous operations.

also drop servers_tbl_rwlock, as we don't register hooks after OSD
starts, there is no need to protect hooks with a rw lock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-01 14:55:33 +08:00
Kefu Chai
7f64774e1b crimson/admin: remove unused variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-01 14:26:38 +08:00
Samuel Just
e9717a4735
Merge pull request #42080 from cyx1231st/wip-seastore-fix-cache
crimson/os/seastore/cache: misc fixes and cleanup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
2021-06-30 21:17:47 -07:00
Yingxin Cheng
29d9c3d588 crimson/os/seastore/cache: rename try_construct_record to prepare_record
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:16:43 +08:00
Yingxin Cheng
b5762ca66b crimson/os/seastore/cache: wait io when read from Cache::extents
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
2243b3d593 crimson/os/seastore/cache: add the missing extent invalid check
see 2bc257beb2

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
054b2eda32 crimson/os/seastore/cache: add missing add_to_read_set()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
5cd2c7c151 crimson/os/seastore/cache: cleanup get_extent_if_cached()
And remove the query_cache_for_extent(t).

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
82203cd353 crimson/os/seastore/cache: fix get_extent_by_type(t) to use Transaction::get_extent()
The original query_cache_for_extent() doesn't call wait_io() if the
extent is pending. And the following get_extent_by_type() already covers
the lookup to Cache::extents and also implements wait_io() correctly.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
1445ba17a1 crimson/os/seastore/cache: add absent extent to read-set before retire
So that transaction can be invalidated if the retiring extent has
conflict.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
a593927273 crimson/os/seastore/cache: remove retired extents from cache
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
91e91e5dca crimson/os/seastore: fix print_detail() of RetiredExtentPlaceholder
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
4c4405ee27 crimson/os/seastore/cache: cleanup try_construct_record() optional return
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:15:21 +08:00
Yingxin Cheng
e52d5469b7 crimson/onode-staged-tree: check invalid transaction only
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:13:07 +08:00
Yingxin Cheng
43ddfe3f3a crimson/os/seastore/cache: retire PRESENT extents and add to retired-set
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:13:07 +08:00
Yingxin Cheng
77839d6252 crimson/os/seastore/cache: invalidate transaction referencing retired extents
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-01 10:13:07 +08:00
Kefu Chai
85ac0c6442
Merge pull request #42103 from athanatos/sjust/wip-seastore-collection-ordering
crimson/seastore: enforce per-collection transaction ordering

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
2021-07-01 10:06:40 +08:00