Commit Graph

113690 Commits

Author SHA1 Message Date
Kefu Chai
e668ce72f6 msg/msg_types.h: add pick_addr()
for picking an addr from an entity_addrvec_t by given protocol type.
so:
  - v2 => v2, any
  - v1 => v1, any
  - any => any, v1, v2

and add a helper of `addr_of_type()` to avoid repeatings.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-28 20:46:47 +08:00
Sebastian Wagner
121af112f0
Merge pull request #36285 from sebastian-philipp/orch-completion-generic
mgr/orch: Add some more type annotations

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-07-28 14:46:20 +02:00
Volker Theile
49561456ef
Merge pull request #36258 from rhcs-dashboard/fix-cpu-stats
mgr/dashboard: cpu stats incorrectly displayed

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-07-28 14:00:25 +02:00
Sebastian Wagner
2310d7fbd0 mgr/orch: Add some more type annotations
Made `orch.Completion` a generic type

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-07-28 12:32:19 +02:00
Sebastian Wagner
aa4c3b2a4e mgr/cephadm: revamp ceph.conf distribution scheduling
Having an in-memeory list doesn't work properly: Especially
when loading the mgr module, we didn't knwo if we should
deploy confs or not.

Now we only distribute ceph.confs, if there is a new mon_map.
We also store that info now in the config store

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-07-28 12:27:07 +02:00
Sebastian Wagner
537412cc96 mgr/cephadm: Add test verifying the initializaiton order
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-07-28 12:18:56 +02:00
Sebastian Wagner
2a25db1cf1
Merge pull request #36012 from adk3798/cephadm_44886
mgr/cephadm: allow use of authenticated registry
2020-07-28 11:54:12 +02:00
Sebastian Wagner
4f7b2d264f
Merge pull request #36262 from sebastian-philipp/orch-readd-apply_dg
mgr/cephadm: re-add `apply_drivegroups()`

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-07-28 11:52:53 +02:00
Nathan Cutler
ca0f1e534a
Merge pull request #36306 from smithfarm/wip-add-octopus-to-release-table
doc/releases: add "octopus" column to Release Timeline

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-07-28 11:33:30 +02:00
Sebastian Wagner
33f63196ca
Merge pull request #36301 from sebastian-philipp/doc-cephadm-status-no-progress
doc/cephadm: `status` doesn't show a progress

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-07-28 11:29:01 +02:00
Or Ozeri
bc89547186 test/librados_test_stub: add version tracking of objects
True RADOS objects have a uint64_t objver associated, increasing upon each write.
This commit adds the same for librados_test_stub objects.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2020-07-28 12:12:31 +03:00
Matthew Oliver
eb604d3338 cephadm: Add tcmu-runner container when deploying ceph-iscsi
Currently when we deploy ceph-iscsi via cephadm it doesn't include a
running tcmu-runner. Which means initiators will be able to login but
you wont see the LUNS on the initiator.

This patch deploys an additional tcmu-runner container along side the
ceph-iscsi container that just runs the tcmu-runner service.

Fixes: https://tracker.ceph.com/issues/46540
Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-07-28 13:16:41 +10:00
Mark Nelson
6a865f805b mds/CInode: Optimize only pinned by subtrees check
Fixes: https://tracker.ceph.com/issues/46727

Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-07-28 02:38:26 +00:00
Kefu Chai
c06e46239e
Merge pull request #36090 from inspur-wyq/wip-37532
mon: fix the 'Error ERANGE' message when conf "osd_objectstore" is filestore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-07-28 10:21:24 +08:00
Kefu Chai
69ff4993fb
Merge pull request #36283 from rzarzynski/wip-bl-raw-privatization
common/bl: don't access raw::data nor raw::len directly. Use getters instead.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-07-28 10:20:00 +08:00
Lucian Petrut
22310e0304 cmake: fix lz4 params when building rocksdb
Recent RocksDB version use slightly different parameter names for
the LZ4 include/lib dirs, we'll have to pass the right ones.

We'll also have to fix the "CMAKE_TOOLCHAIN_FILE" parameter,
which isn't passed properly.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-07-27 16:05:09 +00:00
Nathan Cutler
4bca0b2f36 doc/releases: add "octopus" column to Release Timeline
Octopus has been out for awhile. I suppose this should have been done
earlier, but "better late than never".

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-07-27 17:41:15 +02:00
Nathan Cutler
a92e38f9e7
Merge pull request #36245 from smithfarm/wip-mimic-is-eol
doc/releases: Mimic is EOL

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-07-27 17:39:22 +02:00
Kefu Chai
a0568e06aa
Merge pull request #36279 from tchaikov/wip-crimson-msgr-v2.1
crimson/net: enable on-wire-encryt and v2.1 support

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-07-27 22:53:07 +08:00
Sebastian Wagner
1125a805bd doc/cephadm: status doesn't show a progress
Fixes: https://tracker.ceph.com/issues/45858

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-07-27 16:50:05 +02:00
Nathan Cutler
f1c991856f
Merge pull request #35852 from smithfarm/wip-opensuse-os-recommendations
doc/start/os-recommendations: current state of openSUSE

Reviewed-by: Tim Serong <tserong@suse.com>
2020-07-27 16:25:30 +02:00
Casey Bodley
e43eff6b42
Merge pull request #36269 from dang/wip-dang-46692
RGW - fix bulkupload, broken by zipper

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-07-27 09:54:46 -04:00
Sebastian Wagner
148ca82f74 cephadm: use src/mypy.ini instead
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-07-27 15:15:44 +02:00
Jason Dillaman
90bd1d7a85 librbd: use task finisher thread for image open/close callbacks
There was a potential race condition with utilizing the AsioEngine
to deliver asynchronous image open and close callbacks. This left
the potential for the io_context thread to attempt to destroy itself.

This commit changes the behavior of the image open and close callbacks
to always delete the ImageCtx (now matches the synchronous API behavior)
and it always invokes the callback in Finisher thread whose lifetime is
tied to the CephContext.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-27 09:15:15 -04:00
Jan Fajerski
6a715f6cd4
Merge pull request #36219 from guits/guits-fix_zap_osdid_osdfsid
ceph-volume: filter by osd-id or osd-fsid when zapping
2020-07-27 10:29:21 +02:00
Lucian Petrut
51967594bc common: add platform checks
O_CLOEXEC is not available on Windows. Subprocesses do not inherit
handles unless explicitly requested when creating the handle.
That being considered, we'll avoid setting this flag on Windows.

The zfs check will also be skipped for now.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-07-27 08:26:31 +00:00
Lucian Petrut
9ce91a6c51 common: skip undefined open flags on Windows
O_NOFOLLOW and O_DIRECTORY are not available on Windows.
We'll just skip those flags for now when converting flags.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-07-27 08:25:38 +00:00
Lucian Petrut
e3c28c3393 os: Add missing include
We'll need to explicitly include stddefs.h in order to
use size_t with Mingw.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-07-27 07:57:27 +00:00
Jianpeng Ma
6600814b55 os/bluestore/BlueFS: Don't flush unused device.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2020-07-27 14:59:08 +08:00
Guillaume Abrioux
e94aef6f96 ceph-volume: filter by osd-id or osd-fsid when zapping
2f5c10c12c introduced a bug:

`ceph-volume lvm zap` command fails under certain conditions.

when passing `--osd-id` or `--osd-fsid` to `ceph-volume lvm zap` command
it tries to zap additionnal devices that have nothing to do with the osd
being zapped.

When calling `api.get_lvs()` in `ensure_associated_lvs()` we have to
pass the osd-id/osd-fsid information so only related devices are
returned by `get_lvs()` method

Closes: https://tracker.ceph.com/issues/46627

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-27 08:15:50 +02:00
Yanhu Cao
728ea6ef3a mds: fix mds peer request 'no_available_op_found'
Fixes: https://tracker.ceph.com/issues/46583
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
2020-07-27 11:46:52 +08:00
Kefu Chai
685b322b30 messages/MOSDBoot: pass OSDSuperblock by const ref
MOSDBoot's ctor does not change the parameter, so let's pass by const
reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
176fc9f075 crimson/os/alienstore: always use fsid in bluestore
alienstore should not be stateful in this perspective, it should proxy
all acccess of fsid to bluestore.

there are couple issues in existing implementation:

* when mkfs, bluestore tries to generate a new osd_fsid if the specified
  one is empty. but we explicitly pass the given uuid down to
  AlienStore::mkfs() so the bluestore can use it. so we should pass it
  down instad of storing it locally.
* when persisting superblock in OSD::mkfs(), superblock.osd_fsid() is
  read from store->get_fsid(), if user specifies an empty uuid, we
  should persist the generated uuid in the superblock.

in this change, all access to fsid is proxied to the underlying
bluestore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
c551da8527 stop.sh: stop osd before mon
osd sends a MOSDMarkMeDown message to monitor and waits for its ack
before timeout, so if we can stop osd before stopping mon, stop.sh can
return sooner without waiting until the timeout.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
ec4da142d8 crimson/mgr: only pick the addr of the same type
to avoid the attempts to connect an OSD which is bound to a v2
address to a v1 address of a mgr.

in general, osd is bound to both v1 and v2 addresses, but crimson
msgr does not support multiple bound address at the time of writing, so
to avoid the failures when trying to connect to incompatible addresses,
let's filter out them when connecting to monitor. this change
silence warnings like:

peer_addr_for_me v1:172.21.15.106:60008/0 type doesn't match myaddr
v2:0.0.0.0:6802/26710

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
41e26f611d mon/MgrMap: let get_active_addrs() return a const ref
no need to create a temporary instance for referencing those addresses.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
1dfe180172 crimson/mon: only pick the addr of the same type
to avoid the attempts to connect an OSD which is bound to a v2 address
to a v1 addrss of a monitor.

in general, osd is bound to both v1 and v2 addresses, but crimson msgr
does not support multiple bound address at the time of writing, so to
avoid the failures when trying to connect to incompatible addresses,
let's filter out them when connecting to monitor. this change silence
warnings like:

peer_addr_for_me v1:172.21.15.106:60008/0 type doesn't match myaddr
v2:0.0.0.0:6802/26710

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
1745c31cf6 crimson/osd: print out client caps
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
9e46dd520f auth/cephx: implement random()->get_bytes() for crimson
instead of using CryptoRandom use the C++ standard library for
generating secret.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
00d4028ed9 crimson/admin: catch thrown exception
if the socket file exists, a std::system_error is thrown. and we should
catch it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
7d121754a1 crimson/msgr: Revert "don't advertise the on-wire format v2.1."
This reverts commit a74948bc50.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
6d45ca1e51 crimson/net: enable msgr v2.1 support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
9fefdcdc53 crimson/net: enable on_wire encryption support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
a3bd66bfb3 crimson/net: set is_rev1 for messenger v2.1 support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
d31bdcdd93 crimson/net: keep rx_preamble for msgr v2.1 support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
9dc3c951e5 crimson/net: drop stale TODO
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
f5938cb7f0 crimson/net: use rx_frame_asm for handling data read from wire
by leveraging FrameAssembler, it's much simpler. and it also pave the
road to a better messenger v2.0 and v2.1 protocol support.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 11:29:10 +08:00
Kefu Chai
55e2e1e095 crimson/net: mark abort_ functions [[noreturn]]
otherwise compiler complains if control reaches end of non-void
function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 10:32:01 +08:00
Kefu Chai
9b35cccec2 msg/async/crypto_onwire: drop unused member variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-27 10:32:01 +08:00
Changcheng Liu
e54d9dc5b4 msg: correct read result check
The "r >= 0" is checked under "r <= 0", so the right condition
is "r == 0".

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-07-26 19:22:00 +08:00