Commit Graph

102716 Commits

Author SHA1 Message Date
Sage Weil
de1a8eb641 osd/PrimaryLogPG: impose a per-op return buffer limit (32)
Limit the return value size for writes, so that we don't inadvertantly
stuff too much data in the PGLog.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 13:29:10 -05:00
Sage Weil
c9746ff62e osd/PrimaryLogPG: include op_returns in pg log [dup] records, reply
If the op has the RETURNVEC flag set,

- Record the per-op return value and overall return code in the pg log and
  dup records.
- Include the same in the actual MOSDOpReply back to the client.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 13:29:10 -05:00
Jeff Layton
d070deef7d client: remove Inode dir_contacts field
...nothing uses this, and it's not quite clear what it's for in the
first place.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-25 13:31:46 -04:00
Sage Weil
2027eb2584 osd/osd_types: reset payload_len in merge_osd_op_vector_out_data
This is cleaner and simpler.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:06:10 -05:00
Sage Weil
2870f8e53c osd/osd_types: show in and out byte counts for OSDOp
This will appear in the MOSDOp[Reply] message strings.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:06:10 -05:00
Sage Weil
f0d1816182 cls/hello,ceph_test_cls_hello: test returning data from a mutation
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:06:10 -05:00
Sage Weil
17f0cf3c26 osdc/Objecter: allow mutate ops to have return data
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:06:10 -05:00
Sage Weil
330f6f3ae9 librados: add a RETURNVEC flag
This signals that the client is soliciting and expecting return codes
that are >0 and/or return buffers on update ops.

This should be used with caution, since it means that compound requests
that include read and write requests will *also* persist the read ops in
order to be idempotent, which means that those ops' results may get
truncated.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:06:10 -05:00
Sage Weil
7985d9593d osd/osd_types: add per-op return fields to log [dup] records
Allow an overall positive return value, and also per-op return value and
outdata for each op in a request.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:06:10 -05:00
Sage Weil
aba33749ed osd/PrimaryLogPG: remove finish_proxy_write reply path
There is no case where we have already allocated a reply message here,
since that is only done in execute_ctx() after the transaction is prepared,
something that doesn't happen if we proxy the write through to the base
layer.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 11:01:09 -05:00
Sage Weil
6b3c27fc8a osd/PrimaryLogPG: drop unused MOSDOpReply creation path
It is unclear why this path was originally needed, but it does predate
the removal of the dual-ack removal, so I expect it was related to
that.  It is no longer necesarry.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 10:58:31 -05:00
Sage Weil
c48a29b9ed msg/async: do not register lossy client connections
If our policy is server + lossy, we do not need to track our incoming
client connections by address.  First, because it doesn't do us any good.
Second, it is nicer if we don't, because we can allow multiple incoming
connections from the same peer addr.

Update a couple of tests: one doesn't apply any more, and the other needs
a different way of getting the just-accepted con ref.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 10:39:15 -05:00
Sage Weil
0efec7445e msg/async: allow anonymous client-side connections
If the connection mode is lossy, allow us to open a new connection to
a target, regardless of whether other such connections are already open.
This allows for single-use connections.  If you call this multiple times,
you'll get separate, distinct connections.

We are lucky that the cleanup infrastructure for AsyncMessenger just works
without modification.  :)

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 10:39:15 -05:00
Kefu Chai
a8dd64c801 seastar: pickup unix domain socket support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-25 23:33:21 +08:00
Sage Weil
8d6d084d08 Merge PR #30523 into master
* refs/pull/30523/head:
	mon/MonClient: skip CEPHX_V2 challenge if client doesn't support it

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-09-25 09:49:57 -05:00
Sage Weil
ac746d343b osd/OSDMap: remove remaining g_conf() usage
Use the explicitly-passed cct here instead of relying on the global symbol.
The only user was check_health(), so this is easy.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 09:44:45 -05:00
xie xingguo
4196b13283 osd/OSDMap: do not trust partially simplified pg_upmap_item
If we simplified a partially no-op pg_upmap_item, we shall still
continue to verify that the remaining part is valid.
The bug is introduced by 02e5499b35,
before which we always validate the correctness of a pg_upmap_item
before trying to cancel or simplify it.

Fixes: https://tracker.ceph.com/issues/42052
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-25 21:28:27 +08:00
Sage Weil
a4eb581419 qa/suites/rados/thrash-old-clients: tolerate MON_DOWN
The first thing this test does is upgrade all mons.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-25 08:20:43 -05:00
Jeff Layton
9b6ed03e60 test: librados startup/shutdown racer test
Spawn a bunch of threads, have them do a cluster_connect and immediately
shut the connection down again. Do this in a loop and allow the thread
to exit. Join all of the threads.

This helps ferret out places where state that is global to the program
is inadequately protected.

Test-for: https://tracker.ceph.com/issues/42026
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-25 08:12:06 -04:00
Ricardo Marques
6cc6e8d5db mgr/dashboard: Support iSCSI target-level CHAP auth
Requires `ceph-iscsi` config v11

Fixes: https://tracker.ceph.com/issues/41576

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-09-25 12:09:12 +01:00
Lenz Grimmer
90fcc2e81b
mgr/dashboard: Display the "destroyed" state in OSD list (#30514)
mgr/dashboard: Display the "destroyed" state in OSD list

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-09-25 09:49:21 +00:00
Nathan Cutler
49d740118f install-deps.sh: add EPEL repo for non-x86_64 archs as well
Fixes: https://tracker.ceph.com/issues/42023
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-09-25 10:58:33 +02:00
Nathan Cutler
8495d37695 src/script/backport-create-issue: implement --force option
If --force option is given along with an explicit list of issue numbers,
backport issues will be created regardless of issue status.

In all other usage scenarios, script behavior remains the same (backport
issues are created only if issue status is "Pending Backport").

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-09-25 10:49:32 +02:00
root
07db50e057 mds: Reorganize class members in MDBalancer header
Fixes: https://tracker.ceph.com/issues/42043
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-09-25 14:18:10 +05:30
Kefu Chai
b43cbefc05
Merge pull request #30420 from runsisi/wip-remove-requests
rpm,deb: python-requests is not needed for ceph-common

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-25 16:31:13 +08:00
Rishabh Dave
26a18fca80 doc: move MDS deployment from main quick start to CephFS quick start
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 13:03:09 +05:30
Rishabh Dave
cd3e0acb8d doc: update CephFS Quick Start doc
Also, skip the details about CephX user's keyring and monitor's socket
since the kernel driver can figure out these details automatically now.

Fixes: https://tracker.ceph.com/issues/41872
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 13:02:23 +05:30
Jan Fajerski
122f579b4f
Merge pull request #30422 from rishabh-d-dave/cv-fix-pytest-warning
ceph-volume: fix warnings raised by pytest
2019-09-25 09:05:06 +02:00
Rishabh Dave
a96a32bd2e doc/ceph-fuse: mention -k option in ceph-fuse man page
Fixes: https://tracker.ceph.com/issues/42044
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:21:20 +05:30
Rishabh Dave
dc21d04172 vstart_runner: name booleans for options differently
This achieves 2 things: makes it more understandable for the reader that
the boolean variable is an option and reads less like a method name.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:13:18 +05:30
Rishabh Dave
254f705e27 qa/vstart_runner.py: add an option to remove old log
Add an option "--clear-old-log" to get rid of log entries before making
entries for current batch of tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:13:10 +05:30
Rishabh Dave
e16ad0126b qa/vstart_runner.py: make log initialization code reusable
Move it to a method so that the code be can be reused.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:12:00 +05:30
Rishabh Dave
1ef02b724d qa/vstart_runner.py: make printing of stdout of ps optional
stdout of ps command is generally pretty huge which makes it harder to
interpret logs. Don't print it by default and add "--log-ps-output" to
enable printing it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:11:54 +05:30
Rishabh Dave
842b5590aa qa/vstart_runner.py: add timeout for vstart.sh and stop.sh
Override default timeout for execution of vstart.sh and stop.sh since
neither scripts take more than 100 seconds normally.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:10:22 +05:30
Rishabh Dave
83ff993e95 qa/vstart_runner.py: add an option to teardown cluster
Add "--teardown" option so that vstart_runner.py tears down Ceph
cluster automatically after the batch of tests has completed running.
Also delete build/dev and build/out directories, after running stop.sh.

Update the doc for vstart_runner.py as well.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-09-25 12:09:58 +05:30
xie xingguo
85efa8f99c mgr/balancer: enable pg_upmap cli for future use
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-25 13:29:24 +08:00
Sage Weil
ca29586120 osd/PrimaryLogPG: remove unnecessary claim_op_out_data() calls
In all of these cases, the vector<OSDOp>* ops in question points to
the MOSDOp::ops vector, which is copied by the MOSDOpReply ctor.

Move back to a const MOSDOp* where appropropriate.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 13:36:07 -05:00
Sage Weil
89d7418d90 osd/PrimaryLogPG: do_pg_ops make MOSDOp const, fix comment
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 13:28:05 -05:00
Kefu Chai
f90e3e1026
Merge pull request #30036 from tchaikov/wip-python3-pkgversion
ceph.spec.in: s/pkgversion/version_nodots/

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
2019-09-25 01:40:44 +08:00
Kefu Chai
6d2f66846d ceph.spec.in: s/pkgversion/version_nodots/
`python3_pkgversion` is now defined as 3, while we don't have packages
like python3-Cython yet in EPEL7. but we do have `python36-Cython`. so
let's use `python3_version_nodots` instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-24 23:23:35 +08:00
Lenz Grimmer
a6689e592e
mgr/dashboard: Gracefully handle client/target info not found (#30349)
mgr/dashboard: Gracefully handle client/target info not found

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-09-24 15:11:47 +00:00
Sage Weil
f1f8e9a520 PendingReleaseNotes: add note for 14.2.5 so we can backport this
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 09:26:45 -05:00
Sage Weil
6e46b1c0e5 osd/OSDMap: health alert for non-power-of-two pg_num
Fixes: https://tracker.ceph.com/issues/41647
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 09:26:33 -05:00
Lenz Grimmer
59299c8ea7
mgr/dashboard: Using wrong identifiers in RGW user/bucket data… (#30492)
mgr/dashboard: Using wrong identifiers in RGW user/bucket datatables

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-09-24 13:58:50 +00:00
Kefu Chai
ccae47ea08 auth/Crypto: assert(len <= 256) before calling getentropy()
this assert() will be optimized out in Release mode.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-24 21:31:00 +08:00
Kefu Chai
bb6d9c32ad auth/Crypto: fallback to /dev/urandom if getentropy() fails
we still support Linux 3.10, which does not support `getrandom(2)`.
so even if the glibc installed on the building host offers `getentropy()`
call, we still need to check for the syscall's existence at runtime
in case that the Linux kernel version is lower than 3.17 which
introduced the support of `getrandom(2)`.

in this chance, we detect the existence of `getentropy()` by using it
when constructing `CryptoRandom`, this system call could fail due to 3
reasons:
1. ENOSYS: the kernel does not support `getrandom(2)`
2. EPERM: the syscall is blocked by a security policy
3. EINTR: interrupted by a signal

so we fall back to /dev/urandom in cases of ENOSYS and EPERM, retry on
EINTR, otherwise, an exception is thrown.

Fixes: https://tracker.ceph.com/issues/42018
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-24 21:30:54 +08:00
Sage Weil
d4fdd5a1e6 mon/MonitorDBStore: use const string& for args throughout
const bufferlist& too

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-24 08:30:17 -05:00
Sage Weil
ba4ef272bb Merge PR #30537 into master
* refs/pull/30537/head:
	test/pybind: fix local variables for emacs
	crush: add editor property variables

Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-24 08:25:44 -05:00
Ricardo Marques
d01835fb8c mgr/dashboard: Gracefully handle client/target info not found
Fixes: https://tracker.ceph.com/issues/41779

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-09-24 13:46:22 +01:00
Patrick Donnelly
30909f5a6a
Merge PR #29824 into master
* refs/pull/29824/head:
	qa: whitelist new FS_INLINE_DATA_DEPRECATED health warning
	mds: add a HEALTH_WARN message when inline_data is enabled
	mds: log a warning message when mds is started on an fs with inline_data
	mon: deprecate CephFS inline_data support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2019-09-24 04:32:28 -07:00