Commit Graph

74581 Commits

Author SHA1 Message Date
Sage Weil
dd8fed575a Merge pull request #15649 from liewegas/wip-20274
osd/PrimaryLogPG: record prior_version for DELETE events

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-06-21 21:13:14 -05:00
Sage Weil
8d8f2c3dc3 Merge pull request #15601 from liewegas/wip-no-restful-cert
mgr: remove default cert; disable both restful and dashboard by default

Reviewed-by: Boris Ranto <branto@redhat.com>
2017-06-21 21:10:35 -05:00
Jiaying Ren
c61d12e729 rgw: update rgw_rados.h function doc
+ update system_obj_get_attr func comment
+ drop deprecated func comment for get_obj
+ update func comment for pool_iterate_begin

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>
2017-06-22 09:33:15 +08:00
Haomai Wang
49fe721073 Merge pull request #15755 from Adirl/wip-HUGEPAGES
msg/async/rdma: automatically set RDMAV_HUGEPAGES_SAFE according to conf

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-06-22 09:23:29 +08:00
Sage Weil
e5dd3a3ca2 qa/workunits/rest/test_mgr_rest_api: remove ruleset
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 21:17:14 -04:00
Sage Weil
647533f552 qa/suites/rados/rest/mgr-restful: configure cert
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 21:09:58 -04:00
Sage Weil
025debbb9f pybind/mgr/restful: add 'restful restart' command
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 21:00:35 -04:00
Jason Dillaman
fee2ea1c1c Merge pull request #15757 from runsisi/wip-image-ts
librbd: add create timestamp metadata for image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-06-21 20:51:52 -04:00
Kefu Chai
720aea32bc Merge pull request #15444 from wjwithagen/wip-wjw-freebsd-termwidth
test: Make screencandy optional for FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-22 07:46:57 +08:00
Sage Weil
7879efdd6b osd/PGLog: ignore ERROR entires in _merge_object_divergent_entries
Sometimes ERROR log entries do not have prior_version.  Also,
they aren't actually updates and don't affect the object.

Fixes: http://tracker.ceph.com/issues/20274
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 18:25:47 -04:00
Sage Weil
ba46a54f53 qa/objectstore/bluestore: lower full thresholds
On NVMe we can write really fast... fast enough that we may fill up before
we realize we are nearly full.  It's a problem here only because our
devices are so small that the OSDs fill up proportionally quickly.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 14:52:49 -07:00
Sage Weil
e6d2c0052d mon/PGMap: slightly better debugging around pgmap updates
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 17:50:59 -04:00
Sage Weil
f5def607be qa/suites/powercycle/osd/tasks/radosbench: consume less space
Fixes: http://tracker.ceph.com/issues/20302
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 14:50:04 -07:00
Sage Weil
b8a9ee8eb3 Merge pull request #15783 from liewegas/wip-20358
os/bluestore: move sharedblob to new collection in same shard
2017-06-21 16:01:51 -05:00
Sage Weil
41401c0ad7 Merge pull request #15785 from liewegas/wip-bluestore-fsck
qa/objectstore/bluestore*: fsck on mount

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-06-21 16:01:07 -05:00
Sage Weil
3a9fa481c5 qa/objectstore/filestore-btrfs: test btrfs on trusty only
Centos btrfs hits ENOSPC all the time :(, and now xenial is doing
it too.

See http://tracker.ceph.com/issues/20169
Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 16:44:25 -04:00
Mykola Golub
039b74ddfb Merge pull request #15734 from dillaman/wip-librbd-image-remove
librbd: fix issues with image removal state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-06-21 23:41:33 +03:00
Gregory Farnum
0e51ab1012 Merge pull request #15727 from ovh/bp-runtime-cfg-of-hb-inflator
osd, messages/MOSDPing: bunch of fixes related to ping inflation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-06-21 16:05:40 -04:00
Yuri Weinstein
88fe387614 Merge pull request #15723 from badone/wip-ceph-mon-startup-error-logging
mon: Log errors at startup

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-06-21 13:01:35 -07:00
Sage Weil
faa44ed3bc Merge pull request #15784 from liewegas/wip-thrash-cleanup
qa/tasks/ceph_manager: get osds all in after thrashing
2017-06-21 14:31:34 -05:00
Sage Weil
a6831381ad Merge pull request #15787 from liewegas/wip-19943-fix
ceph-objectstore-tool: do not populate snapmapper with missing clones

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-06-21 14:31:02 -05:00
Vasu Kulkarni
182acd4411 s3a hadoop '2.8.0' tests
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-06-21 11:52:17 -07:00
Vasu Kulkarni
33d8e0714d hadoop s3a '2.7.3' tests
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-06-21 11:52:17 -07:00
Vasu Kulkarni
14b6267cba s3a task to test radosgw compatibility with hadoop s3a interface
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-06-21 11:52:10 -07:00
Kefu Chai
b85e572109 Merge pull request #15813 from tchaikov/wip-cmake
cmake: do not add dependencies to INTERFACE library on cmake < 3.3

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2017-06-22 00:57:17 +08:00
Sage Weil
a9ae65e71b pybind/mgr/restful: remove default path names for the cert
I don't think these are useful.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 12:32:52 -04:00
Sage Weil
4e4d81cd70 pybind/mgr/restful: do not start if no certificate is configured
This removes the default filename, by the way.  We also work around a
problem with make_server where it sets up the socket to listen before
checking for the cert, thereby making it problematic to rebind to the
port shortly thereafter when we do have a socket.  (SO_REUSEADDR would
be appropriate but there doesn't seem to be an easy way to make
make_server use it.)

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-21 12:32:52 -04:00
Boris Ranto
4dc471ed60 restful: Introduce create-self-signed-cert command
This implements a create-self-signed-cert command for the ceph-mgr
restful interface.

It is designed so that it will try to restart the module once the cert
is created.

Signed-off-by: Boris Ranto <branto@redhat.com>
2017-06-21 12:32:49 -04:00
Casey Bodley
81f85d74b2 Merge pull request #15060 from shashalu/fix-slo-dlo-range
rgw: fix SLO/DLO range requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-06-21 12:12:38 -04:00
Sage Weil
be5e6e3662 Merge pull request #15791 from liewegas/wip-20360
Revert "qa: do not restrict valgrind runs to centos"
2017-06-21 10:55:53 -05:00
Kefu Chai
96b2016489 cmake: do not add dependencies to INTERFACE library on cmake < 3.3
otherwise we will have

add_dependencies Cannot add target-level dependencies to INTERFACE
library

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-21 23:34:51 +08:00
Sage Weil
1e7cd3550c Merge pull request #13683 from liewegas/wip-crush-rulesets
crush: detect and (usually) fix ruleset != rule id

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-06-21 08:45:42 -05:00
Matt Benjamin
7873c48eb1 Merge pull request #15789 from linuxbox2/wip-nfs-doc
RGW NFS: add nfs.rst to doc/radosgw
(override because make check fail [missing selinux policy help file] unrelated)
2017-06-21 09:20:40 -04:00
Matt Benjamin
30383502a2 RGW NFS: add nfs.rst to doc/radosgw
This document contains sections by John Wilkins <jowilkin@redhat.com>.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-06-21 08:37:52 -04:00
Abhishek Lekshmanan
72e4f50f25 rgw: add tagging to AWSv4 completion
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-06-21 14:10:45 +02:00
Abhishek Lekshmanan
fcbac83704 rgw_tag: move from std::map -> boost::flat_map
well since flat_map is a vector under the hood it should be faster for
such small sizes, also renaming tags to a more understandable tag_map.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-06-21 14:10:45 +02:00
Abhishek Lekshmanan
9300f0f78b rgw: support IAM policies for object tagging
A very basic support for the s3:{get/put/delete}objecttagging and
related versions for object versioning

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-06-21 14:10:45 +02:00
Abhishek Lekshmanan
72b9d64604 rgw es: add support for obj tags
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 7bff49bd3eadca59d5c23fe1eafa18bb40292463)
2017-06-21 14:10:45 +02:00
Abhishek Lekshmanan
5fb068114b rgw: implement get/put object tags for S3
- New api for get/put/delete object tags
- PUT Object supports x-amz-tagging
- POST API supports x-amz-tagging

This is similar to AWS S3's get/put object tags, tags are stored as an
xattr under the user.rgw.tags key, and are restricted to max 10 tags per
object and key size of 128 and a val size of 256. The API does not reuse
the existing `x-amz-meta` tags.
For PUT obj to optionally accept `x-amz-tagging` and this will end up
storing the tags under x-amz-tagging xattr, the input is expected to be
in the url encoded kv format, we return InvalidTagError if we cannot
parse the tags

A future TODO: use the same xml parser for put and post apis

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

 Conflicts:
   src/rgw/rgw_rest_s3.cc
   src/rgw/rgw_tag.cc
   mostly trivial conflicts from AWSv4 rework, importantly url_decode's
   signature was changed for the nicer, and moved rgw_tag.cc to the
   newer url_decode format

 Conflicts:
  src/rgw/rgw_common.h
  conflict as another PR that used up the error code before this was
  merged
2017-06-21 14:09:49 +02:00
John Spray
170fc31ae1 Merge pull request #15255 from david-z/wip-mgr-fsstatus
pybind/mgr/fsstatus: use mds_mem.dn as dentry counter

Reviewed-by: John Spray <john.spray@redhat.com>
2017-06-21 07:46:02 -04:00
John Spray
20a4762869 Merge pull request #15621 from mingshuaiwang/master
mds: mds perf item 'l_mdl_expos' always behind journaler

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-06-21 07:32:09 -04:00
lisali
51e03c5f42 fio_ceph_objectstore: Print db_statistics when rocksdb_perf is enabled
Currently users can't use ceph admin command to get db stats when running
bluestore fio plugin. This patch is to print db stats when rocksdb_perf
is set in config file.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2017-06-21 07:31:36 -04:00
John Spray
32ff1de9f2 Merge pull request #15662 from renhwztetecs/renhw-wip-client-trim_cache
client: call the lru_remove() twice,when trim cache

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2017-06-21 07:31:27 -04:00
John Spray
623e8b2a13 Merge pull request #15715 from jcsp/wip-jt-rmapply
tools/cephfs: remove `apply` mode of cephfs-journal-tool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-06-21 07:30:53 -04:00
John Spray
21ed003ccf Merge pull request #15779 from renhwztetecs/renhw-wip-client-dentry
client: fix Dentry::dump

Reviewed-by: John Spray <john.spray@redhat.com>
2017-06-21 07:30:34 -04:00
John Spray
7c6990d916 client: check for luminous MDS before sending FLUSH_MDLOG
This is needed to make it safe to run newer clients
on pre-luminous MDS.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-06-21 07:22:58 -04:00
Yan, Zheng
ba7472cbf2 mds: save projected path into inode_t::stray_prior_path
Otherwise, path string like #10000000000/xxx may get saved into
inode_t::stray_prior_path.

Fixes: http://tracker.ceph.com/issues/20340
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-06-21 15:54:28 +08:00
J. Eric Ivancich
bf70c4b9b0 osd/PG: Add two new mClock implementations of the PG sharded operator queue
Create an mClock priority queue, which can in turn be used for two new
implementations of the PG shards operator queue. The first
(mClockOpClassQueue) prioritizes operations based on which class they
belong to (recovery, scrub, snaptrim, client op, osd subop). The
second (mClockClientQueue) also incorporates the client identifier, in
order to promote fairness between clients.

In addition, also remove OpQueue's remove_by_filter and all possible
associated subclass implementations and tests.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-06-20 23:15:36 -04:00
myoungwon oh
5f73991716 osd/osd_types: add flag name (IGNORE_REDIRECT)
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-06-21 11:36:18 +09:00
lu.shasha
9eec1b4ef9 rgw: fix SLO/DLO range requests
Sending any SLO/DLO range requests will retrun InvalidRange error. SLO/DLO should call range_to_ofs after got object size instead of read_op.prepare() to deal with negative offsets.

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
2017-06-21 09:28:24 +08:00