Commit Graph

56341 Commits

Author SHA1 Message Date
Dongsheng Yang
98251eae48 pybind: don't set the default values for the options not specified in copying
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:09 -04:00
Dongsheng Yang
4942076524 pybind: don't set the default values for the options not specified in cloning
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:09 -04:00
Dongsheng Yang
c37df5e691 pybind: don't set the default values for the options not specified in creating
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:09 -04:00
Dongsheng Yang
3508f40037 tools/rbd: set RBD_IMAGE_OPTION_FEATURES_CLEAR when --image-shared specified.
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:09 -04:00
Dongsheng Yang
22f77dd301 librbd: introduce RBD_IMAGE_OPTION_FEATURES_[SET|CLEAR] for features overwriting.
Currently, if we want to use the default options for rbd, we need to omit the
RBD_IMAGE_OPTION_FEATURES, but if we want --image-shared. we need to overwrite
something bese on the default value of image options.

This patch introduce two flags in image_options, RBD_IMAGE_OPTION_FEATURES_SET
means, we want to set something after you get the features from default, parent
or user. And RBD_IMAGE_OPTION_FEATURES_CLEAR will do a bit clear.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:09 -04:00
Dongsheng Yang
dd299e0fca librbd/copy: set RBD_FEATURE_STRIPINGV2 in feature if stripe related options specified
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:08 -04:00
Dongsheng Yang
d97fe01cb1 librbd/clone: set RBD_FEATURE_STRIPINGV2 in feature if stripe related options specified
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:08 -04:00
Dongsheng Yang
6f5571f74a librbd: set the default options when they are not specified.
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:08 -04:00
Dongsheng Yang
a9db19991d tools/rbd: only set the options specified by user in get_image_options().
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-07-22 07:04:03 -04:00
Haomai Wang
4e7946c2f1 Merge pull request #10394 from tchaikov/wip-16772
cmake: disable unittest_async_compressor

Reviewed-by: Haomai Wang <haomai@xsky.com>
2016-07-22 18:23:09 +08:00
Gaurav Kumar Garg
b9a2048c66 rgw: better error message when user has no bucket created yet
Currently when there is no bucket in the cluster and user execute
 "$radosgw-admin user  stats --uid=testid" command then rgw admin
not throwing meaningful error message.

With this fix it will show proper meaningful error message.

Fixes: http://tracker.ceph.com/issues/16444

Reported-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>
2016-07-22 12:14:23 +02:00
Orit Wasserman
95b46c9d5c Merge pull request #10381 from SUSE/wip-doc-rgw-cgi-warning
doc/radosgw: rename config.rst to config-fcgi.rst
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-07-22 12:01:20 +02:00
Abhishek Lekshmanan
d4ecc956f6
rgw multisite: preserve zone's extra pool
In current multisite scenarios,if a bucket is created in master, we end
up storing multipart metadata in `$source-zone.rgw.buckets.non-ec` pool
instead of the zone's own non-ec pool, so we end up additionally
creating this pool and storing multipart metadata entries in it. Also if
a bucket is created in a secondary zone, and we initiate a multipart
upload, before mdlog sync with master, we end up getting errors during
complete multipart requests as omap entries are partly stored in the
`$zone.rgw.buckets.non-ec` as well as `$source-zone.rgw.buckets.non-ec`
pools which leads to total number of parts mismatch.

Fixes: http://tracker.ceph.com/issues/16712

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2016-07-22 10:57:45 +02:00
Kefu Chai
1e8388c345 cmake: disable unittest_async_compressor
as async compressor is not used anywhere. and haomai agrees to disable
this test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-22 16:04:30 +08:00
Kefu Chai
5864626d27 pybind/ceph_argparse: handle non ascii unicode args
we raise UnicodeDecodeError at seeing non-ascii args if we fail to match
it with any command signatures. instead, we should use a unicode string
for representing the error in that case. please note, the exception is
not printed at all in real-world. =)

Fixes: http://tracker.ceph.com/issues/12287
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-22 15:55:18 +08:00
Kefu Chai
f11acf2b15 debian: use python:Depends for ceph-{common,base,osd}'s deps
* ceph-base: use ${python:Depends} instead of listing the python
  dependencies manually, dh_python2 will scan the requirements
  of ceph-detect-init. and fill the subst var for us.
* ceph-common: add ${python:Depends}, as it packages ceph,
  and ceph-brag client.
* ceph-osd: it packages ceph-disk, so should add ${python:Depends}
  as its dependencies.

dh_python2 will figure them out.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-22 15:02:12 +08:00
Kefu Chai
7db7f236b3 debian: depends on python-all-dev instead of python-dev
* debian/control: Build-Depends: s/python-dev/python-all-dev/, per
    https://wiki.debian.org/Python/FAQ#Build_dependencies

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-22 15:02:12 +08:00
Kefu Chai
48f4576de1 debian: remove ${shlibs:Depends} from python-* packages
* debian/control:
  as we have listed the linked libraries in Depends section, for example,
  python-rados depends on librados. and we don't need `dpkg-shlibdeps` to
  help figure out shared library substvar dependencies for us. by removing
  them, we can silence the warnings of
```
warning: dpkg-shlibdeps: package could avoid a useless dependency if
debian/python-rados/usr/lib/python2.7/dist-packages/rados.x86_64-linux-gnu.so
was not linked against libpthread.so.0 (it uses none of the library's
symbols)
```
  -lpthread is introduced by `python-config --ldflags` but it turns out we
  are not using any symbols from pthread in the extension directly. and
  pthread is included in glibc. so this does not added any extra
  dependency to python-* pacakges. but it's desirable to have less
  warnings.
* debian/rules: exclude python-* packages from dh_shlibdeps, as we will
  not use it to prepare the shlib deps substvars for these packages any
  more.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-22 15:02:12 +08:00
Kefu Chai
abe5da0e3a debian: do not run dh-python against non-python packages
some packages do not package python modules or scripts. so override
dh_python2 to exclude them.

this change silences warnings like:
```
warning: dpkg-gencontrol: package ceph-mon: unused substitution
variable ${python:Provides}
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-22 14:23:30 +08:00
Kefu Chai
aacba7f1e8 Merge pull request #8150 from wjwithagen/patch-3
.gitignore: exclude coredumps, logfiles and temporary testresults

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-22 13:39:21 +08:00
Daniel Gryniewicz
1756b74935 Add op to get Lifecycle configuration
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2016-07-22 12:22:23 +08:00
Ji Chen
abe4b1378c RGW:lifecycle feature[rebase]
As same as amazon S3 interface,"PUT Bucket lifecycle" and
"DELETE Bucket lifecycle" have been implemented,
"GET Bucket lifecycle" not realized yet as S3cmd has not
realize it also.
The feature`s main point is to remove expire file per day.
Files transfer from hot layer to cold layer is not supported.
ToDo:Maybe to transfer from replicate pool to EC pool or
from ssd to sata pool will be valuable.

Now put all buckets which should do lifecycle into shard
objects in .rgw.lc pool.

lifecycle config file format:
<LifecycleConfiguration>
    <Rule>
        <ID>sample-rule</ID>
        <Prefix></Prefix>
        <Status>enable</Status>
        <Expiration>
           <Days>1</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>

Signed-off-by: Ji Chen <insomnia@139.com>
2016-07-22 12:22:12 +08:00
Yehuda Sadeh
01b10c3aaa Merge pull request #9089 from SirishaGuduru/bugfix15603
doc: Modification for "TEST S3 ACCESS" section  in "INSTALL CEPH OBJECT GATEWAY" page

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Yehuda  Sadeh <yehuda@redhat.com>
2016-07-21 14:12:09 -07:00
Casey Bodley
692fbf2ae5 Merge pull request #10388 from yehudasa/wip-rgw-sync-err-injection
rgw: add configurables for {data,meta} sync error injection

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-21 17:07:41 -04:00
Yehuda Sadeh
e84148a609 rgw: add configurables for {data,meta} sync error injection
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-21 13:26:09 -07:00
Daniel Gryniewicz
a0294e6450 Revert RGW torrent
Revert the RGW torrent commit for now, it was causing issues.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2016-07-21 12:02:58 -04:00
Michal Jarzabek
4c3a1f81dd msg/Message.h: remove code duplication
Remove code duplication in constructors.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-07-21 16:03:50 +01:00
Kefu Chai
0555abadf0 Merge pull request #10377 from chhabaramesh/extent_alloc
os/bluestore:Fix size calculation in bitallocator

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-21 23:01:35 +08:00
Daniel Gryniewicz
f2bea590f7 Merge pull request #9589 from zhouruisong/master
rgw:add a s3 API of make torrent for a object
2016-07-21 09:54:19 -04:00
Nathan Cutler
d788bae60d doc/radosgw: rename config.rst to config-fcgi.rst
This file documents how to configure RGW to use Apache/FastCGI, so rename
the file and modify the title and intro to make that clear.

Also, add a note that CGI can pose a security risk - e.g. http://httpoxy.org

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-21 14:17:08 +02:00
John Spray
a2e7ea9c78 Merge pull request #10244 from jcsp/wip-cephfs-health-doc
doc/cephfs: explain the various health messages

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-07-21 12:32:45 +01:00
John Spray
ef1405ab19 doc/cephfs: explain the various health messages
Signed-off-by: John Spray <john.spray@redhat.com>
2016-07-21 12:32:05 +01:00
xie xingguo
a55bd6ddb7 os/bluestore: move sanity check to a proper place
This is obviously not the proper place to check the allocation
result. Also the origin check logic is more portable, thus we
drop the "assert(r == 0)" here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-21 18:41:42 +08:00
Ramesh Chander
0be5d5f973 os/bluestore:Fix size calculation in bitallocator
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-07-21 02:47:15 -07:00
Kefu Chai
88f383c7ed Merge pull request #10149 from wjwithagen/patch-12
CMakeList.txt: link ceph_objectstore_tool against fuse only if WITH_FUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-21 17:22:10 +08:00
xie xingguo
fa63e4648c os/bluestore: fix bitmap allocating failure if max_alloc_size is 0
If max_alloc_size is 0(means no limit), the original logic will
always hardcode need_blks to 0, which is incorrect.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-21 16:42:33 +08:00
Willem Jan Withagen
2f7264e815 src/tools/CMakeLists.txt: conditionalise the FUSE usage
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-21 09:59:08 +02:00
Kefu Chai
278ea12b12 Merge pull request #10366 from ceph/wip-cmake-debian
debian: bump compat to 9

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
2016-07-21 11:00:55 +08:00
Kefu Chai
390ac1d630 Merge pull request #10361 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-21 10:42:15 +08:00
Yehuda Sadeh
1b1c86f25c Merge pull request #10355 from yehudasa/wip-16742
rgw: back off bucket sync on failures, don't store marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-20 15:43:40 -07:00
Yehuda Sadeh
aacc31f72c rgw: fix collection of object sync errors
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 14:50:18 -07:00
Yehuda Sadeh
f5801c726e rgw: fix marker tracker completion handling
Was not tracking high markers correctly. Could only work if there was a single
hole in the completion range. Just keep a map of all the complete entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 14:50:18 -07:00
Yehuda Sadeh
5528932e4c rgw: collect() stops if error encoutered
and returns true if needed to be called again

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 12:42:52 -07:00
Kefu Chai
c1c6cfc49c debian: bump compat to 9
dh_auto_configure set -DCMAKE_BUILD_TYPE=None so the default does not
take effect at all. this is on purpose, see [1].

and dpkg-buildflags is able to produce the suggested compiling flags by
debian policy. dh_* can pass these flags to cmake only if it works at
compat>=9, see [2].

and we are safe to move to compat 9, as jewel's supported debian based
distro is jessie (with debhelper 9.20150101), ubuntu trusty (with
debhelper 9.20131227ubuntu1)

---
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701233#35
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653916

Fixes: http://tracker.ceph.com/issues/16744
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-21 02:02:58 +08:00
Casey Bodley
d21a3cd3f2 Merge pull request #10368 from oritwas/wip-rgw-upgrade-period-commit
rgw: fix upgrade from old multisite to new multisite configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-20 13:53:56 -04:00
Ali Maredia
31b2f483d5 Merge pull request #10360 from ceph/wip-include-include
cmake: add_subdirectory(include)

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-20 12:56:34 -04:00
John Spray
e0a1321ced Merge pull request #10315 from onyb/wip-cephfs-pypi
pybind: Rework cephfs/setup.py for PyPI

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-20 17:34:05 +01:00
John Spray
297500399e Merge pull request #10316 from batrick/mds-standby-assertions
mds: add assertions for standby_daemons invariant

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:43:43 +01:00
John Spray
92c589032f Merge pull request #10347 from renhwztetecs/renhw-wip-test-add-fsdump
test/mds: add fs dump in test_ceph_argparse.py

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:43:00 +01:00
John Spray
27574785df Merge pull request #10202 from xiaoxichen/fix_16588
mds/MDSMap default metadata pool to -1 (was: output None instead of 0 when no fs present.)

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:36:05 +01:00