Commit Graph

54236 Commits

Author SHA1 Message Date
Nathan Cutler
048aa3f0e7 Merge pull request #9106 from SUSE/wip-15869
rpm: unconditionally set ceph user's primary group to ceph (SUSE) 

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2016-06-01 18:14:38 +02:00
Kefu Chai
bbf51e1562 cmake: install cython modules
* fix CYTHON_ADD_MODULE() macro. because python_add_module() offered by
  FindPythonLibs.cmake creates a target with name of ${name}, which conflicts
  with existing targets like "rbd" or "rados". so we can not reuse the
  name in ${name}.pyx. and instead, we should specify the target name
  explicitly.
* add distutils_install_cython_module() function to build and install
  cython modules.
* we can split build and install of cython module, but the install phase
  always tries to build the module. so keep it this way. will look at it
  later on.
* move the variables initializations into the Distutils.cmake module.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:50:05 +08:00
Kefu Chai
dfea8e41bd cmake: install compressor plugins into ${pkglibdir/compressor
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
4f05601730 cmake: install erasure plugins into ${pkglibdir}/erasure-code
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
225f345642 cmake: remove duplicated file from ceph-osd
objclass/class_api.cc is already included in libosd

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
c389c8d377 cmake: fix dependencies on tracing headers
group the header dependencies by its tp .so, so the traced target can
depend on them by the name of ${name}-tp.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
8588e3d55e cmake: libradosstriper's OUTPUT_NAME should be radosstriper
and s/libradosstriper/radosstriper/ otherwise the created .so
filename would be liblibradosstriper.so with the default prefix.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
6936a811a3 cmake: install the artifacts the packaging requires
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
33d50ed004 cmake: install init script to etc/init.d
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
ce942bd619 cmake: add the autoconf path variables back
* partially revert 7a602ec.
* the directory variables created by automake, like "prefix", "bindir",
  and "libdir", are used for generating configuration_file() for substitution,
  and they should have the same names with ones from autotools.
* also fix the ${pkglibdir}, it should be the ${libdir}/${PACKAGE}. so
  the plugins are not installed into ${libdir}, in which the installed shared
  objects are supposed to be shared with other applications.
* install shared libraries into ${CMAKE_INSTALL_LIBDIR} instead of
  ${prefix}/lib. this complies to what ceph.spec.in requires:
  ceph.spec.in expects the shared libraries to be installed into
  ${_libdir}, and ${_libdir} is /usr/lib64 on an amd64 machine.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
c192846c06 cmake: install ceph-post-file
and related pubkey for sftp

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
43d78c9939 cmake: add ceph-brag
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
c1a37101ea cmake: project name should be "ceph"
so it would be easier to figure out paths, also
CMAKE_INSTALL_DOCDIR is composed using the PROJECT_NAME also.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
b02aaebc45 cmake: install ceph-{detect-init,disk}
add a cmake module named Distutils.cmake for setting up python modules
using setup.py.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
ed888f6983 cmake: add ceph_rgw_{json,multi}parser
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
f67ebef597 cmake: compile and install ceph-bluefs-tool
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:45 +08:00
Kefu Chai
6df0bc8691 cmake: s/ceph_test_xattr_bench/ceph_xattr_bench/
to match with automake and packager

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:45 +08:00
Kefu Chai
81cd483936 cmake: rename ceph-psim to ceph_psim
to match with automake and packager

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:45 +08:00
Kefu Chai
b6da5c5135 cmake: fix bash_completion install path
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:45 +08:00
Sage Weil
1c2c6cc1d8 os/bluestore: fsck: check for dup overlay keys
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
380392fcec os/bluestore: fsck: use common helper to verify blobs and refs
The checks are the same (or should be--we had missed a few).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
cb7f0f75d3 os/bluestore: add FLAG_NOCACHE flag; do not cache unbuffered writes
Add a Buffer flag to mark that a buffer should not be cached once it is
stable.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
029ced9680 os/bluestore: do not use buffered bdev in write path
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
b6d7c97daf os/bluestore: do not rely on bdev buffered reads in read path
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
247bcdbbfd os/bluestore: no buffered write in wal path
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
7679407378 os/bluestore: populate buffer cache on read
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
74dd955728 os/bluestore: keep intrusive_list of WRITING buffers
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
6e5249a4be os/bluestore: simple per-collection lru for buffers
Size these using a global config.  This is only a starting point--we'll
obviously have to rework this to share memory across collections.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Sage Weil
63279896a0 os/bluestore: use bufferptr for csum_data
encode/decode of vector<char> is not optimized.  Bufferptr is a more
natural type here anyway.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:50 -04:00
Igor Fedotov
6cec4f388d ceph_test_objecstore: Adds a test case for compression stuff verification (incomplete)
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Igor Fedotov
c307c0ffdd os/bluestore: Fixes configuration observation.
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Igor Fedotov
8f8fd3a722 os/bluestore: Cosmetic fixes in bluestore logging
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Igor Fedotov
e2f6a66ded os/bluestore: Enables cow for cloning at bluestore for store test
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Igor Fedotov
c7ed3aa7a2 os/bluestore: Fixes Bnode serialization/deserialization and removes legacy Bnode::ref_map
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Igor Fedotov
b0981b3391 ceph_test_objectstore: extends SimpleObjectTest with the case where write happens for neighboring csum blocks to verify for potential alignment issue
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Igor Fedotov
5361cb887f os/bluestore: Removes legacy block_size retrieval
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:49 -04:00
Sage Weil
326bb0f865 os/bluestore: use WriteContext and do_alloc_write for _do_write_small
Kill some mostly-duplicated code

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:49 -04:00
Sage Weil
eb977e6ba3 os/bluestore: consolidate WriteContext items into a write_item
Also include b_off in there.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:49 -04:00
Sage Weil
3d49c2eb57 os/bluestore: avoid unnecessary write_onode calls
_wctx_finish callers always write the onode; we only need to worry about
our changes to the bnode.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:49 -04:00
Sage Weil
b0cabb78db os/bluestore: drop unused _pad_* methods
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:49 -04:00
Sage Weil
24578bc8f8 os/bluestore: drop unused _pad_zeros args
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:49 -04:00
Sage Weil
a9a5e63d99 os/bluestore: fix offset skew check
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Sage Weil
ea1a787c65 os/bluestore: ~0x -> ~
e.g., 0x432da000~1000 instead of 0x432da000~0x1000

I think it's sufficiently clear that the value after ~ should have the same
base as the first bit, and it's easier to read.  And less text.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Igor Fedotov
183db05a35 compressor: Extends decompressor interface to be able to provide compressed data length.
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-06-01 11:40:48 -04:00
Sage Weil
fe6aaca1db os/bluestore: compress on write
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Sage Weil
502473a95c os/bluestore: do not partially deallocate compressed blobs
Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Sage Weil
931264ecab os/bluestore: _do_write_big: limit size of blobs based on compression mode
We may want to compress in smaller chunks based on hints/policy.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Sage Weil
53b73328dd os/bluestore: track new compression config options
Class-wide Compressor, compression mode, and options.  For now these are
global, although later we'll do them per-Collection so they can be pool-
specific.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Sage Weil
a7c9c84eac os/bluestore/bluestore_types: add length to the compression_header_t
Snappy fails to decompress if there are extra zeros in the input buffer.
So, store the length explicitly in the header to avoid feeding them into
the decompressor.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00
Sage Weil
c4f1facaa5 os/bluestore: fix BufferSpace::read()
- we weren't reading from 'clean' buffers
- restructured loop a bit chasing another bug (but it ended up being
  in the caller)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-06-01 11:40:48 -04:00