Commit Graph

72011 Commits

Author SHA1 Message Date
Jason Dillaman
58b6520707 common/bit_vector: utilize deep-copy during data decode
BlueStore utilizes an in-memory cache of bufferlists. If the bit vector
directly manipulates the shallow-copied bufferlist, it can corrupt
the CRC.

Fixes: http://tracker.ceph.com/issues/19863
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-05-09 11:31:52 -04:00
Sage Weil
7a4dfe8949 Merge pull request #14952 from tchaikov/wip-19813
mgr: do the shutdown in the right order

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-09 10:01:12 -05:00
Kefu Chai
a0ee9e663a Merge pull request #14478 from theanalyst/doc/dev/ccache
doc: dev add a note about ccache

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-09 20:46:29 +08:00
Kefu Chai
1116556ff4 Merge pull request #14886 from badone/wip-async-osd-scrub-sleep
osd: Implement asynchronous scrub sleep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-09 20:27:17 +08:00
Kefu Chai
73340eedb8 Merge pull request #14985 from johnzzpcrystal/store_errno
msg/async/net_handler: errno should be stored before calling next function

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-05-09 20:26:24 +08:00
Abhishek Lekshmanan
1d9558ac42 doc: dev add a note about ccache
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-05-09 10:53:50 +02:00
Dan Mick
6a70b0705f Merge pull request #14984 from dmick/wip-osd-dup
test/osd/osd-dup.sh: lower wb fd throttle limits

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-08 19:48:27 -07:00
Yan, Zheng
69de7bc3a2 Merge pull request #14907 from ukernel/wip-19820
ceph-fuse: use user space permission check by default
2017-05-09 10:34:00 +08:00
Jason Dillaman
41dd93d740 Merge pull request #14976 from trociny/wip-19871
rbd-nbd: relax size check for newer kernel versions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-08 19:41:40 -04:00
Jason Dillaman
a5c36ced4b Merge pull request #14656 from chardan/jfw-wip-halflife_atomic_t-rodan
rbd, librbd: migrate atomic_t to std::atomic

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-08 19:41:02 -04:00
Matt Benjamin
7b2cd270b6 Merge pull request #14925 from fangyuxiangGL/fix_unneccesary_write
rgw: don't do unneccesary write if buffer with zero length
2017-05-08 15:52:00 -04:00
Sage Weil
0b449db512 Merge pull request #14990 from wjin/fix
osd: clean nonused work queue

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-08 11:34:48 -05:00
Sage Weil
39d5bdf0d0 Merge pull request #14962 from liewegas/wip-denc-dump
include/denc: support ENCODE_DUMP

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-05-08 11:33:31 -05:00
Kefu Chai
a4bffab1e5 Merge pull request #14995 from SUSE/wip-mgr-rest-exit
pybind/mgr/rest: completely terminate cherrypy in shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-09 00:18:55 +08:00
Sage Weil
439fdbe3bf include/denc: clean up ENCODE_DUMP a bit
- PATH_MAX for string
- ENCODE_DUMP -> ENCODE_DUMP_PATH
- some comments!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-08 11:54:54 -04:00
Sage Weil
840581e241 Merge pull request #14999 from ivancich/wip-mod-git-subtree-README
doc: Re-word the warnings about using git subtrees.
2017-05-08 09:22:38 -05:00
J. Eric Ivancich
b2d4b48152 doc: Re-word the warnings about using git subtrees.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-05-08 10:17:47 -04:00
Jesse Williamson
40762b8985 rbd_mirror: migrate std::atomic<unsigned> to std::atomic<bool>
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2017-05-08 07:14:04 -07:00
Jesse Williamson
08c6d5d5d1 librbd: migrate atomic_t to std::atomic<>
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2017-05-08 07:14:04 -07:00
Jesse Williamson
93621f5cb8 tools (rbd*): migrate atomic_t to std::atomic<>
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2017-05-08 07:13:57 -07:00
Tim Serong
afd9c3a93e pybind/mgr/rest: completely terminate cherrypy in shutdown
cherrypy.engine.stop() doesn't actually completely kill cherrypy,
which means cherrypy.engine.block() never returns, so this module's
thread never completes cleanly.

Signed-off-by: Tim Serong <tserong@suse.com>
2017-05-08 23:03:55 +10:00
Haomai Wang
867b1d83c4 Merge pull request #14956 from liewegas/wip-bdev-ioc-lock
os/bluestore: fix use after free race with aio_wait

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Jianpeng Ma <jianpeng@intel.com>
2017-05-08 20:14:31 +08:00
Wei Jin
27cc945a6d osd: clean nonused work queue
it was restructured by 3cc48278bf

Signed-off-by: Wei Jin <wjin.cn@gmail.com>
2017-05-08 15:25:46 +08:00
Sage Weil
916c5e3332 Merge pull request #14919 from tchaikov/wip-19825
mon: check is_shutdown() in timer callbacks

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-07 22:26:03 -05:00
Sage Weil
fd6178082a Merge pull request #14968 from TsaiJin/wip-lock-pg-when-scrub-available
osd: sched_scrub() lock pg only if all scrubbing conditions are fulfilled

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Pan Liu <liupan1111@gmail.com>
2017-05-07 22:23:55 -05:00
Kefu Chai
2b8961c973 mgr: do the shutdown in the right order
Fixes: http://tracker.ceph.com/issues/19813
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-06 22:45:16 +08:00
Kefu Chai
260fea0f59 mgr: do not use pointers for MgrStandby members
it's simpler and easier to manage the life cycle of them this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-06 22:45:16 +08:00
Loic Dachary
6152fd9f01 Merge pull request #14970 from dachary/wip-crush-documentation
crush: update documentation for negative choose step

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-05-06 08:09:03 +02:00
Zhou Zhengping
1dbcd90cb5 msg/async/net_handler: errno should be stored before next calling
Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com>
2017-05-06 08:51:43 +08:00
Sage Weil
952a7beb61 Merge pull request #14979 from liewegas/wip-19874
mon: fix prime_pg_temp overrun
common/Finisher: fix uninitialized variable warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-05 17:42:38 -05:00
Sage Weil
209a4b8f3a Merge pull request #14958 from ovh/bp-fix-finisher-warning
common/Finisher: fix uninitialized variable warning
common/Finisher: fix uninitialized variable warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-05 17:42:26 -05:00
Sage Weil
7f72100be5 Merge pull request #14793 from dmick/wip-prio-and-daemonperf
ceph: perfcounter priorities and daemonperf updates to use them
2017-05-05 17:41:55 -05:00
Sage Weil
7c8a2a1766 Merge pull request #14510 from liewegas/wip-bluefs-sync-write
os/bluestore/BlueFS: add bluefs_sync_write option

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-05-05 17:41:22 -05:00
Sage Weil
425769fc10 Merge pull request #14305 from liewegas/wip-zipkin-partial
osd: zipkin tracing
2017-05-05 17:40:58 -05:00
Sage Weil
c5a46089f0 messages/MCommand: fix type on decode
Wow, this has been broken since v0.38, but apparently
the message never made it into the object corpus so
we never noticed!

In reality the bug is harmless: decode_message() will
set_header which clobbers whatever version the default
ctor fills in, so this only affects ceph-dencoder's
test.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 18:02:45 -04:00
Sage Weil
0eb0694002 ceph-object-corpus: kraken objects
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 18:02:45 -04:00
Dan Mick
b37d0b49da test/osd/osd-dup.sh: lower wb fd throttle limits
osd-dup.sh was losing OSDs to EMFILE, mostly on arm64 (not clear why).

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-05-05 14:47:14 -07:00
Orit Wasserman
0c5b2a9026 Merge pull request #14981 from cbodley/wip-rgw-bi-list-plain
cls/rgw: list_plain_entries() stops before bi_log entries
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-05-06 00:04:11 +03:00
Sage Weil
b89dff472c test/encoding/generate-corpus-objects: fix typo
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 16:31:02 -04:00
Casey Bodley
b29a1633a5 cls/rgw: list_plain_entries() stops before bi_log entries
list_plain_entries() was using encode_obj_versioned_data_key() to set
its end_key, which gives a prefix of BI_BUCKET_OBJ_INSTANCE_INDEX[=2]

that range between start_key and end_key would not only span the
BI_BUCKET_OBJS_INDEX[=0] prefixes, but BI_BUCKET_LOG_INDEX[=1] prefixes
as well. this can result in list_plain_entries() trying and failing to
decode a rgw_bi_log_entry as a rgw_bucket_dir_entry

Fixes: http://tracker.ceph.com/issues/19876

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-05 15:48:20 -04:00
Sage Weil
b38b663a71 test/encoding/generate-corpus-objects.sh: simplify object corpus population
Script that generates corpus objects.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 15:20:18 -04:00
Sage Weil
40fe1d181c qa/workunits/rgw/run-s3tests.sh: run s3-tests
This works out of the box with a vstart environment and

 RGW=1 ../src/vstart.sh -n -l
 PATH=bin:$PATH ../qa/workunits/rgw/run-s3tests.sh

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 15:20:18 -04:00
Jason Dillaman
128f593fe8 Merge pull request #14051 from SpandanKumarSahu/bug#17195
rbd: stop indefinite thread waiting in krbd udev handling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-05 14:40:55 -04:00
Sage Weil
e8ab3fdc87 os/bluestore: remove useless IOContext::num_reading
If we are a syncrhonous read, we don't need this: we don't aio_wait for
sync reads.  If we are an aio_read, we are in the aio_running count anyway,
and there is also no purpose for this counter.

I'm a bit unsure about the NVME use of this counter; I switched it to use
num_running (pretty sure we aren't mixing reads and writes on a single
IOContext) *but* it might make more sense to switch to a private counter.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:39:30 -04:00
Sage Weil
1026593173 os/bluestore/BlockDevice: fix racy aio_{wake,wait}() on IOContext
Thread 1 (_do_read)                 Thread 2 (_aio_thread)
queues aio
ioc->aio_wait()
  locks ioc->lock
  num_waiting++
                                    finishes aios
                                    ioc->aio_wake
                                      if (num_waiting)
                                        lock ioc->lock (blocks)
  num_running == 0, continue
  ioc->unlock
                                        ioc->lock taken
do_read destroys ioc
                                        use after free, may block forever

The last bit of the race may vary; the key thing is that thread 2 is
taking a lock that thread 1 can destroy; thread 2 doesn't have it pinned
in memory.

Fix this by simplifying the aio_wake, aio_wait.  Since it is mutually
exclusive with a callback completion, we can avoid calling it at all when
a callback in present, and focus on keeping it simple.

Avoid use-after-free by making sure the last decrement happens under
the lock in the aio_wake/wait case.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:39:30 -04:00
Sage Weil
5457991143 os/bluestore/KernelDevice: remove weird aio_wake from sync read()
I'm not entirely sure why we were ever signaling the cond here; the read
is synchronous!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:39:29 -04:00
Sage Weil
b4eab62c5e CMakeLists.txt: WITH_BLKIN off for now
We are unconditionally linking lttng-ust, which make selinux complain. We
should either

 - fix selinux rules and unconditionally link
 - dlopen at runtime based on an option (like we do for the current
   tracepoints)
 - ???

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:05:24 -04:00
Sage Weil
3eb554c1fc blkin: fix test
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:05:24 -04:00
Sage Weil
32d3119696 blkin: fix cmake compile options
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:05:24 -04:00
Sage Weil
90e52907be blkin: remove installed files
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-05 14:05:24 -04:00