Commit Graph

70244 Commits

Author SHA1 Message Date
Kefu Chai
8a3b48551a osd/OSDMap: use _pg_to_up_acting_osds() to get_pg_acting_primary()
avoid unnecessary swap() call of vector<int> which will be dropped on
the floor after getting its first element, which can be retrieved using
another parameter using _pg_to_up_acting_osds() directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-31 18:32:43 +08:00
Kefu Chai
73c75b97c5 mgr: use OSDMap::get_pg_acting_primary() to get the primary
OSDMap::pg_to_acting_osds() assumes that the "acting" parameter is not
null.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-31 18:31:05 +08:00
Kefu Chai
63f69e04df tests: add MGR=1 so 'pg dump' won't be blocked
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-31 14:26:09 +08:00
Kefu Chai
8f8575a4eb Merge pull request #14260 from dmick/wip-workunit-clone-with-branch
workunit: request branch when cloning

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-31 13:17:46 +08:00
Kefu Chai
9ca7ccf5f1 tasks/workunit.py: specify the branch name when cloning a branch
c1309fb failed to specify a branch when cloning using --depth=1, which
by default clones the HEAD. and we can not "git checkout" a specific
sha1 if it is not HEAD, after cloning using '--depth=1', so in this
change, we dispatch "tag", "branch", "HEAD" using three Refspec classes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-03-30 20:30:09 -07:00
Sage Weil
b782b3716e Merge pull request #14219 from ShiqiCooperation/master
os/bluestore: clean up Invalid return value judgment

Reviewed-by: Sage Weil <sage@redhat.com>
2017-03-30 17:23:09 -05:00
Sage Weil
b0ed37a2dc Merge pull request #14228 from ifed01/wip-bluestore-fix-store-test
test/store_test: fix bluestore test cases disablement

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2017-03-30 17:21:55 -05:00
Sage Weil
578b0f7cfc Merge pull request #13617 from liewegas/wip-mgr-commands
mon,mgr: tag some commands for ceph-mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-30 17:12:00 -05:00
Sage Weil
d0a3f156b2 mon/ConfigKeyService: wait for quorum
Among other things, this prevents a mgr activation stall if it
happens to request config-keys from a mon that is out of
quorum.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-30 17:57:35 -04:00
Sage Weil
319b72380e mon/MgrMonitor: show delta
This is mostly because I couldn't debug a weird state where
it kept updating but it appeared the addr was not changing
(always '-').

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-30 17:57:35 -04:00
Sage Weil
dc49857a65 mon/MgrMap: implement dump, add to ceph-dencoder
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-30 17:57:35 -04:00
Sage Weil
dacb05251d mon/MgrMonitor: print MgrMap to log on each change
The other services do this.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-30 17:57:35 -04:00
Sage Weil
31419d0e0f mon/MgrMonitor: health warn/err if no active mgr
Start warning once mons are luminous; start erroring once
require_luminous is set in osdmap.  Allow a grace period for
mgr to restart or standby to take over before we turn a warning
into an error.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-30 17:57:34 -04:00
Orit Wasserman
0008a1a51d Merge pull request #14183 from joscollin/wip-rgw-cleanup-removeunwantedheaders
rgw: Removed Unwanted headers
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-03-30 20:44:12 +03:00
Yehuda Sadeh
68d26da678 Merge pull request #14194 from joscollin/wip-rgw_rados-warning
rgw: warning, output may be truncated before the last format character

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-03-30 10:43:37 -07:00
Mykola Golub
2e68099a17 Merge pull request #14253 from liupan1111/wip-remove-debug-message
rbd-nbd: remove debug messages from do_unmap

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-03-30 20:24:03 +03:00
Kefu Chai
ea093baec9 Merge pull request #13999 from tchaikov/wip-extract-creating-pgs
mon,osd: decouple creating pgs from pgmap

Reviewed-by: Sage Weil <sage@redhat.com>
2017-03-31 00:35:45 +08:00
Pan Liu
cd748f9b26 rbd-nbd: remove debug message from do_unmap
Global context is not initialized when do_unmap is called.

Signed-off-by: Pan Liu <liupan1111@gmail.com>
2017-03-31 00:23:12 +08:00
Kefu Chai
d3e6a2869b tools/ceph-objectstore-tool: always set first_committed
otherwise the workaround of 7fb3804fb is triggered. we need to remove
that workaround later on.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
9331cedd96 mon: switch to osdmon when handling osd_pg_creates subs
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
616e970ada mon/OSDMonitor: avoid search and lookup anti-pattern
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
ebde28b683 mon/OSDMonitor: pass by reference not pointer of const param
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
f61853ae89 mon/OSDMonitor: switch to the new creating_pgs
* prime_pg_temp(): switch to the new creating_pgs
* get_trim_to(): switch to the new creating_pgs

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
a316eb3078 mon: handle MOSDPGCreated messages
MOSDPGCreated messages are used to prune the creating_pgs_by_osd_epoch
and creating_pgs, by updating created_pools. as once a pool is created
we will not send MOSDPGCreate to its acting_primary OSD anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
a2e39d91c9 osd: send pg-created message if any pg is newly created
add an option named "osd_created_report_interval" to specify the
interval to check and send the "pg_created" mesages to mon

because pg could update its state when it is still in the pg_stat_queue,
for example, to change its state to PG_STATE_CLEAN, we cannot tell if we
have sent a "pg-created" message for it or not without introducing a new
member variable in PG.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:18 +08:00
Kefu Chai
e5ea630e9f messages/MPGCreated: add MPGCreated
add a new message type MPGCreated. osd is supposed to send this message
to monitor to inform that any pg(s) is created and activated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
43ded67243 mon/OSDMonitor: add send_pg_create() to OSDMonitor
OSDMonitor will handle the pg-create subscriptions after luminous.
1. scan new pools to get the pgs to be created
2. send pg creates using the collected pgs
3. trim the creating_pgs using the "created!" messages from OSD.

please note that we need to wait for the OSDMonitor::mapping to be fully
populated, so we cannot scan the incrementa map for creating pgs until
it is applied and accepted by other monitors.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
df6dafe1bb mon: acquire lock when accessing mon->session_map
we will access the mon->session_map for sending the osd-pg-creates
messages when finishing osdmapping in OSDMonitor, this could happen in
another thread without the protection of Monitor::lock, or in the same
thread already guarded by Monitor::lock. so instead of changing
Monitor::lock to a recursive lock, a new lock is introduced to protect
session_map.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
11f82480a0 mon/OSDMonitor: run mapping on peons also
otherwise subcriptions on peons won't get the creating_pgs notification
mapping updated. we want to send the notification from peons also. and
the notifications should be updated with the updated pg mapping.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
19f8db9692 mon/OSDMonitor: mapping is not optional anymore
as pg_creatings needs mapping to get the acting_primary osd of the
creating pg, so we can send the pg-create message to it if it subscribes
to this information, mapping should always be available now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
472c64268d mon: dispatch osd_beacon message to OSDMonitor
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
78484dc71b osd: send osd-beacon to mon periodically
add an option named "osd_beacon_report_interval" to specify the interval
to send osd-beacon.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
8f3432d5b6 messages/MOSDBeacon: add beacon msg
osd will send beacon message to monitor periodically to inform it that
"i am still alive!", previously, monitor use the pg-stats to check the
status of OSD, but since osd will only send pg stat to mgr after
luminous, we use a dedicated msg for this purpose.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
20900e41fb mon/OSDMonitor: s/check_sub/check_osdmap_sub/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
738f947aa9 mon/OSDMonitor: update comment in update_from_paxos()
this change updates the comment for 7fb3804fb, 97462a3 and e807770,
to reflect the reason why we need to fix latest_full in current code.
as the fix is not a workaround for cuttlefish anymore, it resolves the
issue where

0. mon.c has a latest_full of 5
1. mon.c is shutdown and out of sync with the quorum
2. mon.c starts sync
3. mon.c now has osdmap[31..50], and the latest_full is still 5.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
89810eeb31 mon/PGMonitor: remove unused variable
last_pg_scan is not used in PGMonitor::update_from_paxos(), so remove
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
d86f15ca65 mon/PGmonitor: remove unused last_sent_pg_create
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Kefu Chai
454da5e7b2 mon: pass const variables by const ref not pointer
* PGMapUpdater::check_down_pgs(): pass a const reference to pgmap
  instead of a pointer
* PGMapUpdater::register_new_pgs(): pass a const reference to pgmap
  instead of a pointer

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 20:21:17 +08:00
Haomai Wang
1f6750063a Merge pull request #14250 from Adirl/fix_last_poll_dev
msg/async/rdma: Device::last_poll_dev must be positive

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-03-30 18:37:39 +08:00
Amir Vadai
fb7787dd58 msg/async/rdma: Device::last_poll_dev must be positive
Make Device::last_poll_dev `unsigned` - it could overlap and should not
be negative.
Also, add missing initialization.

Change-Id: I558f02292ae4f5d1ce48d9bcda5047d382604237
Signed-off-by: Amir Vadai <amir@vadai.me>
2017-03-30 13:29:08 +03:00
Haomai Wang
20bbdb93a2 Merge pull request #14249 from Adirl/rdma-cm-4_1
msg/async/rdma: Debug prints for ibv_*

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-03-30 17:40:49 +08:00
Amir Vadai
42078045bc msg/async/rdma: Debug prints for ibv_*
Setting RDMA_DEBUG to 1, will enable debug prints for every ibv_*
function called.
This makes it easier to debug RDMA issues.

Issue: 995322
Change-Id: I49d327d5d4386b44264f5619d50f2dbc7d76dcae
Signed-off-by: Amir Vadai <amir@vadai.me>
2017-03-30 11:42:19 +03:00
shiqi
27b100baa4 os/bluestore: clean up Invalid return value judgment
Signed-off-by: shiqi <1454927420@qq.com>
2017-03-30 14:33:53 +08:00
Haomai Wang
5d23e6431c Merge pull request #14245 from tchaikov/wip-revert-14201
Revert "msg/async/rdma: Debug prints for ibv_*"
2017-03-30 13:21:36 +08:00
Kefu Chai
7e0f0748ef Revert "msg/async/rdma: Debug prints for ibv_*"
This reverts commit 7d1475f627.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-30 12:58:48 +08:00
Haomai Wang
4100803cfe Merge pull request #14201 from Adirl/rdma-cm-4
msg/async/rdma: Introduce RDMAConnMgr + Debug prints

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-03-30 08:00:40 +08:00
Dan Mick
b227a70e53 Merge pull request #14214 from dmick/wip-workunit-shallow-clone
tasks/workunit.py: when cloning, use --depth=1

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-29 11:36:05 -07:00
Jason Dillaman
49366c20de Merge pull request #14202 from tchaikov/wip-cast-size_t-arm32
rbd: use min<uint64_t>() explicitly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-03-29 13:33:49 -04:00
Igor Fedotov
51542d9acd test/store_test: fix bluestore test cases disablement
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2017-03-29 16:34:00 +00:00
Sage Weil
cc51f56bb2 mon/PGMap: set initial pg timestamp as osdmap modified stamp
Currently the PGMap may register the new pg in multiple places: the
mgr or mon, and the timestamp is when the mon or mgr gets around to
recording it into its PGMap.  Make this deterministic by using the OSDMap
mtime (which is when the PG was *actually* created).

This fixes a problem where the pg create command has one timestamp (from
the mon) and the pgmap on the mgr has another, so an observer will see the
last_scrub_stamp (and other timestamps) change when nothing has actually
happened.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-29 11:39:27 -04:00