Commit Graph

100899 Commits

Author SHA1 Message Date
Yingxin Cheng
f8053d08ee crimson/net: STANDBY state for lossless server or peer
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:22:38 +08:00
Yingxin Cheng
dd59586ef0 crimson/net: allow REPLACING state wait for protocol exit
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:22:23 +08:00
Yingxin Cheng
49a08e8bc3 crimson/net: send AckFrame for lossless policy
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:18:15 +08:00
Yingxin Cheng
6cacf1f7b2 crimson/net: maintain the sent queue for lossless policy
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:17:43 +08:00
Yingxin Cheng
492263962c crimson/net: reset write state with reset_write()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:52 +08:00
Yingxin Cheng
babc9c24fd crimson/net: allow connecting state reentrant
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:52 +08:00
Yingxin Cheng
675a50326c crimson/net: reset handshake status when connecting/accepting
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:52 +08:00
Yingxin Cheng
b7c7dc0b26 crimson/net: pending_q to store the pending(sending) messages
We cannot left the pending messages in the out_q, because with lossless
policy, they can be partially sent and even acknowledged.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:52 +08:00
Yingxin Cheng
4fa1c4c07d crimson/net: wait_write_exit() to wait for writer stopped
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:51 +08:00
Yingxin Cheng
b3f1e56d6c crimson/net: is_queued() to check if there's any pending writes
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:51 +08:00
Yingxin Cheng
04f8a35d79 crimson/net: fix variables for stateful connection
server_cookie, client_cookie, connect_seq and global_seq are identifiers
of a stateful connection.

We already have some related implementations, but they are stub code
when implement lossy policy and cannot work properly.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 17:02:42 +08:00
Jeegn Chen
3bfb5c2621 osd: support osd_scrub_extended_sleep
1. always take osd_scrub_sleep for manually initiated
   scrubs
2. when scrub_time_permit() return true for scheduled
   ones, the existing osd_scrub_sleep is used
3. when scrub_time_permit() return false for scheduled
   ones, there may be 2 scenarios
   3.1 if osd_scrub_extended_sleep <= osd_scrub_sleep,
       let's take osd_scrub_sleep
   3.2 otherwise, let's take osd_scrub_extended_sleep

Fixes: http://tracker.ceph.com/issues/40955
Signed-off-by: Jeegn Chen <jeegnchen@tencent.com>
2019-08-12 16:54:36 +08:00
Yingxin Cheng
469a9cda73 crimson/net: clean up, exsiting_conn and existing_proto
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 16:34:45 +08:00
Yingxin Cheng
014a662b20 crimson/net: next_step_t for explicit decision of next state
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 16:34:45 +08:00
Yingxin Cheng
b41af731b4 crimson/net: introduce 3 ways to abort the active protocol state
* abort_in_fault(): a fault is happening and needs to be handled.
* abort_protocol(): abort the current protocol state due to preemptive
                    state change.
* abort_in_close(): close this connection and abort the current protocol
                    state due to some fatal error.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-08-12 16:34:38 +08:00
Kefu Chai
2e2414b3df ceph-objectstore-tool: update-mon-db: do not fail if incmap is missing
there is chance that we could use an OSD which does not have incmap of a
certain epoch for rebuilding the monstore. and since OSD does not read
and store the incmap if the MOSDMap message already has the fullmap of
that fullmap, and if an OSD does not have previous fullmap, monitor
will just send it the fullmao. so it's not unusual that an OSD has
a fullmap of some epoch without corresponding incmap.

Fixes: https://tracker.ceph.com/issues/41177
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-12 13:06:01 +08:00
Matt Benjamin
801d2f0449
Merge pull request #28157 from Kriechi/docs-rgw-ldap
docs: improve rgw ldap auth options
2019-08-11 20:45:29 -04:00
Yuval Lifshitz
929c062ae9 rgw: don't throw when accept errors are happening on frontend
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-08-11 10:06:05 +03:00
Josh Durgin
3f18ed55aa
Merge pull request #28227 from sseshasa/monCachePriority
mon/OSDMonitor: Use generic priority cache tuner for mon caches

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-09 14:23:39 -07:00
Casey Bodley
bc45261470
Merge pull request #29540 from cbodley/wip-rgw-user-rename
rgw: followup for 'user rename'

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-09 16:57:25 -04:00
Casey Bodley
13f1595335
Merge pull request #29558 from theanalyst/rgw-cache-lock
rgw: fix unlock of shared lock in RGWCache

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-09 13:03:35 -04:00
Abhishek Lekshmanan
2b6dbe31c8 rgw: fix unlock of shared lock in RGWCache
similar to https://github.com/ceph/ceph/pull/29538/ we unlock a shared_lock with
unlock causing a crash. Also scope the single line if statements to make the
code more concise

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-09 17:54:22 +02:00
Sage Weil
f011c13547 Merge PR #29292 into master
* refs/pull/29292/head:
	os/bluestore: warn on no per-pool omap
	os/bluestore: fsck: warning (not error) by default on no per-pool omap
	os/bluestore: fsck: int64_t for error count
	os/bluestore: default size of 1 TB for testing
	os/bluestore: behave if we *do* set PGMETA and PERPOOL flags
	os/bluestore: do not set both PGMETA_OMAP and PERPOOL_OMAP
	os/bluestore: fsck: only generate 1 error per omap_head
	os/bluestore: make fsck repair convert to per-pool omap
	os/bluestore: teach fsck to tolerate per-pool omap
	os/bluestore: ondisk format change to 3 for per-pool omap
	mon/PGMap: add data/omap breakouts for 'df detail' view
	osd/osd_types: separate get_{user,allocated}_bytes() into data and omap variants
	mon/PGMap: fix stored_raw calculation
	mon/PGMap: add in actual omap usage into per-pool stats
	osd: report per-pool omap support via store_statfs_t
	os/bluestore: set per_pool_omap key on mkfs
	osd/osd_types: count per-pool omap capable OSDs
	os/bluestore: report omap_allocated per-pool
	os/bluestore: add pool prefix to omap keys
	kv/KeyValueDB: take key_prefix for estimate_prefix_size()
	os/bluestore: fix manual omap key manipulation to use Onode::get_omap_key()
	os/bluestore: make omap key helpers Onode methods
	os/bluestore: add Onode::get_omap_prefix() helper
	os/bluestore: change _do_omap_clear() args

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-09 10:40:45 -05:00
Alfredo Deza
8363d89a4d
Merge pull request #29528 from tchaikov/wip-build-doc-with-python3
admin/build-doc: use python3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-08-09 11:17:19 -04:00
Matt Benjamin
a7b29647fd
Merge pull request #29560 from linuxbox2/wip-rgwf-advance
rgw_file: dont deadlock in advance_mtime()
2019-08-09 10:01:03 -04:00
Sage Weil
b8501164ef os/bluestore: warn on no per-pool omap
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 08:21:18 -05:00
Jan Fajerski
b8d6dcbe9f ceph-volume: never log to stdout, use stderr instead
We should never print log messages to stdout, as this should be reserved
for output of ceph-volume.

Fixes: https://tracker.ceph.com/issues/41158

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-08-09 14:26:16 +02:00
alfonsomthd
bc8e811b08 mgr/dashboard: adapt bucket tenant tests to new behaviour
Fixes: https://tracker.ceph.com/issues/41175
Signed-off-by: alfonsomthd <almartin@redhat.com>
2019-08-09 10:05:46 +02:00
Kefu Chai
2db496017a
Merge pull request #29400 from wjwithagen/wjw-fix-do_freeBSD.sh
do_freebsd.sh: update build scripts to resemble Jenkins scripts

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-09 14:09:42 +08:00
Kefu Chai
ed8ff905fc
Merge pull request #29495 from ifed01/wip-ifed-finisher-improve
common/Finisher: remove some lock acquisitions.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-09 12:24:28 +08:00
Kefu Chai
e5165db985
Merge pull request #29477 from tchaikov/wip-osd-pg-as-mutex
osd: pg as a mutex

Reviewed-by: Greg Farnum gfarnum@redhat.com
Reviewed-by: Samuel Just <sjust@redhat.com>
2019-08-09 12:22:26 +08:00
Kefu Chai
abce85bbd2
Merge pull request #29486 from runsisi/wip-fix-verbose
ceph.in: fix verbose print

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-09 12:21:29 +08:00
Kefu Chai
32b33d3258
Merge pull request #29465 from penglaiyxy/wip_bluestore_caculated_revert
os/bluestore: no need to add tail length (revert PR#29185)

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-08-09 12:20:46 +08:00
Kefu Chai
88c6c039d1
Merge pull request #29488 from majianpeng/bluestore-remove-lock
os/bluestore: no need protected by OpSequencer::qlock.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-08-09 12:19:27 +08:00
Kefu Chai
3efc51fa1d
Merge pull request #29385 from kamoltat/wip-qa-tasks-mgr-test-progress-bug-fix
qa/tasks/mgr/test_progress.py: fix bug in 9b4dbf0

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-09 12:18:40 +08:00
Dan Mick
8096d69f9a
Merge pull request #29365 from liewegas/wip-telemetry-post
telemetry/server: misc fixes
2019-08-08 16:32:15 -07:00
Matt Benjamin
0d3338c2d2 rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-08 18:57:15 -04:00
Sage Weil
6c690ae285 os/bluestore: fsck: warning (not error) by default on no per-pool omap
Unconditionally error if the global per_pool_omap was set, though!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:37 -05:00
Sage Weil
dee8f8cb7e os/bluestore: fsck: int64_t for error count
Just to be on the safe side.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
dbdd1d9b6e os/bluestore: default size of 1 TB for testing
Less ENOSPC from vstart.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
52a2d4b24d os/bluestore: behave if we *do* set PGMETA and PERPOOL flags
pgmeta trumps perpool.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
9bbe8d04c7 os/bluestore: do not set both PGMETA_OMAP and PERPOOL_OMAP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
1eb10f3233 os/bluestore: fsck: only generate 1 error per omap_head
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
b1e44c3298 os/bluestore: make fsck repair convert to per-pool omap
- error if objects not per-pool
- error if per_pool_omap not set
- convert as we go, and set the flag at the end

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Yuri Weinstein
690385362c
Merge pull request #29541 from yuriw/wip-yuriw-upgrade-rbd-master
qa/tests: added rbd task on ec
2019-08-08 15:22:48 -07:00
Sage Weil
0d9410036a Merge PR #27542 into master
* refs/pull/27542/head:
	qa/tasks: Fix ambiguous store_thrash, thrash_store

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-08 15:52:58 -05:00
Sage Weil
8827bc1022 Merge PR #29493 into master
* refs/pull/29493/head:
	qa/tasks/mgr/mgr_test_case: get mgrmap from 'mgr dump', not status
	qa/tasks/ceph_manager: no newlines in 'ceph -s' output
	mon: make mon summary more concise in 'ceph -s'
	mon/MgrStatMonitor: set initial service_map 'modified' to cluster mkfs
	mon: remove double-nesting of "osdmap" for ceph status
	mon/MgrMap: make print_summary (used by 'ceph -s') more concise

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-08 15:52:45 -05:00
Sage Weil
a0fca4eaf6 Merge PR #29511 into master
* refs/pull/29511/head:
	common/config: respect POD_MEMORY_REQUEST *and* POD_MEMORY_LIMIT env vars
	common/config: let diff show non-build defaults
	common/config: do no include multiple 'default' values

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-08 15:49:29 -05:00
Sage Weil
01fcf0e401 Merge PR #29517 into master
* refs/pull/29517/head:
	osd: include PastInterals in pg_notify_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-08-08 15:49:17 -05:00
Adam C. Emerson
ad88d4dced Merge pull request #29559 from adamemerson/wip-putobj-crash
rgw: Move upload_info declaration out of conditional

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2019-08-08 15:14:03 -04:00