Commit Graph

80999 Commits

Author SHA1 Message Date
Sage Weil
553048fbf9 osd/OSDMap: track newly removed and purged snaps in each epoch
Instead of maintaining a set of snapids that have been removed over
all time, instead note just the newly removed and newly purged snaps
in each OSDMap epoch.  This is easier to consume for both the Objecter
and OSD.

Also keep the interval of snaps that have been removed but not perged
in each OSDMap.  This is extremely convenient because it frees the OSDs
from having to maintain this information in parallel even when they may
not have PGs belonging to those pools.  These structures will be large
right when the ugprade happens and the pg_pool_t::removed_snaps gets copied
to the new fields, but in the steady state it will be relatively small,
reflecting only the set of snaps that are currently being removed.

This also provides convenient visibility into the "trimming snaps" set
that the cluster is working on.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:23 -06:00
Sage Weil
37c4affa25 mon/OSDMonitor: clear pending_metadata* in create_pending
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:23 -06:00
Sage Weil
1f133a2350 mon/OSDMonitor: reset OSDMap state before decode
This ensures we don't have any cruft left over in fields that decode()
assumes are initialized from the ctor (and not a previous instance).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
e89649dca5 mds/SnapServer: fix reset()
pg_pool_t's snap_seq is an upper bound on any allocated (or removed)
snapids.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
b9c5a24395 osd/osd_types: SnapSet: remove get_first_snap_after()
No users.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
1b1eec29ae include/types: flat_set operator<<
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
3119cf5cee include/mempool: add flat_set alias
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
ea308ad54e include/interval_set: add get_end() to iterator
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
df7523b882 qa/suites/rados/singleton/all/thrash-eio: more whitelist
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
81d63f2994 osd/OSDMap: improve osdmap flag dumping in json
Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
c8bfe3fa53 osd/PG: share_pg_info shares past_itnervals, not PastIntervals()
In reality we only call this when the PG is peered and thus past_intervals
is empty, but this is more defensive in case that changes someday!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:16:22 -06:00
Sage Weil
c536d4c294 osd/osd_types: note about removed_snaps hack
We add in the new snap_seq just to try to keep the interval_set
contiguous.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-12-01 21:15:02 -06:00
Sage Weil
27e06ffa4c
Merge pull request #18974 from liewegas/wip-22145
osd/PG: restart recovery if NotRecovering and unfound found

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-11-29 12:48:37 -06:00
Sage Weil
4cfe31c63b osd/PG: restart recovery if NotRecovering and unfound found
If we are in recovery_unfound state waiting for unfound objects, and we
find them, we need to restart the recovery reservation process so that we
can recover.  Do this by queueing DoRecover() event instead of calling
queue_recovery() (which won't do anything since we're not in
recoverying|backfilling pg states).

Make the parent Active state ignore DoRecovery so that if we are already
in some phase of recovery/backfill the event gets ignored.  It is already
handled by the other important substates that care, like Clean (for
repair's benefit).

I'm not sure why states like Activating are paying attention tot his vevent...

Fixes: http://tracker.ceph.com/issues/22145
Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-29 12:44:59 -06:00
Adam C. Emerson
d07588a1aa Merge pull request #18954 from adamemerson/wip-hole-in-the-bucket-dear-liza
rgw: Add try_refresh_bucket_info function
rgw: Add retry_raced_bucket_write
rgw: Handle stale bucket info in RGWPutMetadataBucket
rgw: Handle stale bucket info in RGWSetBucketVersioning
rgw: Handle stale bucket info in RGWSetBucketWebsite
rgw: Handle stale bucket info in RGWDeleteBucketWebsite
rgw: Handle stale bucket info in RGWPutBucketPolicy
rgw: Handle stale bucket info in RGWDeleteBucketPolicy
rgw: Expire entries in bucket info cache

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-11-29 13:17:12 -05:00
Jos Collin
61123c4070
Merge pull request #19215 from joscollin/wip-mount-fuse-ceph-typo
doc: Fix typo in mount.fuse.ceph

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-11-29 22:48:07 +05:30
Jos Collin
15c01a0932
Merge pull request #19209 from joscollin/wip-githubman-ktdreyer
githubmap: Add ktdreyer

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-11-29 22:45:23 +05:30
Kefu Chai
99c0306324
Merge pull request #19208 from tchaikov/wip-spdk-17.10
spdk: update SPDK to v17.10

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2017-11-30 00:39:18 +08:00
Josh Durgin
8f75008776
Merge pull request #19238 from tchaikov/wip-22266
tools/ceph_monstore_tool: rebuild initial mgrmap also

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: John Spray <jspray@redhat.com>
2017-11-29 08:19:40 -08:00
Matt Benjamin
134734ab74
Merge pull request #19079 from adamemerson/wip-static-ptr
Static Pointer
2017-11-29 11:17:22 -05:00
Jos Collin
3dfb57e0ed doc: Fix typo in mount.fuse.ceph
Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-11-29 21:43:46 +05:30
Patrick Donnelly
1ce4a41bb4
Merge PR #19212 into master
* refs/pull/19212/head:
	doc: Add section about MDS cache limits

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-11-29 07:20:19 -08:00
Andrew Schoen
eea4bb1367
Merge pull request #19213 from ceph/wip-bz1518264
ceph-volume: correctly fallback to bluestore when no objectstore is specified

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2017-11-29 09:05:56 -06:00
Orit Wasserman
1ad4b8df6b
Merge pull request #18267 from joscollin/wip-update-radosgw-admin-doc
doc: Update commands and options in radosgw-admin
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-11-29 15:08:47 +02:00
Kefu Chai
f63d1da4af tools/ceph_monstore_tool: rebuild initial mgrmap also
Fixes: http://tracker.ceph.com/issues/22266
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-29 20:22:07 +08:00
John Spray
967fc9cf8a
Merge pull request #18649 from b-ranto/wip-restful-fixes
mgr/restful: A couple of restful fixes

Reviewed-by: John Spray <john.spray@redhat.com>
2017-11-29 12:11:58 +00:00
Barbora Ančincová
4f590990b0 doc: Add section about MDS cache limits
Signed-off-by: Bara Ancincova (bara@redhat.com)
2017-11-29 12:46:49 +01:00
Jos Collin
76a8dc1e31 doc: Update all OPTIONS sections in radosgw-admin
Updated all OPTIONS sections in RADOSGW-ADMIN doc page, using the latest options from rgw_admin.cc::usage()

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-11-29 16:48:32 +05:30
Kefu Chai
b3ceafe7d8 cmake,rpm,deb: update to accommodate SPDK v17.10
* cmake/modules/BuildSPDK.cmake: add lvol
* cmake/modules/BuildDPDK.cmake: add pci and bus_pci
* ceph.spec.in, cmake/modules/BuildSPDK.cmake, debian/control:
  re-introduce libuuid dependency, as 17.07 added lvol. and the latter
  depends on uuid.
* cmake/modules/BuildSPDK.cmake: avoid introducing local variable of
  `iface_libs`.
* cmake/modules/patch-dpdk-conf.sh: disable
  CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES, this option introduces the
  balanced allocation of memory. but it also requires libnuma-dev.
  let's disable it for now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-29 18:03:23 +08:00
Kefu Chai
b1378b343a
Merge pull request #9571 from yehudasa/wip-rgw-redirect-zone
rgw: add a default redirect field for zones

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-11-29 16:38:54 +08:00
Kefu Chai
44842235bd
Merge pull request #19159 from shinobu-x/drop_using_namespace_std
common: drop namespace using directives for std

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-11-29 16:16:39 +08:00
Kefu Chai
bc020436ca
Merge pull request #19165 from xiaoxichen/fix_used_pct
mon/PGMap: Fix %USED calculation bug.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-29 16:14:40 +08:00
Kefu Chai
2eb0006334
Merge pull request #19156 from cyx1231st/fix_blkin_tracing
blkin: Fix unconditional tracing in OSD

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-29 16:13:34 +08:00
Kefu Chai
f05b093f02
Merge pull request #19149 from rzarzynski/wip-common-perfguards
common:  RAII-styled mechanism for updating PerfCounters

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-29 16:12:57 +08:00
Kefu Chai
bde50ca234
Merge pull request #19112 from xiaoxichen/osd_mapped_pool
OSD/admin_socket: add get_mapped_pools command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-29 16:12:02 +08:00
Kefu Chai
f39bff5e91
Merge pull request #18959 from cooboos/wip-cephdencoder-LCtype
ceph-dencoder: add support for RGWLifecycleConfiguration

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-29 16:10:46 +08:00
Kefu Chai
34ffafd7ca
Merge pull request #19174 from smithfarm/wip-cmake-exclude-gitignore
cmake: mgr: exclude .gitignore

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-29 15:39:14 +08:00
Kefu Chai
12dc5733c9
Merge pull request #18938 from tchaikov/wip-cmake
cmake: check gcc version not release date for libstdc++ saneness

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-29 15:38:16 +08:00
Patrick Donnelly
38f051c22a
Merge PR #19206 into master
* refs/pull/19206/head:
	doc: add pre-requisite to enable CephFS snapshot

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-11-28 12:02:52 -08:00
Adam C. Emerson
11f54d730d common: Add static_ptr
Add static_ptr, a pointer-like class that contains its own storage,
avoiding use of the heap.

The full range of *_pointer_cast functions are included.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-11-28 14:29:04 -05:00
Kefu Chai
6a9e622cf9
Merge pull request #19199 from tchaikov/wip-22136
qa/ceph-disk: enlarge the simulated SCSI disk

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2017-11-28 23:46:49 +08:00
Alfredo Deza
f0ac2dfdbc ceph-volume lvm.activate correct detection and fallback of objectstore flags
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-28 09:22:19 -05:00
Alfredo Deza
8d22856d6c ceph-volume tests.devices.lvm verify objectstore flags in activate
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-11-28 09:21:34 -05:00
Jason Dillaman
2241b9da12
Merge pull request #19163 from vshankar/rbd-image-map-mem-leak
rbd-mirror: ImageMap memory leak fixes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-11-28 09:03:12 -05:00
Jos Collin
158c0f1a1b .githubmap: Add ktdreyer
Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-11-28 16:44:11 +05:30
Jos Collin
6863b4914e
Merge pull request #19172 from joscollin/wip-mount-fuse-ceph-doc
doc: doc for mount.fuse.ceph

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-11-28 16:38:13 +05:30
Jos Collin
6c39818eaf doc: doc for mount.fuse.ceph
Created doc for mount.fuse.ceph.

Fixes: http://tracker.ceph.com/issues/21539
Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-11-28 15:56:09 +05:30
Kefu Chai
c05e842009 spdk: update SPDK to v17.10
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-28 18:06:34 +08:00
Yehuda Sadeh
5f84a72989 test_multi: add a test for zone redirect
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-11-28 01:27:43 -08:00
Yehuda Sadeh
d94e9cc973 rgw-admin: add option to configure redirect_zone
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-11-28 01:27:43 -08:00