Commit Graph

17739 Commits

Author SHA1 Message Date
Sage Weil
51e402e3c2 rados: fix load-gen 'max-ops'
This was mixed up with min/max_op_len.  And max_ops wasn't being used
the initial object creation stage, flooding the OSDs.  Or during run().

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-16 10:40:33 -08:00
Sage Weil
706b6910bb osd: recover_primary_got() -> recover_got()
This is called on primary and replicas alike.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-16 09:46:10 -08:00
Sage Weil
a4e2395fc9 osd: clear missing set on replica when restarting backfill
The primary does the same in PG::activate().

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-16 09:34:47 -08:00
Sage Weil
6b02f9fae1 osd: rev osd internal cluster protocol
Prevent backfill code from talking to pre-backfill code.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-13 17:13:34 -08:00
Samuel Just
0ded7e4dac ReplicatedPG: munge truncate_seq 1/truncate_size -1 to seq 0/size 0
Truncate with seq 1 and size -1 is a noop.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
Reviewed-by: Sage Weil <sage@newdream.net>
2012-01-13 12:34:01 -08:00
Sage Weil
224a65a776 Merge remote branch 'gh/master' into wip-backfill 2012-01-13 08:35:47 -08:00
Samuel Just
4815cafddf ReplicatedPG: Update stat accounting for truncate during write
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
Reviewed-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-01-12 17:47:10 -08:00
Yehuda Sadeh
8ceb388396 rgw: wrap cls_cxx_map_* with try/catch around decoding 2012-01-12 16:39:30 -08:00
Yehuda Sadeh
05d8ecbe1c rgw: bucket index creation and init in a single operation 2012-01-12 16:23:00 -08:00
Yehuda Sadeh
876d829af7 librados: add ObjectOperation::exec 2012-01-12 16:23:00 -08:00
Sage Weil
dc628a5b4e secret: move null check before strlen(key_name) deref
Coverity cid: 98
Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-12 16:14:52 -08:00
Yehuda Sadeh
d41ddcdf79 osd: stat op, don't compare in memory state to object
might be that object is being created by the current compound request.
2012-01-12 16:10:02 -08:00
Sage Weil
f42c658d22 osd: fill in empty item in peer_missing for strays
If we search_for_missing() on a host, make a corresponding entry in our
peer_missing map (if it isn't already there).  This ensure we get (empty)
entries for strays, which makes all_unfound_are_queried_or_lost() happy.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-12 15:09:18 -08:00
Yehuda Sadeh
10b00316b7 rgw: don't crash when copying a zero sized object 2012-01-12 15:02:09 -08:00
Samuel Just
845aa534e3 ReplicatedPG: Do a write even for 0 length operation
Otherwise, a 0 length write to an offset past the end of the file will
cause the internal accounting to reflect the full size of the file, but
not the file on disk.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-01-12 13:21:19 -08:00
Samuel Just
80f57f9656 ReplicatedPG: fix stat accounting error in CEPH_OSD_OP_WRITEFULL
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-01-12 13:21:19 -08:00
Sage Weil
cfa39bfb23 qa/client/gen-1774.sh
Capture Alexandre's script for reproducing #1774 here for posterity, until
we write a properly harnessed test for this.  Currently, workunits can't
mount/unmount, and we don't have a way to make ceph-fuse drop it's cache.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-12 12:59:07 -08:00
Sage Weil
6cf77532de osd: fix PG::Log::copy_up_to() tail
The tail needs to refer to the entry preceeding the first entry in the
log.  This updates copy_up_to() to match the basic structure of the other
copy_*() methods.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-12 11:46:27 -08:00
Sage Weil
805513be21 osd: reset last_complete on backfill restart
Since last_backfill is hobject_t(), we can set this equal to last_update.
This fixes a problem where last_complete preceeds the abbreviated log we
send to the replica below.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-12 11:07:02 -08:00
Andrey Stepachev
1e56367e5e client: avoid taking inode ref in case of nonexistent dir
Signed-off-by: Andrey Stepachev <octo@yandex-team.ru>
Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-12 10:38:33 -08:00
Sage Weil
cedd92bea6 Merge branch 'wip-makefile' 2012-01-12 10:35:03 -08:00
Sage Weil
71131371de COPYING: note licenses for all files, not just the default
This (mostly) copies debian/copyright for now, but there are format
restrictions for that file.  Suggestions for a cleaner way to handle this
are welcome.  In the meantime, this is better...

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-01-12 10:03:27 -08:00
Sage Weil
54e0dfc17f debian/copyright: note acx_pthread.m4 license
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-01-12 10:03:24 -08:00
Kacper Kowalik (Xarthisius)
6b55b6b336 Makefile: Add headers that were omitted in make dist and prevented tests from building
Signed-off-by: Kacper Kowalik (Xarthisius) <xarthisius@gentoo.org>
2012-01-12 09:17:56 -08:00
Kacper Kowalik (Xarthisius)
c9e028f4e5 Makefile: Handle corner case of crypto++ correctly
i.e. use c++ while compiling, append to CRYPTO_LIBS instead of LIBS

Signed-off-by: Kacper Kowalik (Xarthisius) <xarthisius@gentoo.org>
2012-01-12 09:17:42 -08:00
Kacper Kowalik (Xarthisius)
c5144eed4e Makefile: Use ACX_PTHREAD in configure.ac and resulting flags in src/Makefile.am
instead of hardcoded flags

Signed-off-by: Kacper Kowalik (Xarthisius) <xarthisius@gentoo.org>
2012-01-12 09:17:24 -08:00
Kacper Kowalik (Xarthisius)
7bf01b1161 Makefile: Add recent acx_pthread.m4 that has a fix for nostdlib issue.
See http://code.google.com/p/protobuf/issues/detail?id=188 for details

Signed-off-by: Kacper Kowalik (Xarthisius) <xarthisius@gentoo.org>
2012-01-12 09:17:06 -08:00
Samuel Just
b93bf285c9 PG: gen_prefix should grab a map reference atomically
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-01-11 13:50:24 -08:00
Yehuda Sadeh
38b9b50307 rgw-admin: add pool rm and pools list 2012-01-11 13:37:37 -08:00
Yehuda Sadeh
e2c02543c7 rgw-admin: clean up unused commands 2012-01-11 13:05:47 -08:00
Sage Weil
ac1e105e88 osd: bound log we send when restarting backfill
Use the new tunable from b1da5115aa.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-11 13:04:11 -08:00
Josh Durgin
79085ad088 rados.py: avoid getting return value of void function
rados_ioctx_locator_set_key is void. The return value seems to have
been uninitialized, so the tests failed rarely.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-01-11 12:22:14 -08:00
Josh Durgin
85552cf84f pg: remove unnecessary guard from calc_trim_to()
The num_objects check doesn't make sense, and could only make trimming
happen more often than it should. Sage did not remember why it was
added.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-01-11 11:13:47 -08:00
Josh Durgin
b1da5115aa pg: add a configurable lower bound on log size
This helps prevent problems with retrying requests being detected as
duplicates. The problem occurs when the log is trimmed too
aggressively, and an earlier tid is removed from the log, while a
later one is not. The later request will be detected as a duplicate
and responded to immediately, possibly violating the ordering of the
requests.

Partially fixes #1490.
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-01-11 11:13:36 -08:00
Sage Weil
8a9dbc4762 Merge remote branch 'gh/master' into wip-backfill 2012-01-11 10:34:35 -08:00
Sage Weil
734737f339 osd: limit size of log sent to reset backfill targets
Need to replace magic number with new tunable, once that is merged.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-11 06:41:13 -08:00
Alexandre Oliva
8f9549f0e8 client: start caching readdir results after readdir_start
Use upper_bound rather than lower_bound to compute the initial pd within
insert_trace, so that we don't attempt to remove it if it happens to be
in the same frag as the new reply.

Fixes: #1774
Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-01-10 20:25:32 -08:00
Sage Weil
5d9896084e monclient: fix resolve_addrs() call
This was broken in def36668a1 it looks like.
Passing uninitialized memory to resolve_addrs(), and needlessly
allocating a buffer.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-10 16:39:23 -08:00
Sage Weil
f09b21eff6 resolve_addrs: return ipv4 and ipv6 addrs
Fixes: #1891
Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-10 16:35:40 -08:00
Samuel Just
9e9b5c6f15 ReplicatedPG: fix typo in stats accounting in _rollback_to
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-01-10 16:22:30 -08:00
Sage Weil
d4815e5bd4 osd: send log with backfill restart
This makes backfill restart less of a special case: we send an info AND
log, just like we do normally.  Code paths are more similar than before.

The main change here is that the backfill target gets a pg log with recent
history, which allows it to more reliably detect dup operations.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-10 16:14:13 -08:00
Sage Weil
c7d92d1d3f osd: fail to peer if interval lacks any !incomplete replicas
We need at least one non-incomplete replica during a rw interval in order
to peer.  The backfilling/incomplete replicas get log entries, but not
all object writes, so they are (mostly) excluded from the peering process
(find_best_info(), in particular).

We can't do this during the PriorSet calculation because we don't have
their PG::Info yet.  But, once we get it, we need to make sure at least one
of the replicas during the last rw interval is not incomplete, or else we
should mark the pg DOWN (just like the PriorSet calculation does).

This logic mostly mirrors that of PriorSet, but additionally requires
the replicas be !incomplete.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-10 13:23:00 -08:00
Greg Farnum
3b81fa5845 mon: allow specifying pg_num and pgp_num when creating new pools.
Right now this is only exposed via the monitor command interface:
osd pool create <poolname> [pg_num [pgp_num]]
but it can be expanded to other interfaces as appropriate.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-01-10 11:26:45 -08:00
Tommi Virtanen
69aface0cf auth: Fix Doxygen warnings.
Match prototype and implementation argument names and types
(textually, that is use std:: prefix).

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
2012-01-10 11:20:56 -08:00
Tommi Virtanen
741f6d5448 Fix several doxygen warnings, to minimize noise. Only changes comments.
Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
2012-01-10 11:20:56 -08:00
Tommi Virtanen
b891652877 librados: Make API docs use @note instead of @bug for now.
Asphyxiate doesn't yet support all of the Doxygen markup.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
2012-01-10 11:20:56 -08:00
Samuel Just
1f46e7c915 FileStore: assert on ENOSPC even for SETXATTR
Otherwise we can get corrupt object attributes on ext*.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-01-10 11:17:42 -08:00
Greg Farnum
ac9b2d092f mds: initiate monitor reconnect if beacon acks take too long
If it takes 2*mds_beacon_grace (default 30 seconds total) seconds
to get an ack back, maybe it's the monitor and not us. Try a reconnect,
which will just add the teensiest bit of load if we're wrong.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-01-10 10:47:22 -08:00
Greg Farnum
45897b52af mds: remove beacon_killer code.
This no longer does *anything* except print out
useless warning messages.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-01-10 10:47:22 -08:00
Sage Weil
ba00f95ad6 osd: make less noise when filestore is already up to date
Signed-off-by: Sage Weil <sage@newdream.net>
2012-01-10 09:49:41 -08:00