Commit Graph

36168 Commits

Author SHA1 Message Date
Sage Weil
027f74f998 Merge pull request #2637 from dachary/wip-9245-test-mon-pending-giant
mon: obsolete osdmonitor_prepare_command (giant)
2014-10-03 11:24:44 -07:00
Loic Dachary
73d5bdb987 tests: remove tests requiring osdmonitor_prepare_command
Although they have been useful, their maintenance and fragility caused
more trouble than their benefits.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 038de0031b)
2014-10-03 19:00:34 +02:00
Loic Dachary
e8657814a6 mon: obsolete osdmonitor_prepare_command
And the mon_advanced_debug_mode option that is only used in the context
of osdmonitor_prepare_command.

http://tracker.ceph.com/issues/9245 Fixes: #9245

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 2abc8f400f)
2014-10-03 19:00:16 +02:00
Loic Dachary
d0c2d7d572 test: minor case fix
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 6f69837456)
2014-10-03 19:00:00 +02:00
Sage Weil
d0ceb3ab0a os/FileStore: do not loop in sync_entry on shutdown
Normally we repeat a sync if the journal is full and asks for it or if we
have pending sync waiters.  This isn't for correctness: it's to move things
along quickly.

If we are shutting down *and* the journal is full, however, we currently
can get stuck in a loop where we repeatedly sync.  However, the journal
never gets unfull because we aren't expiring anything.

We tried to fix this in 682b9daacb but that
didn't work because we don't tell the journal to stop before we shut down
the filestore sync thread.

I'm pretty sure there is another issue in the journal code as

 full_state == FULL_FULL
 submit_manager.op_seq = 14761
 apply_manager.committing_seq = 14761
 apply_manager.committed_seq = 14761

which seems odd.  But I have no longs.  Meanwhile, this fixes the hang.

Fixes: #9545
Signed-off-by: Sage Weil <sage@redhat.com>
2014-10-03 08:26:11 -07:00
Joao Eduardo Luis
7e2bd2294f mon: Monitor: let 'handle_command()' deal with caps validation
If a given client doesn't have the required caps when running a command,
it must receive an EACCES or EPERM reply.  This is already handled by
Monitor::handle_command(), which does an exceptionally good job at it.

Therefore, and unlike other messages that do not expect return values,
we can't simply drop the message if the client doesn't have the
appropriate capabilities, or things can get very weird very fast from
the user's perspective.  Dropping the message for a command without a
reply has roughly the same effect as loss of quorum (timeout, pipes
failing) and confusion may ensue from it.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:20 +01:00
Joao Eduardo Luis
f0653c0401 qa/workunits: mon: auth_caps: account for mon blank caps
test creating and entity with blank caps with and without '--force'
being specified.  without '--force' they must fail with EINVAL as the
monitor will not be able to parse them.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:20 +01:00
Joao Eduardo Luis
87d06f11c4 mon: MonCommands: adjust indentation for 'auth add'
Eye-candy.  We changed indentation of a few other entries and this one
was just too darn obvious, itching all over, night terrors, the whole
nine yards.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:20 +01:00
Joao Eduardo Luis
c7d5c25324 qa/workunits: mon: auth_caps: variables must be local
We have variables with the same name that are being shared!  We don't
hit any issues with it currently because the code just kind of works
even though that happens.  Add a bit of new logic that relies on an
immutable return code (for instance) and we're in the woods.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
7e85bebd74 qa/workunits: mon: auth_caps: cleanup on successful exit
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
5488f464c5 test/vstart_wrapper.sh: use cephx
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
74695617c3 mon: Monitor: remove redudant call to osdmon's dispatch
Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
c0e3bc9a30 mon: Monitor: check caps and source before dispatching messages
Only dispatch messages that a client may send if said client has at
least MON_CAP_R, and only dispatch internal monitor messages if peer is
a monitor.

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
3d78285dfa mon: Monitor: create logical divisions on dispatch() based on msg nature
Instead of a single switch(), have multiple switch() and order them by
increasing necessity of privileges.

This patch thus divides the big switch into:

- messages not requiring auth/caps checks at all
- messages which caps shall be checked somewhere else
- messages the Monitor class needs to deal with but only require a
  client to have enough caps for the monitor to consider handling them
- messages that only a monitor is allowed to send.

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
d0113e07d8 test: mon: test-mon-msg: Check which msgs the mons handle and reply to
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
ea96863d2a qa/workunits: cephtool: test auth profiles
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
d6b702ce46 mon: MonCap: add new profiles
Adds three new profiles:

  read-only:  able to issue all read-only (MON_CAP_R) commands.  Any
  command that may take additional caps (MON_CAP_W or MON_CAP_X) won't
  be allowed.

  read-write: able to issue all read-write (MON_CAP_R | MON_CAP_W)
  commands.  Commands that require MON_CAP_X will not be allowed.

  role-definer: solely able to issue commands on the 'auth' subsystem,
  to which all caps are given (MON_CAP_R | MON_CAP_W | MON_CAP_X).

Fixes: #8899

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
940c4e2812 mon: Monitor: match command module caps against what's on MonCommands.h
We were checking the command's permissions against what we perceived as
the 'module' from parsing the command's "prefix" (specified by the
client).  This caused troubles with cap checks for commands without a
submodule clearly defined, such as 'status' or 'health' (vs 'mon dump'
or 'osd pool set', which are of submodule 'mon' and 'osd' respectively).

As such, we now grab the command's submodule (right now solely for caps
checks) from the monitor's internal representation of the commands
(defined in mon/MonCommands.h and built at compile time and stashed in
'mon_commands').  Given that commands such as 'health', 'fsid' or
'status' have properly defined modules in MonCommands.h, we simply rely
on that representation for all commands.  Which is what we should have
been doing from the start anyway, because we shouldn't be relying on the
client to point us to what we want to authenticate against.

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-03 16:24:19 +01:00
Joao Eduardo Luis
861246158e mon: AuthMonitor: validate caps when creating or changing mon caps
The monitor doesn't really know how to validate caps not meant for it.
The MDS or the OSD may very well allow blank caps for instance, while
the monitor categorically does not.  We can't simply state a capability
is invalid because we wouldn't take it as such.

On the other hand, we must check monitor caps and make sure they are
correct, otherwise malformed caps can go unnoticed for a while,
sometimes even being hard to understand what may have gone wrong.

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-10-03 16:24:10 +01:00
Loic Dachary
eb2f0f497f ceph-disk: bootstrap-osd keyring ignores --statedir
The STATEDIR variable is used to initialize the bootstrap-osd keyring
before it gets a chance to be overriden by --statedir. Replace it with
{statedir} so that it can be substituted after all options have been
parsed.

http://tracker.ceph.com/issues/9653 Fixes: #9653

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit fa0bd06b46)
2014-10-03 15:53:28 +02:00
Guang Yang
19be358322 PG::actingset should be used when checking the number of acting OSDs for a given PG.
Signed-off-by: Guang Yang <yguang@yahoo-inc.com>
2014-10-03 01:47:28 +00:00
Sage Weil
8253ead174 osdc/Objecter: use SafeTimer; make callbacks race-tolerant
The RWTimer event cancellation is racy.  Instead, just make all of our
callbacks tolerate cancellation races.  This is already true of most of
them (in fact, they are probably broken because they try to take a write
lock while holding a read lock).  Fix C_CancelOp so that it calls the
other op_cancel (that takes a tid).

Then switch the RWTimer back to a SafeTimer.  Put it in unsafe callbacks
mode because we don't want to introduce lock cycles with timer_lock.

Fixes: #9582
See also: #9650
Signed-off-by: Sage Weil <sage@redhat.com>
2014-10-02 15:22:56 -07:00
Samuel Just
a1aa06b7fb ReplicatedPG: dump snap_trimq on pg query
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-10-01 14:31:19 -07:00
Samuel Just
34f38b68d8 ReplicatedPG: do not queue the snap trimmer constantly
Previously, we continuously requeued the snap trimmer while in
TrimmingObjects.  This is not a good idea now that we try to
limit the number of snap trimming repops in flight and requeue
the snap trimmer directly as those repops complete.

Fixes: #9113
Backport: giant, dumpling, firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-10-01 14:31:18 -07:00
Samuel Just
b29bf00f68 ReplicatedPG: clean out completed trimmed objects as we go
Also, explicitely maintain a max number of concurrently trimming
objects.

Fixes: 9113
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-10-01 14:31:16 -07:00
Samuel Just
b1ca1f23ff Merge pull request #2525 from ceph/wip-9487
RFC: initialize purged_snaps on backfill

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-01 13:45:27 -07:00
Sage Weil
7aacccd0bb Merge pull request #2620 from ceph/revert-2604-wip-9113
Revert "ReplicatedPG: clean out completed trimmed objects as we go"
2014-10-01 13:23:21 -07:00
Samuel Just
0f731ae580 Revert "ReplicatedPG: clean out completed trimmed objects as we go" 2014-10-01 13:18:46 -07:00
Sage Weil
da9ae5c92e ceph.spec: fix typo
Signed-off-by: Sage Weil <sage@redhat.com>
2014-10-01 12:33:38 -07:00
Sage Weil
b1d5dc6bc4 Merge pull request #2614 from ceph/wip-rpm-epoch
rpm: set epoch 1 to match fedora

Reviewed-by: Boris Ranto <branto@redhat.com>
2014-10-01 12:23:44 -07:00
Yehuda Sadeh
f26ba0461e Merge branch 'wip-9013' into giant 2014-10-01 10:28:38 -07:00
Yehuda Sadeh
63d0ec7b2c rgw: add civetweb as a default frontend
Fixes: #9013
Originally the configurable was empty, now setting explicitly both
fastcgi and civetweb (on port 7480).

Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-10-01 10:27:51 -07:00
Sage Weil
8388836208 ceph.spec.: add epoch
This is done in fedora packaging.  Do it here too so that you can move
between upstream packages (from ceph.com) and fedora and other derivatives
will builds.

Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
2014-10-01 06:02:02 -07:00
Sage Weil
17544a460a Merge pull request #2615 from ceph/wip-fsls-json
mon: fix JSON `fs ls` output

Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-01 05:54:45 -07:00
John Spray
a470c96e6b mon: fix JSON fs ls output
A missing list terminator was causing unparseable output.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-10-01 12:09:16 +01:00
Josh Durgin
fd5af13949 Merge pull request #2564 from ceph/wip-8587
rgw: subuser creation fixes

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-09-30 17:24:02 -07:00
Sage Weil
cefb1a35ca mon: wait for paxos writes before touching state
Fix two other paths where may change the mon state so that we wait for the
pending write first.  start_election() in particular can be triggered at
almost any time if we see an election message from another mon.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-30 17:19:37 -07:00
Sage Weil
46c1d93653 mon: flush paxos write before setting shutdown state
Paxos::commit_finish() makes assertions about our state (writing |
writing-previous).

Fixes: #9635
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-30 17:19:35 -07:00
Sage Weil
4072ef782e mon: move paxos write wait into a helper
Avoid duplicated code.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-30 17:15:25 -07:00
Sage Weil
544d8255c9 Merge pull request #2613 from ceph/wip-8822
osd: swap state spinlock for atomic_t

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-09-30 14:45:27 -07:00
Samuel Just
624aaf2a4e PG: release backfill reservations if a backfill peer rejects
Also, the full peer will wait until the rejection from the primary
to do a state transition.

Fixes: #9626
Backport: giant, firefly, dumpling
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-09-30 14:15:34 -07:00
Sage Weil
62e2bca8d8 osd: swap state spinlock for atomic_t
We are hitting a strange issue with valgrind and pthread spinlocks. Avoid
the issue by using an atomic_t here (which is simpler anyway).

Avoids: #8822
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-30 11:56:30 -07:00
Gregory Farnum
e30969b19e Merge pull request #2612 from ceph/wip-9562-backport
osdc/Filer: drop probe/purge locks before calling objecter

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-09-30 10:49:05 -07:00
John Spray
a8ac4b62a5 osdc/Filer: drop probe/purge locks before calling objecter
Fixes: #9562

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8dc94a2d8c)
2014-09-30 18:46:25 +01:00
Greg Farnum
0ea20a668c Locker: accept ctime updates from clients without dirty write caps
The ctime changes any time the inode does. That can happen even without
the file itself having changed, so we'd better accept the update whenever
the auth caps have dirtied, without worrying about the file caps!

Fixes: #9514
Backport: firefly

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2014-09-30 10:24:51 -07:00
Loic Dachary
9c4616dcd2 Merge pull request #2556 from dachary/wip-9572-erasure-code-blaumroth
erasure-code: restore jerasure BlaumRoth default w

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-30 01:48:26 +02:00
Sage Weil
5281ac7aa9 Merge pull request #2601 from ceph/wip-python-flask
debian: python-flask is needed by ceph, not python-ceph

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-09-29 16:24:50 -07:00
Sage Weil
fe3434f41c debian: move ceph_rest_api.py into ceph
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-29 14:28:32 -07:00
Sage Weil
8cda623e0b ceph.spec.in: move ceph_rest_api.py into ceph
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-29 14:24:01 -07:00
Sage Weil
0d1a4171c1 Merge pull request #2602 from athanatos/wip-9574
PG: check full ratio again post-reservation

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-29 14:14:40 -07:00