Asynchronous Block-Level Storage Replication
Go to file
Thomas Schoebel-Theuer 8e2de8288d light: fix missing versionlink upon slow or defective IO
Some primary appeared to have died, and was rebooted.
In the meantime, the old secondary was forcefully switched
to primary.

Afterwards, the old primary = new secondary got stuck because 2
versionlinks, which had been _produced_ by _himself_, were
missing, but they were present at the new primary = old secondary!

How could this happen?

All transaction logfiles were fully present and correct everywhere.

However, the old primary kern.log showed that a problem with the
RAID system must have existed. In addition, the RAID controller
errorlog also reported some problems which appeared to have healed.

Problem analysis shows the following possibility:

The transaction logger can continue to write data, even via
fsync(), while the _writeback_ of other parts of the /mars filesystem
(e.g. symlink updates) got stuck for a long time due to an IO problem.

Usually, slow or even missing symlink updates are no problem because
upon recovery after a reboot, everything is healed by transaction
replay (possibly replaying much more data than really necessary,
but this does not affect semantics, and it is even advantageous
when RAID disks might contain defective data).

There is one exception: after a logrotate, the corresponding new
versionlink should appear after a small time. Otherwise, the
above mentioned scenario could emerge.

We use sync_filesystem() to ensure that any versionlink update
to a _new_ versionlink is either guaranteed to become persistent,
or (in case of IO problems) the mars_light thread will hang, which
will be (hopefully) noticed soon by monitoring.
2016-02-03 22:01:48 +01:00
contrib contrib: speedup mars_check.sh 2015-10-07 10:42:18 +02:00
docu doc: update version 2016-01-21 08:10:26 +01:00
kernel light: fix missing versionlink upon slow or defective IO 2016-02-03 22:01:48 +01:00
pre-patches infra: add some pre-patches up to kernel 3.15 2014-06-18 12:10:55 +02:00
scripts infra: fix transitive option propagation in gen_config.pl 2014-08-07 15:26:02 +02:00
test_suite all: distinguish contrib from maintained code 2014-06-18 12:10:54 +02:00
userspace marsadm: fix edge cases of try_to_avoid_splitbrain() 2016-02-03 22:00:47 +01:00
.gitattributes
.gitignore test_suite: current state 2013-12-12 10:59:01 +01:00
AUTHORS all: clarify license GPLv2+ 2014-11-25 18:09:17 +01:00
ChangeLog all: release light0.1stable23 2016-01-21 08:11:24 +01:00
COPYING
INSTALL all: distinguish contrib from maintained code 2014-06-18 12:10:54 +02:00
NEWS doc: README points to http://schoebel.github.io/mars/ 2013-11-18 13:41:50 +01:00
README all: clarify license GPLv2+ 2014-11-25 18:09:17 +01:00

MARS Long Distance Replication Software

See http://schoebel.github.io/mars/

Very detailed documentation / motivation / use cases can be found there.
Please look into the docu/ subdirectory.

License: GPLv2+ (see file COPYING)
Documentation is under FDL.

This is intended for the whole project, and is valid for all parts
from the original author as well as parts from 1&1.

Any further contributors to contrib/ subdirectory are strongly encouraged
to put their files also under GPL or FDL, or at least a compatible
license (see newest descriptions at fsf.org).