mirror of https://github.com/schoebel/mars
user-manual: rework cluster setup
This commit is contained in:
parent
4ec20fdbd7
commit
eb709782f0
|
@ -2273,7 +2273,7 @@ name "sec:Setup-Primary-and"
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
If you already have some production data on your severs on LVM, you may
|
If you already have some production data on your servers via LVM, you may
|
||||||
skip some of the following subsections.
|
skip some of the following subsections.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -2306,6 +2306,21 @@ external
|
||||||
\emph default
|
\emph default
|
||||||
DRBD metadata (which is not touched by MARS).
|
DRBD metadata (which is not touched by MARS).
|
||||||
Internal DRBD metadata is reported to also work, because it resides at
|
Internal DRBD metadata is reported to also work, because it resides at
|
||||||
|
the end of the block device.
|
||||||
|
However, you will waste some small amount of storage.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Migrating back to DRBD is also possible, provided you re-initialize the
|
||||||
|
DRBD meta-data again.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
For the following instructions to work, you must be
|
||||||
|
\family typewriter
|
||||||
|
root
|
||||||
|
\family default
|
||||||
|
on your servers.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
|
@ -2319,6 +2334,11 @@ name "subsec:Setup-Hardware"
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
When using hardware RAID controllers, you will need to build your RAID sets
|
||||||
|
with the corresponding tools.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
\noindent
|
\noindent
|
||||||
\begin_inset Graphics
|
\begin_inset Graphics
|
||||||
|
@ -2352,6 +2372,19 @@ writeback
|
||||||
goldcaps for saving the cache content into some flash chips.
|
goldcaps for saving the cache content into some flash chips.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
In the following sections, we assume that two RAID sets are already built,
|
||||||
|
and are accessible as
|
||||||
|
\family typewriter
|
||||||
|
/dev/sdb
|
||||||
|
\family default
|
||||||
|
and
|
||||||
|
\family typewriter
|
||||||
|
/dev/sdc
|
||||||
|
\family default
|
||||||
|
.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
Setup LVM
|
Setup LVM
|
||||||
\begin_inset CommandInset label
|
\begin_inset CommandInset label
|
||||||
|
@ -2363,6 +2396,96 @@ name "subsec:Setup-LVM"
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\noindent
|
||||||
|
\begin_inset Graphics
|
||||||
|
filename images/MatieresCorrosives.png
|
||||||
|
lyxscale 50
|
||||||
|
scale 17
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
Execute the following instructions only once after hardware deployment,
|
||||||
|
or if you want to re-install your server.
|
||||||
|
Otherwise, you may delete existing data.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
First step is create the LVM meta-information on the RAID sets
|
||||||
|
\family typewriter
|
||||||
|
/dev/sdc
|
||||||
|
\family default
|
||||||
|
and
|
||||||
|
\family typewriter
|
||||||
|
/dev/sdc
|
||||||
|
\family default
|
||||||
|
:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
pvcreate /dev/sdb
|
||||||
|
\family default
|
||||||
|
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
pvcreate /dev/sdc
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Check your physical volumes:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
pvs
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Create a volume group:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
vgcreate vg /dev/sdb /dev/sdc
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Check your volume group:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
vgs
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Create a LV for /mars:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
lvcreate -i 2 -L 100G -n mars lv
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Check your list of LVs:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
lvs
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
Setup Cluster Nodes
|
Setup Cluster Nodes
|
||||||
\begin_inset CommandInset label
|
\begin_inset CommandInset label
|
||||||
|
@ -2376,12 +2499,12 @@ name "subsec:Setup-your-Cluster"
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
For your cluster, you need at least two nodes.
|
For your cluster, you need at least two nodes.
|
||||||
In the following, they will be called A and B.
|
In the following, they will be called hostA and hostB.
|
||||||
In the beginning, A will have the
|
In the beginning, hostA will have the
|
||||||
\family typewriter
|
\family typewriter
|
||||||
primary
|
primary
|
||||||
\family default
|
\family default
|
||||||
role, while B will be your initial
|
role, while hostB will be your initial
|
||||||
\family typewriter
|
\family typewriter
|
||||||
secondary
|
secondary
|
||||||
\family default
|
\family default
|
||||||
|
@ -2390,23 +2513,21 @@ secondary
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
You must be
|
On each of hostA and hostB, create the
|
||||||
\family typewriter
|
|
||||||
root
|
|
||||||
\family default
|
|
||||||
.
|
|
||||||
\end_layout
|
|
||||||
|
|
||||||
\begin_layout Enumerate
|
|
||||||
On each of A and B, create the
|
|
||||||
\family typewriter
|
\family typewriter
|
||||||
/mars/
|
/mars/
|
||||||
\family default
|
\family default
|
||||||
mountpoint.
|
mountpoint:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
mkdir /mars
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
On each node, create an
|
On each host, create an
|
||||||
\family typewriter
|
\family typewriter
|
||||||
ext4
|
ext4
|
||||||
\family default
|
\family default
|
||||||
|
@ -2435,10 +2556,16 @@ xfs
|
||||||
|
|
||||||
\family default
|
\family default
|
||||||
.
|
.
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
mkfs.ext4 /dev/vg/mars
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
On each node, mount that filesystem to
|
On each host, mount that filesystem to
|
||||||
\family typewriter
|
\family typewriter
|
||||||
/mars/
|
/mars/
|
||||||
\family default
|
\family default
|
||||||
|
@ -2447,7 +2574,18 @@ On each node, mount that filesystem to
|
||||||
\family typewriter
|
\family typewriter
|
||||||
/etc/fstab
|
/etc/fstab
|
||||||
\family default
|
\family default
|
||||||
|
., or to create a systemd unit
|
||||||
|
\family typewriter
|
||||||
|
mars.mount
|
||||||
|
\family default
|
||||||
.
|
.
|
||||||
|
Here we just mount it by hand:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
mount /dev/vg/mars /mars
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
|
@ -2468,11 +2606,15 @@ marsadm
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
On node A, say
|
On hostA:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\family typewriter
|
\family typewriter
|
||||||
marsadm create-cluster
|
marsadm create-cluster
|
||||||
\family default
|
\family default
|
||||||
.
|
|
||||||
\begin_inset Newline newline
|
\begin_inset Newline newline
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
@ -2481,7 +2623,7 @@ This must be done
|
||||||
exactly once
|
exactly once
|
||||||
\emph default
|
\emph default
|
||||||
, on exactly one node of your cluster.
|
, on exactly one node of your cluster.
|
||||||
Never do this twice or on different nodes, because that would create two
|
Never do this twice or on different hosts, because that would create two
|
||||||
different clusters which would have nothing to do with each other.
|
different clusters which would have nothing to do with each other.
|
||||||
The
|
The
|
||||||
\family typewriter
|
\family typewriter
|
||||||
|
@ -2493,29 +2635,44 @@ marsadm
|
||||||
\family typewriter
|
\family typewriter
|
||||||
/mars/
|
/mars/
|
||||||
\family default
|
\family default
|
||||||
partition and start over with step 3 at that node.
|
partition and start over with step 2.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
On node B, you must have a working
|
Current versions of MARS need a working
|
||||||
\family typewriter
|
\family typewriter
|
||||||
ssh
|
ssh
|
||||||
\family default
|
\family default
|
||||||
connection to node A (as
|
connection from hostB to hostA (as
|
||||||
\family typewriter
|
\family typewriter
|
||||||
root
|
root
|
||||||
\family default
|
\family default
|
||||||
).
|
).
|
||||||
Test it by saying
|
Test it on hostB:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\family typewriter
|
\family typewriter
|
||||||
ssh A w
|
ssh hostA w
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\family default
|
\family default
|
||||||
on node B.
|
This should work without entering a password.
|
||||||
It should work without entering a password (otherwise, use
|
Hint: you may use
|
||||||
\family typewriter
|
\family typewriter
|
||||||
ssh-agent
|
ssh-agent
|
||||||
\family default
|
\family default
|
||||||
to achieve that).
|
and
|
||||||
|
\family typewriter
|
||||||
|
ssh -A
|
||||||
|
\family default
|
||||||
|
for achieving that.
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
In addition,
|
In addition,
|
||||||
\family typewriter
|
\family typewriter
|
||||||
rsync
|
rsync
|
||||||
|
@ -2524,8 +2681,12 @@ rsync
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Enumerate
|
\begin_layout Enumerate
|
||||||
On node B, say
|
On hostB
|
||||||
\family typewriter
|
\family typewriter
|
||||||
|
:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
marsadm join-cluster A
|
marsadm join-cluster A
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -2551,18 +2712,88 @@ marsadm create-cluster
|
||||||
*-cluster
|
*-cluster
|
||||||
\family default
|
\family default
|
||||||
operations when the kernel module is loaded.
|
operations when the kernel module is loaded.
|
||||||
All other operations should have no such restriction.
|
All other operations don't have such a restriction.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\emph default
|
\emph default
|
||||||
that, do
|
that, do on each of your hosts:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\family typewriter
|
\family typewriter
|
||||||
modprobe mars
|
modprobe mars
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Check that mars is running everywhere:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
marsadm view all
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Ignore any error message that no resources are yet defined.
|
||||||
|
But you should check that no warnings about network connections are appearing.
|
||||||
|
Both cluster nodes should be able to communicate with each other over the
|
||||||
|
MARS ports (default 7777 to 7779).
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Enumerate
|
||||||
|
Additional checks, to be executed on
|
||||||
|
\emph on
|
||||||
|
all
|
||||||
|
\emph default
|
||||||
|
of your hosts:
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\family typewriter
|
||||||
|
netstat -lp --tcp | grep 777
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
netstat --tcp | grep 777
|
||||||
|
\begin_inset Newline newline
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\family default
|
\family default
|
||||||
on each node.
|
Both variants should show up some healty connections.
|
||||||
|
If not, fix your network configuration and/or firewalling etc.
|
||||||
|
Details are outside of the scope of this manual.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\noindent
|
||||||
|
\begin_inset Graphics
|
||||||
|
filename images/MatieresCorrosives.png
|
||||||
|
lyxscale 50
|
||||||
|
scale 17
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
Beware of asymmetric connections, caused by inappropriate firewall rules.
|
||||||
|
|
||||||
|
\emph on
|
||||||
|
Any
|
||||||
|
\emph default
|
||||||
|
host must be able to communicate with
|
||||||
|
\emph on
|
||||||
|
any
|
||||||
|
\emph default
|
||||||
|
other host of the
|
||||||
|
\emph on
|
||||||
|
same
|
||||||
|
\emph default
|
||||||
|
cluster.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Section
|
\begin_layout Section
|
||||||
|
|
Loading…
Reference in New Issue