The short answers that matter for a 2-node edge/ROBO box: how to update, what happens when hardware fails, and how to swap a dead drive. If it isn't here, ask on the TWINSTOR thread.
One host at a time, and never touch the second until the first is back and healthy. You cannot half-update it into an unsafe state — see the note at the end.
On the first host, in turn:
tar xzf twinstor-test-<version>.tar.gz
cd twinstor-test-<version> && sudo ./install.sh
twinstor upgrade-apply # safe, gated restart; refuses unless healthy, blocks until reconverged
twinstor status # wait for: health: healthy (ready, Connected, UpToDate/UpToDate)Repeat on the second host only once the first is healthy again.
You can't get it half-done. The update is active the moment the service restarts — the daemon applies the new storage-safety settings live on every start, regardless of the on-disk config version. So if twinstor status later shows the config as outdated, that is expected and safe; it does not mean an unprotected pool. Refreshing the on-disk config is optional. On a healthy serving pool DRBD will not reset the live peer connection to apply it, so twinstor regenerate-config defers rather than forcing the change. To adopt it, do so in a maintenance window, one node at a time:
twinstor regenerate-config --persist-only --yes # stage it, then reboot this node; wait for healthy before the otherThe whole point of a 2-node box is that a single failure is a non-event. Here is what TWINSTOR does on its own, with nobody on site:
The classic ROBO power blip. The other host was already serving every VM's storage, so storage never stops. XHA restarts the lost host's VMs on the survivor (typically a few minutes). When the host powers back on it rejoins and resyncs by itself — nothing to do.
Both hosts down at once. Both boot, agree on which copy is current (no coin-flip, no split-brain), and bring the VMs back automatically. Your job is to wait for them to boot.
That host fences itself and evicts; its VMs restart on the survivor. Reconnect it and it rejoins and resyncs.
Absorbed. A flap shorter than the timeout stays Connected — no resync, no VM impact, no drama.
The VMs don't even notice: TWINSTOR drops the dead disk and keeps serving from the peer's copy (a little more latency, no failover). See "a drive died" below.
Storage keeps serving — the replication and the iSCSI target live in the kernel; systemd just restarts the service. Your VMs see nothing.
In every case the rule is the same: keep serving if it is safe, fail closed if it is not, and heal automatically once the hardware is back. If TWINSTOR ever needs a human it raises an alert in XO.
Not urgent. Nothing is down. When a data disk fails, the node keeps serving every VM from the peer's copy — so no VM stops and nothing fails over. You have simply lost the second copy on that one host until the disk is replaced. Fix it whenever it's convenient:
twinstor status shows the progress and tells you when both copies are current again. (A one-command replace-backing helper and a "backing device not found" alert make a device swap even simpler.)A failed disk is a "fix it this week" errand, not a 2 a.m. page — which is exactly the point of holding a full second copy on the other host.
Run twinstor status on both hosts. You want readiness: ready, connection: Connected, and disk UpToDate/UpToDate. Green on both = you are fully protected.
Usually nothing — it heals itself and raises an XO alert only if it genuinely needs a human. When in doubt, check twinstor status.
Every write is mirrored to the second host and only acknowledged once both hosts have it (synchronous replication, DRBD protocol C). Both hosts always hold a complete, current copy, so a single failure never loses committed data.
This is the one genuinely hard case for any two-node system — the CAP trade-off, not a defect. If a fault leaves both hosts up but unable to see each other and both keep taking writes, the two copies diverge and no merge exists: someone's most recent writes have to give. TWINSTOR is built to make that rare, small, and never silent. It fences an isolated host to stop both sides writing in the first place; if a true split still happens it keeps the copy with the newest writes — decided from DRBD's generation history, not a guess — so the rollback is the smaller side, bounded to the last few seconds; and it raises an XO alert naming exactly what was rolled back. For a workload that can't tolerate even that, run its two VMs' replicas anti-affine (never on the same host) so a single failure can't split them.
Only while you are running on a single node — the other one down or with a failed disk. That is why you fix a failed node or drive promptly: to get back to two copies. And keep your backups — replication protects against hardware failure, not against a deleted VM or ransomware.