[Catalyst] FW: Clustering catalyst apps

[Matt S Trout]

Johan Lindström wrote:
> At 22:14 2006-05-08, Dave C wrote:
>> The key to offer the "five nines" availabilty (99.999%, or under 5
>> minutes a year) is to examine faults in every aspect, including
>> application, hardware, network, facility, and OS to identify single
>> points of failure.
> 
> If you go that far, don't forget to make sure your two independent ISPs 
> really are independent and don't buy their upstream bandwidth from the same 
> provider :)
> 
> That happened to us a couple of years ago; the upstream provider had some 
> downtime and we were mightily upset.

At $ork[-mumble], we had two links, both physically entirely separate 
(our BNetworkAdminFH had ensured that they even went out different sides 
of the building). Unfortunately, one time some bastards half-filled a 
bunch of wheely-bins full of petrol, lit it, waited a few seconds, then 
emptied the bins down carefully-chosen manholes over comms line 
intersections, reducing fibre-optic bundles to slag quickly and 
effectively. They got about 80% of the major intersections in the area, 
naturally including both our lines.

Sometimes even not having a single point of failure won't save you.