You have prepared for disasters. You are ready for almost everything. You have the data secured, employees are trained in evacuation and you have appropriate drills, your sprinkler system has just been tested and your security force and disaster team are brimming with pride.
And then they get tested, and you start to discover things overlooked.
In a building in the Midwest, a typical summer storm-related blackout occurred. Power went down, but the computer back up power supplies kept the system running. Emergency lights came on and the non-essential people left, as the emergency generator kicked in to power the key areas of the building including the data centers, executive offices, hallways, and the main lobby. The main lobby?
That’s right, the main lobby was lit up like a Christmas tree, while all around was dark. The unseen halls were lit, the unseen data center was running, and the lobby stuck out like a sore thumb in the middle of a dark city.
The security and the disaster recovery people were so proud of their achievement that no one thought of it until a VP came down and said “Why are all those people outside staring at the lobby?”
Well one more light went on – this one metaphorical – and the lights were turned out in the lobby.
Now there is a slightly modified protocol – when the emergency system kicks in, keep safety lights on, darken the lobby except for emergency lights, and make sure all of the shades are drawn on the executive floor. (The data center has no external windows, and is not likely to become an attractive nuisance.)
Why was this change made? Well, in the test there was no real issue. But this was just a test. What if it were a real situation. In this case a well-lit lobby would become an attractive resting place for those trying to escape rain or snow. While in a regional crisis it might well be appropriate to shelter as many people as possible, in a normal crisis this would not be appropriate, and would potentially cause a lot of problems.
How had this been overlooked? In this case nobody had given thought to the psychological implications of a brightly lit lobby when all around was dark. Although the generator was capable of handling the lobby, in this case it was not a swell idea.