You have 1.5 million new messages

By Dan Ross

When Gov. Matt Blunt appointed me as the chief information officer for the state, one of his charges was to make certain that if disaster strikes the Capitol or another state office building, we can recover critical information. We have systems to recover information in the event of a disaster.

Unfortunately, I have read false reports in the media and statements by Attorney General Jay Nixon and his political agents asserting that my office has easy access to the hundreds of millions of e-mail messages sent and received each year by state employees. They falsely claim that requests for massive amounts of information including thousands of e-mails on "backup tapes" can be responded to quickly and produced with little cost, resources or manpower.

At the risk of sounding like a stereotypical office IT guy, Jay Nixon and his political operatives could use a basic lesson in technology. I want to set the record straight so everyone can know the difference between our current disaster recovery system and the governor's proposed e-mail retrieval system.

Imagine opening your e-mail to find 1.5 million new messages every day. This is what my office manages. We provide e-mail for approximately 30,000 workers processing approximately 1.5 million messages per day, or 50 e-mails per employee. We use IBM's Tivoli Storage Manager (TSM) to provide disaster recovery services for this system which "saves" information with a backup that provides a snapshot of information at the end of the day.

The TSM system is designed to support information recovery including e-mails at a single point in time. For example, if a server crashes we can use this system to retrieve information. While the TSM is used for disaster recovery, it is not designed for permanent e-mail retention and retrieval.

As a result, recovering multiple messages for large periods of time is extremely resource-intensive and requires significant manpower and system assets. Additionally, placing this level of stress on the TSM creates a disaster recovery problem for all other systems supported by it. If allowed to continue, the outcome of a future server failure could include the loss of multiple days of critical data.

Under the current system, recovering massive amounts of e-mails also comes with a high cost. For example, Jay Nixon's operatives have requested thousands of e-mails from 43 accounts in the governor's office. This request will take approximately 30 minutes per e-mail account per day (30 minutes x 43 accounts x 136 days of e-mail), or 2,924 hours labor. Further estimates are that the 2,924 hours estimated time for retrieval will cost $35 per hour, or $102,340.

Additionally, each e-mail must be reviewed to see if it is within the scope of the request, and, if so, whether the record is exempt under Chapter 610 RSMo such as information regarding medical, personnel, security, law enforcement and legal actions or litigation. Estimates are that the time involved in this process is approximately 2 hours per account per day (2 hours x 43 x 136), or 11,696 hours. Additionally, the estimates for the review time of 11,696 hours will cost $40 per hour or $467,840. This is the amount quoted by the attorney general for research time for records requests to his office.

While costly, this estimate is significantly less than what Secretary of State Robin Carnahan has estimated for her office. Carnahan's office said it will cost $91,000 to retrieve e-mails from one account in the secretary of state's office. At that rate, it would cost $3.9 million to retrieve e-mails from 43 accounts in her office.

Last November, Governor Blunt issued two directives. First, he directed that all e-mails in his administration are to be retained forever. To date, administration agencies are the only ones in state government retaining all of their e-mails, and my office is helping them do it.

Unfortunately, this causes the TSM to operate well past its designed parameters, which places other state government information systems at risk. Backups were taking 12 hours to process, so we upgraded our systems to prevent loss of data. To retain all e-mails and keep the backup system operable, we purchased system enhancements totaling $589,922, and similar enhancements are required every six months. Therefore, a new system is badly needed.

The governor's second directive was for my office to create a permanent e-mail archive system. This new system will not only help us retain every e-mail in the governor's administration while alleviating the stress on our disaster recovery system, but it will help us more easily retrieve information when it is requested.

The e-mail archive system is currently being implemented and will address four key points: e-mail retrieval, e-mail system performance and cost reduction, policy automation and administration, and inbox management. We have Symantec archive system engineers on-site to assist with its implementation.

Once implemented, when massive amounts of e-mails are requested, we will be able to retrieve them more quickly and at a much more affordable cost, which will mean greater efficiency and transparency for taxpayers.

Dan Ross of Jefferson City is the chief information officer for Missouri.