The Texas Lottery Commission is facing legislative scrutiny again following allegations by a former employee that the commission spent $1.3 million on a disaster recovery facility that doesn’t work.
Lottery officials, who denied the accusation, took legislative staff on a tour of the site following a hearing to make their case, but much of the issue centers around technical details and how “operational” is defined.

The case began with a complaint from a former employee. Staff allegations have become a pattern at the lottery, which in recent years has had more whistleblowers than a kazoo factory.   In the most recent case, last June, the lottery advertised for a jackpot it couldn’t fund. A key staff member who tried to raise the alarm about the problem was fired, for what officials said was other reasons.
In the latest case, Shelton Charles, who ran the agency’s computer networking system, was fired after alleging problems at the agency’s disaster recovery site. Lottery officials said Charles was fired because he was a disgruntled worker who refused even to meet with his supervisor without a request in writing.

 Charles, who said he plans a lawsuit against the agency over his termination, said he was covering his back in what had become a “hostile workplace.”  On Nov. 14,  before the House Licensing and Administrative Procedures Committee, both sides aired their cases and argued over whether the expensive facility was operational or not.

Rep. Ismael “Kino” Flores (D-Mission), committee chairman, said he was skeptical about the commission’s side of the story, because Charles’ allegations seemed indicative of a pattern, as did the commission’s characterization of Charles as disgruntled.  “My frustration,” Flores said, “is that any time we get one employee to come up with something, they’re either disgruntled, or they get fired. We haven’t had a good employee yet. And yet they’re batting .999 with us. Everything they said is true. We found out through an employee of the [VLT bill drafting] contract, and we found out everything else we have found out through employees of the agency. But y’all either terminate them or call them bad employees.”

Disaster recovery facility
All state agencies maintain a disaster recovery site to back up crucial information and functions in case of an emergency.  Most agencies’ backups are housed at a secure and geographically remote  facility in West Texas. The Lottery Commission, however, was granted a waiver in 1992 to build its own facility, which it maintains inside a warehouse just outside Austin. The cost: $1.3 million.
Why a disaster recovery site for the lottery? The question is about more than just keeping powerball and charitable bingo running in the aftermath of a natural disaster. The lottery generates significant funds for the state, and even the slightest outage, caused by an event as simple as a fire in the commission headquarters on Sixth Street, could have huge consequences for the budget – particularly school finance.

Lottery commission representatives, including Commissioner Thomas Clowe and interim executive director Gary Grief testified that the facility is up and running and is performing all  its functions.  But when Rep. Corbin Van Arsdale (R-Tomball) asked when the facility originally got up and running, no commission spokesman could answer. Grief responded that it was “some time” in 1998 or 1999. He said he was unable to find any records that could pin down an exact date, a point which Van Arsdale and Flores said they found hard to believe.

Lottery Commission representatives instead made the case that the facility was a perpetual work in progress. Because technology is always changing, they argued, upgrades were ongoing, and the facility would never be in a final state.  “Because it changes every day. The minute we sign off on a disaster recovery continuity plan, that plan will be out of date the very next day,” said administration chief Mike Fernandez.  “So I guess in state government, you tell [contractors], ‘If you’re working on a contract, you really don’t have to finish it, you just keep it ongoing.’...,” Flores said. “All I’ve heard so far is, ‘We’re not supposed to be done. This is an ongoing project, and we’re going to keep going,  and we’re going to keep adding, and it’s going to change.’ That’s a real good line. That’s a very good line. You never have to be accountable for anything...That’s a great way to milk the system.”

Grief, who at the time was the commission’s Y2K compliance officer, said  getting the facility running was a massive project.  “It was undertaken over a period of many months,” he said, “and there were many different systems that were brought online, equipment moved in, computer rooms built out. I think it would be disingenuous for me to try to tell you that, on X date, everything was done.”

As for the status of the facility now, staff told the committee that the facility has the ability to back-up priority data on a real-time basis and less critical functions on a delayed basis. That, they argued, made the facility operational.

Data recovery vs. recovery strategy
But the definition of what constitutes operational is different in the mind of Charles – the employee who raised the issue. In his view – and he helped write the disaster recovery strategy - just getting the data back is not enough. Running the Lottery Commission in times of emergency, he said, requires far more than the data. It requires servers and increased bandwidth - things he says the facility lacks.

“We would have at least four to eight hours of restoration time per server to get that site up,” Charles said. “Now, if you have a single tape backup unit sitting at that warehouse, how will you meet those deadlines? You can’t. The laws of physics prevent that.”  The facility, he said, has never even been tested. Although it has the ability to communicate with G-Tech, the commercial company that runs the lottery games, the connection, Charles asserts, is inadequate.

“If I pull the plug out there, right now,” he said, “I want to see it come up, I want to be able to communicate with G-Tech...You can’t, because you don’t have sufficient bandwidth. You don’t have real-time applications that have to run. Those real-time applications do not have sufficient bandwidth.”  The lottery has a garden hose, he said. To deal with all the data G-Tech requires, it needs a fire hose.  Flores asked Charles how he knew that.

“Because I’m the engineer who worked on the design. I’m the analyst that helped design that disaster recovery plan,” he said. “I know that because I’ve been at that site, I’ve worked at that site. Every piece that’s in that site, I’m intimate with. Especially the communications side because I managed it all.”

A culture of fear
Charles said that he made the decision to come forward, knowing that it would ruin his decade-long relationship with the lottery. Like other employees who have testified before him, he described an oppressive atmosphere in which employees daily fear being terminated.

Rep. Delwin Jones (R-Lubbock) said he was disappointed that the committee couldn’t get straight and simple answers.  “I sense that everybody’s almost afraid to answer a question,”  Jones said. “Is there any reason to be afraid? To say, ‘I don’t know’?...Is there some kind of fear complex among all the employees out there?”

Rep. Tony Goolsby (R-Dallas) said the agency, despite hitting some bumps in the road, was still among the most profitable in state government.  “Great,” Charles responded. “Profit is a wonderful thing. But that agency should still be held accountable to the people of Texas, and it should still be held accountable to good business practices.”  Those practices, he said, included a working disaster recovery facility.

“If we accept the responsibility to build...a backup system,” he said, “that backup system should work. If we spend $1.5 million, it should work. Somebody should be able to tell you the date that that was turned on. They can’t. If they can’t, it’s because it doesn’t work.”.

Lottery Commission staff invited legislators to tour the backup facility. Legislators, admitting that they wouldn’t know what to look for, chose to send a staff delegation instead. On Nov. 17, a group including representatives from the State Auditor’s office, the Director of Information Resources and legislative staff toured the facility. They plan to report back to the committee soon on what they found.

Comments (0)

Comment	smaller | bigger
Name
Write the displayed characters
Subscribe via email (Registered users only) I have read and agree to the Terms of Usage.
	Add Comment