Gov. Tim Walz faces troubled IT agency, the only department still lacking a permanent leader

Gov. Tim Walz came into office hoping to address high-profile issues such as health care and education, but now he finds himself confronting a more immediate problem — the state's beleaguered technology system.

"It keeps me up at night," Walz said in an interview last week.

The same day that Walz rolled out seven new commissioners to enact his agenda in early January, a computer hardware problem paralyzed a Metro Council program for transporting people with disabilities and the elderly, leaving some stranded for hours.

The only top job Walz has yet to fill is the head of the state's IT system, known as MNIT. Walz appointed an acting boss while he continues looking for a chief information officer, but for now MNIT remains the only department without a permanent Walz appointment — underscoring the agency's pervasive problems and the governor's struggle get to someone to take them on.

Walz's team approached Republican Rep. Jim Nash — who works in cybersecurity in the private sector — about applying for the job of new MNIT commissioner.

Nash declined. "It's a job that even under a Republican governor I don't know that I would have taken," he said.

Nash's reluctance is understandable. State government computers run thousands of applications that help the state complete its myriad responsibilities — often collecting sensitive, private data that organized criminals are trying to steal. As in the private sector, computers aid government in nearly every sphere of activity, from animal and human health records to forestry management, highway traffic monitoring, welfare eligibility and police records. Additionally, the agency's budget is set by the Legislature, which means the technology leaders compete against education, health care and natural resources for money.

Many of these programs rely on software that was developed decades ago and run on hardware like mainframe computers reminiscent of the campy '90s tech movie "Hackers." The risks of failure — including a catastrophic data breach that could expose Minnesotans to hundreds of millions of dollars in damages — are significant, according to Walz, lawmakers, technology experts and even the agency itself.

Aaron Call, the state's chief information security officer and a former police officer, said his bleakest scenario even includes loss of life: "There are systems we support that provide capabilities to State Patrol. When they aren't able to get information, that's my biggest fear."

Nash, of Waconia, said that at the current trajectory, the likelihood of a major breach in the state's cyberdefenses is inevitable. "It's not a matter of if; it's a matter of when," he said.

The agency's troubles have become well known due to some high-profile failures since the 2012 Legislature ordered state government to consolidate information-technology functions under MNIT, in the hopes of achieving cost savings, better security and accountability.

The rollouts of the state's health insurance marketplace, MNsure, and a new vehicle registration system were both deeply flawed, with the latter a decade and $100 million in the making.

And the problems are ongoing: On that Friday in early January when Walz was preparing to take office, a computer hardware failure wreaked havoc on Metro Transit phones, internet, website and the scheduling and routing system for Metro Mobility, the program for the elderly and riders with disabilities.

The system was down for about seven hours, causing delays and forcing the agency to reimburse customers who had to find rides elsewhere.

State agencies are often relying on aging hardware and a dearth of backup systems due to a lack of funding, which means without new money, expect more routine outages, MNIT said. "If the money isn't invested, then we'll continue to see failures," Call said.

Although the problems are technical, they are also deeply political.

There's no constituency at the State Capitol for more IT spending — no teachers union, disability advocates or farmers to lobby for tax dollars for IT spending.

Walz summed up the political challenge: "The constituency is everyone when it fails, and no one when it works."

And the Legislature's patience is wearing thin. A broad — and likely critical — legislative audit examining the agency is expected to be published soon. Republican lawmakers have been especially quick to attribute MNIT failures to symptoms of dysfunction in the administration of former DFL Gov. Mark Dayton and government more broadly.

Since 2016, the GOP-controlled Legislature denied $30 million in MNIT requests to upgrade its cybersecurity. The agency operates 16 data centers — a number that security specialists find stunningly large — but the agency needs money to reduce that number.

"The more you're exposed to the internet, the more facilities you have, the more exposure you have," said Call, the agency's security specialist.

The agency's budget is often misunderstood. Although it spent $623 million last year to run the state's IT systems, just $2.6 million of that came from the Legislature, with the rest charged to state agencies for products and services.

That funding stream can cause conflict between agencies. MNIT's central office acknowledges it was not fully plugged into problems with the vehicle registration project, which was being run out of the Department of Public Safety.

"There's a natural tension as we work to consolidate," said Jon Eichten, MNIT deputy commissioner

Nash is more blunt, describing a culture of resistance to MNIT among some agencies.

The technology agency also faces challenges familiar to many high-tech employers.

Call said his best people can leave after just two years for a 50 percent raise in the private sector. Even established private companies struggle to find software architects, senior developers and managers; state government contracts are collectively bargained, and the state can't match the industry's skyrocketing salaries.

Republicans said the agency should aggressively seek contractors who can bring private-sector experience.

MNIT, which has about 2,300 employees, is already spending about $133 million this year on contractors. The agency said many of its computer applications are so old that only MNIT veterans know how to run them.

Walz brings experience with knotty government IT problems. In Congress, he sat on a Veterans Affairs subcommittee that was formed to conduct oversight of an eight-year, $19 billion project to merge the health records of the Department of Defense and the VA.

He praised MNIT employees and said he understands resistance to change: "We're remaking the plane as it's flying."

Walz said that in his hunt for a new chief information office, who will make about $150,000 per year, he is competing with seven-figure salaries in the private sector. He wants a skilled manager who understands the government's current IT mission but also has a vision for the future.

Walz said he knows he has to get it right. He said, "I'm not going to risk all these agencies, their data or the effectiveness of government by getting this one wrong."

J. Patrick Coolican • 651-925-5042