'You can’t automate accountability': Tech executives tell students what AI skills will matter

ASPEN, Colo. — Artificial intelligence tools can make experienced engineers more productive while slowing down entry-level engineers who have not yet learned how to judge the technology’s work, May Mobility CEO and founder Edwin Olson said.

Olson spoke June 9 during “The Cost of Being Wrong,” a roundtable at Fortune Brainstorm Tech in Aspen. The panel focused on who remains accountable when AI produces a bad answer or unsafe decision, using examples from legal citations, cybersecurity audits, company workflows and autonomous vehicles.

During audience questions, I asked, “For students and early-career workers wanting to enter a field where trust and security matter so much, what AI mistake should they be aware of, and how do they avoid it?”

Gregor Stewart, chief AI officer at SentinelOne, said workers should resist treating large language models as though they think like people.

“LLM reasoning is not reasoning,” Stewart said. “LLM thinking is not thinking.”

The words people use to describe AI can make the systems seem more human than they are, Stewart said. That assumption can also enter the way companies design systems, particularly when they expect a model to eventually approximate human behavior.

Large language models generate sequences of tokens that correspond with an input, Stewart said. Even when their answers are usually useful, they can behave outside a user’s expectations.

Caitlin Halferty, head of data and analytics at Thomson Reuters, offered a different emphasis. She said AI can help workers move beyond the limits of their own experience and find approaches they did not initially consider.

Halferty described customer retention work at Thomson Reuters as one example. Instead of limiting the company’s analysis to an existing predictive model, she said employees used sentiment analysis to find signals in the data that could help retain customers.

“We would never have gotten there if we had been constrained to sort of the existing process and ways of thinking,” Halferty said.

Olson said the effect of AI tools depends partly on the worker’s existing knowledge.

At May Mobility, experienced engineers know what they are trying to build and can identify whether an AI-generated solution is good, Olson said. AI agents allow those engineers to complete work that had previously been limited by the number of hours available.

Entry-level engineers may instead rely on the system to supply both the answer and the judgment needed to evaluate it.

“They don’t know what good looks like, and they’re leaning on the AI itself for the wisdom and the direction of what should the architecture look like, and it gets it wrong,” Olson said.

Olson said early-career engineers should master technical fundamentals while learning how to direct AI agents with broad knowledge but limited judgment.

He did not present supporting data for the productivity comparisons during the roundtable. His comments described what May Mobility had observed as engineers began using AI agents.

May Mobility develops autonomous vehicle systems, where an incorrect decision can carry consequences beyond a flawed document or broken line of code.

“You’ve got a three-ton vehicle that might hit somebody and could cause a death or a serious injury,” Olson said.

Olson said May Mobility does not place all control in one large, unexplained model. Its vehicles use AI systems to generate possible actions and a supervisory system to evaluate them before control is assigned.

That structure allows the company to investigate whether a failure came from the options generated, the way the options were evaluated or the way the vehicle executed the selected plan, Olson said.

Halferty described a separate review process at Thomson Reuters. Before the company integrates an AI capability into a product, it conducts a data impact assessment and involves privacy, security or other teams when the product requires their review.

For legal professionals using AI-generated material, Halferty said the individual remains responsible for checking the source, citation and link behind an answer.

Stewart said automated systems can create another accountability problem by producing more material than workers can realistically inspect.

In cybersecurity, AI can examine a large number of events and generate additional findings for a security team, he said. The output can become more complicated than the original material the system reviewed.

“You end up in this space where you’ve got so much work that’s been done, so much work to audit, that you can’t truly be accountable,” Stewart said.

He rejected the idea that a worker or company could point to the system when a decision caused harm.

“You can’t automate accountability,” Stewart said. “You’re still responsible.”

Halferty said Thomson Reuters is encouraging employees to use AI through a leadership directive, access to tools and peer-to-peer demonstrations. Employees who find an effective use can show colleagues how they applied it.

The company also presents that work as a skill employees can take with them if they later accept another job, she said.

“You’re upskilling on the company,” Halferty said. “You’re learning and showcasing your AI skills, and that’s going to help your career more than anything else.”

Olson said employers would continue hiring skilled people, but their expectations would increase as AI tools allowed workers to produce more.

“The people who are able to use tools to increase their productivity are going to get the promotions,” Olson said. “And they’re going to be accountable for their work.”