Before generative AI, finding information inside a law firm generally followed a familiar pattern of search, retrieve, and evaluate. It had to be that way.
It’s an open secret that firms have spent years scattering information among document-management platforms, inboxes, file shares, SharePoint sites, and business applications. But at least until a lawyer judged which of the five versions of a contract was the final, no one wrote a brief or took any next steps in a case.
Now, AI absorbs material from across an organization and delivers a polished answer, often without clarifying how it got there or acknowledging the disorder hiding in the underlying data.
“We’ve always known that organizations don’t have confidence in their information,” said Kurt Thies, global head of Information Governance Strategy at the legal services consultancy Morae. “But the consequence of that now has hit a whole new level.”
According to Morae’s research, only 26% of legal leaders feel confident that their organizations’ information is ready for effective AI analysis. Only 37% say they even know where all their legal information is stored.
The striking gap illustrates the importance of information governance and how firms need to prioritize it before adopting more tools that process bad data at extraordinary speed.
‘ROT’ Doesn’t Become Fresh Data
Information-governance professionals use “ROT” to describe redundant, obsolete, and trivial data points. When AI enters the picture, the clutter can shape the output.
Consider a contract that exists in 15 versions, each with its own edits, comments, and abandoned negotiating positions. A lawyer reviewing search results can investigate which copy was executed. An AI system given access to all 15 may treat each as useful evidence.
The same goes for a 20-year-old agreement that no longer governs the relationship, or trivial notes such as text messages about a lunch with the client.
“AI doesn’t inherently know which information is authoritative, current, or appropriate,” Thies said. “But it is utilizing it in its outcome.”
Risk Can Arise With the Wrong Response
Bad information hygiene also creates exposure to risk. Only 30% of legal leaders are confident that their information is properly classified as confidential or sensitive, and AI-enabled software struggles to tell the difference.
“Not all data is equal,” Thies said. “You have certain information that is high risk, high obligation.”
Organizations have regulatory reasons to retain certain records and an obligation to protect personally identifiable information from cybersecurity breaches. Yet material like this often sits in the same sprawling information environment as ordinary business files.
Thies offered the scenario of someone who has been keeping HR files on a shared drive. The arrangement may have felt safe if that employee was the only person using the folder, but the guardrail disappears when an AI system gains access.
Failing to properly classify or index information beforehand creates a potential privacy incident with Social Security numbers or health information. The system locates personally identifiable information, confidential business records, or privileged material and could expose those materials to someone who never should have seen it.
Relevance Has to Be Established First
The fragmented landscape of a typical firm’s information potentially gives new meaning to the sentiment “garbage in, garbage out,” Thies said.
A retention policy should bring order to that landscape, but in many firms the policy exists mainly on paper. Many organizations aren’t confident it has been applied to properly dispose of records when they’re eligible.
Records that are no longer relevant remain available to an LLM alongside current, authoritative material.
“AI lacks the organizational context unless it is given parameters through the governance of that information,” Thies said. “Here’s where we keep our official records in this repository. This is what you can rely upon. This is where we’re going to point our LLMs to use this information.”
Information Governance as an AI Enabler
Legal organizations are pouring money into AI because the potential productivity gains are significant. Trust, however, has not followed.
Only 33% of senior legal professionals trust the results of AI-assisted legal work, according to a Morae survey of 850 senior legal professionals. And 67% are concerned the cost of human verification is outweighing the productivity benefits of AI.
Thies noted that information governance is not an administrative brake on innovation. Rather, it’s a reminder to take an honest account of what their systems contain and where their highest-value information lives.
Otherwise, AI won’t provide trusted and reliable results.
“The industry can get there, but only by confronting the condition of its information first,” Thies said. “Right now, we know we’re not there yet.”
The post Is ‘ROT’ Data Thwarting Your Firm’s AI Adoption? appeared first on Above the Law.
Recent Comments