BLOG

OpenAI for Law: Better Legal AI Starts with Better Context

Heath Harris

Heath Haris

VP of Applied AI & Enterprise Transformation, NetDocuments

I can’t remember the last time a managing partner asked me which model scored higher on a benchmark. I can remember plenty of conversations about governance, adoption, security, and whether AI can actually work with the firm’s knowledge.

That’s because legal organizations are entering a different phase of AI adoption. The debate is becoming less about models and more about what sits behind them. Those conversations almost always lead to the same place: context. Most firms already have the answers somewhere. The challenge is finding the right information, at the right time, for the right matter.

AI doesn’t have a knowledge problem. Most firms don’t either. They have an access problem. That’s why we’re excited to see NetDocuments included in OpenAI’s connector ecosystem as part of OpenAI for Law.

To be clear, this is not a new capability. NetDocuments customers already connect governed content to ChatGPT through Model Context Protocol (MCP). What’s changing is discoverability. Customers can now more easily find and activate NetDocuments within ChatGPT, making it simpler to bring trusted legal knowledge into the AI experience they already use. That means lawyers can stay in ChatGPT while securely accessing the governed context already housed in NetDocuments.

The bigger story is what this represents. Across the legal industry, we’re seeing a shift away from treating AI as a collection of disconnected tools and toward a more durable strategy: building a governed knowledge foundation that can power many AI experiences.

What Great Legal AI Really Needs

Most discussions about AI start with the model.

Legal work starts somewhere else.

A model doesn’t know your matter history. It doesn’t know which agreement was ultimately signed, how a similar issue was resolved three years ago, or which negotiating position your team successfully used with a client six months ago.

The information exists. The challenge is making it accessible, secure, and useful.

When connected to NetDocuments through MCP, ChatGPT can securely search and retrieve documents, uncover relevant precedent, compare agreements, summarize information, and assist with drafting using the organization’s own work product and knowledge. That’s a fundamentally different experience than asking an AI to figure everything out from scratch.

I’ve spent much of my career helping organizations rethink how legal work gets done. One lesson shows up repeatedly: technology creates the most value when it understands the environment it’s operating in. Legal AI is no exception.

Why Context Matters More Than Ever

One of the challenges in today’s AI market is separating excitement from evidence. That’s one reason we recently introduced the Legal Context Engineering Benchmark report.

The objective was simple: hold the model constant, improve the information available to it, and measure the result. The findings confirmed what many firms are already experiencing in practice – when AI starts with better information, it spends less time searching, less time guessing, and more time producing useful work.  

Using the same frontier model, richer legal context reduced the cost per correct answer by 48% while maintaining answer quality. For a large firm or corporate legal department, that could represent a million dollars in annual savings. We published the full methodology so that any organization can run the test itself.

That’s an important distinction, because the industry spent the first wave of AI adoption focused on models. The next wave will be defined by how effectively organizations organize, govern, and deliver their knowledge to those models.

Models, applications, and providers will continue to evolve rapidly. Most organizations don’t control any of that.

What they do control is their knowledge. Matter history. Precedent. Client relationships. Work product. Expertise. Those aren’t just data points. They’re the accumulated knowledge and expertise of the firm.

No model can supply that on its own. It must be built and governed, and the system where legal work already lives is the natural place to serve it from. In many ways, that’s the real competitive advantage, and that’s exactly what our Legal Context Graph is built to capture.

One thing legal teams learn quickly is that the answer is rarely sitting in a single document. It’s scattered across matters, emails, precedent, work product, and the people who know where to look. Lawyers spend an incredible amount of time reconstructing those connections. The Legal Context Graph is designed to do more of that work automatically. It works at three levels: what a document is and what it contains, how documents relate to each other inside a matter, and where expertise and precedent sit across the firm. That is a different exercise than classifying documents as they are filed.

One Governed Foundation, Many AI Experiences

Legal organizations are not standardizing on a single AI tool. They’re using ndMAX, ChatGPT, Claude, Microsoft Copilot, Gemini Enterprise, Harvey, Legora, Perplexity, and a growing list of specialized solutions.

It’s one of the reasons we invested early in MCP and an open, AI-agnostic architecture. Long before interoperability became a major industry discussion, we believed customers shouldn’t have to rebuild their strategy every time a new model entered the market. We think that’s the wrong tradeoff. Organizations shouldn’t have to choose between adopting new AI tools and maintaining a consistent knowledge foundation with a live context layer underneath them.

This becomes even more important as the industry moves beyond chat interfaces and into agentic workflows. Agents don’t just answer questions. They perform work. To do that effectively, they need access to the same information, relationships, permissions, and guardrails that humans rely on every day.

The value isn’t in building a new foundation for every AI tool. The value is in building it once and making it available wherever work happens.

A few years ago, the idea that firms could connect one knowledge foundation to multiple AI tools felt ambitious. Today, many are realizing it’s the only scalable approach.

Governance Must Travel with the Knowledge

Of course, access to information is only part of the equation. Legal organizations also need confidence that AI operates within the same governance framework that protects their information today.

As AI capabilities become more sophisticated, the questions become more important.

  • Who has access to information?
  • How are ethical walls enforced?
  • Can activity be audited?
  • What happens when an agent performs work on behalf of a user?

Those aren’t technical questions. They’re business and risk questions.

Our philosophy has been consistent from the beginning: governance must travel with the knowledge. When NetDocuments content is accessed through MCP, existing permissions, matter-level restrictions, ethical walls, retention policies, and audit controls continue to apply. The Legal Context Graph doesn’t just connect information; it carries the governance rules for that information along with it, no matter which AI tool is asking the question.

Every jump in capability raises the stakes for governance. As these systems become more autonomous, governance becomes more important. The industry is beginning to recognize something we’ve believed from the start: AI alone is not the advantage. The combination of AI, knowledge, and governance is.

The Next Chapter

The industry is moving from experimentation to accountability. Clients want transparency. Firm leaders want measurable value. Lawyers want technology that helps them move faster without creating new risks.

Meeting those expectations requires more than access to the latest model. It requires a foundation.

The legal industry isn’t suffering from a model shortage. If anything, we have the opposite problem. New models show up every few months and new assistants show up every few weeks. That’s not where I see firms struggling.

The harder challenge is organizing what the firm already knows, governing it properly, and making it available wherever work happens. That’s why we’ve invested in MCP, the Legal Context Graph, and an open ecosystem that spans leading AI models, legal technology platforms, and productivity tools.

If your firm is already using ChatGPT, Claude, Copilot, Gemini, or another AI platform, the question is no longer whether AI belongs in legal work.

The question is whether it’s operating with the right knowledge and governance behind it.

Because the future of legal AI isn’t about picking a winning model. It’s about making sure every model can benefit from the same trusted knowledge and governance framework.

    Learn more about the industry’s first Legal Context Graph and see results in the Legal Context Engineering Benchmark report.