What Attorneys Need to Know About AI Architecture

AI is no longer optional for legal departments. Every major law firm and corporate legal function is either deploying AI tools today or evaluating them for near-term adoption. But adopting AI without understanding its architecture is like signing a contract without reading the key terms. You may get a workable outcome. You may also be accepting exposures you did not intend.

This is not a technical deep dive. You do not need to become an engineer. But you do need to know which architectural decisions carry real consequences for privilege, data governance, and cost — and you need to know what questions to ask before someone else makes those decisions for you.

In Your Legal Data Belongs to You, we explored why data sovereignty has become urgent in the era of AI. In Why We Built on Microsoft, we explained the platform decision that shapes everything else. This article builds on both: three architectural decisions that every attorney evaluating AI tools should understand.

---

Decision One: Where Does the AI Run?

This is the most consequential question, and it is the one most often glossed over in vendor demos. Where AI processing physically occurs determines who can access your data, which jurisdictions govern it, and whether privilege survives the interaction.

There are three models in the market today, each with different implications for legal departments.

External API processing. Your data leaves your environment, travels to the AI vendor's infrastructure, is processed there, and returns. The vendor's servers may sit in a jurisdiction you did not choose. Your privileged documents are, for some duration, resident on someone else's infrastructure. Data may be retained temporarily — where "temporarily" is defined by the vendor's policies, not yours.

Embedded in the vendor's platform. AI runs within the vendor's application — a step beyond raw API calls. Many vendors implement substantive data segregation today: logical multi-tenant separation, dedicated tenants, and in some cases physically isolated compute environments. The architecture works when the vendor implements it well and the customer can verify it. The question for legal departments is not whether the vendor is well-intentioned but whether the controls are auditable. Confirm in writing what segregation model applies, where data physically resides during processing, what audit evidence supports the claim, and what happens to your data after the interaction completes. Your compliance posture is the vendor's commitments plus your verification — not the vendor's commitments alone.

Within your own tenant. AI operates inside your Microsoft 365 boundary. Data never crosses your perimeter. Processing is governed by your existing security policies, your Conditional Access rules, your Data Loss Prevention configurations. This is the Microsoft 365 Copilot model, and it is the approach Spaarke uses.

Consider two scenarios that make the distinction concrete.

A litigation team uses an AI tool to analyze privileged strategy documents — case assessments, settlement positions, internal recommendations from outside counsel. Under the external API model, those documents leave the organization's boundary for processing. Under the tenant model, the analysis happens inside the same environment where those documents already reside, governed by the same policies that protect them.

A contracts team asks AI to compare proposed terms against the organization's negotiation history — past positions, concession patterns, counterparty behavior across dozens of prior deals. Under the external model, that institutional knowledge flows to someone else's servers. Under the tenant model, the comparison runs on your data, in your environment, with no egress.

The difference is not theoretical. It is the difference between structural privilege protection and contractual assurance. As we detailed in Tenant Dedicated Deployment: The New On-Premises, structural compliance — where your architecture enforces the boundary — is fundamentally more reliable than promissory compliance, where a vendor agreement is the only thing standing between your privileged data and exposure.

---

Decision Two: What Data Grounds the AI?

Where the AI runs determines data security. What data grounds the AI determines output quality. These are separate questions, and conflating them is a common mistake.

There are three approaches in the market, and they produce fundamentally different results.

Public training data. The AI knows what the internet knows. It can summarize general legal concepts, draft boilerplate language, and answer questions about publicly available case law. Useful for general tasks. Unreliable for anything specific to your organization, your matters, or your business context.

Retrieval-augmented generation (RAG) with your documents. The AI retrieves your documents at query time and uses them to inform its responses. This is better — the outputs reflect your actual files rather than generic knowledge. But quality depends heavily on which documents are indexed, how well they are organized, and whether the retrieval mechanism finds the right materials. Garbage in, garbage out still applies.

Grounded in structured operational data. The AI draws on your organization's matter history, spend patterns, negotiation precedents, outside counsel performance data, and accumulated institutional context. This is not document retrieval. It is reasoning over the structured Memory layer that captures how your organization actually operates — decisions made, outcomes achieved, patterns observed over years of practice.

This third approach is what the Inference layer of the Legal IQ stack delivers. When AI is grounded in your organization's operational memory, the difference in output quality is not incremental. It is categorical.

A generic AI tool asked to estimate the cost of an employment matter will give you an industry range. An AI grounded in your operational data will tell you what your department has historically spent on similar matters, with which firms, at what stage costs typically escalate, and where your estimates have historically been optimistic. One gives you a number. The other gives you a decision framework built on your own experience.

The Legal IQ stack's Memory layer — the accumulated decisions, rationale, and institutional context described in our earlier article — is what makes this possible. Without it, AI is just a faster way to process documents. With it, AI becomes a genuine extension of your organization's collective judgment.

---

Decision Three: What Is the True Cost Model?

AI pricing in legal technology is opaque by design. Understanding the real cost structure — not just the line item on the quote — is essential for evaluating long-term value and avoiding adoption friction.

Three pricing models dominate the market.

Per-seat licensing. Every user pays a fixed fee regardless of how much they use AI capabilities. Predictable for budgeting, but potentially wasteful if adoption is uneven. Some seats may generate hundreds of AI interactions per month while others generate none.

Per-query or per-token pricing. You pay based on usage — every question asked, every document analyzed, every summary generated. This scales with actual consumption, but it creates a chilling effect on adoption. When every query has a visible cost, teams self-censor. Attorneys hesitate to ask exploratory questions. Legal ops professionals avoid running analyses that might not yield immediate results. The tool that was supposed to drive efficiency becomes something people use sparingly to avoid budget scrutiny.

Bundled within existing licensing. AI capabilities are delivered as part of an existing enterprise agreement — in Spaarke's case, through the Microsoft 365 ecosystem. The incremental cost is absorbed within licensing your organization already manages. No per-query anxiety. No adoption friction.

The headline seat number is rarely the whole bill. The legal AI market in 2025–2026 has produced a consistent set of cost vectors that surface only after deployment scales:

• Required companion subscriptions. Some legal AI tools sit on top of a research subscription that has to be licensed separately — Thomson Reuters CoCounsel Core requires Westlaw Precision; Lexis+ AI requires a Lexis+ base license. The "AI seat" is rarely the whole bill, and the companion can match or exceed the AI line item.
• AI processing and semantic-search overhead. Model inference, semantic indexing of newly added documents, search-query volume against the indexed corpus, and custom-model fine-tuning are all compute-driven and scale with use. They show up as overage line items, capacity uplifts, or premium-tier features. These are the costs that surprise procurement teams who modeled the deployment as a flat per-seat line.
• Consumption and token overages. Even at fixed per-seat tiers, heavy document review or agent workflows can trigger usage overages above bundled allowances. Ask for the overage rate card in writing — token rates, document-processing fees, agent-run charges — and model what happens at 2× and 3× initial projections.
• Implementation and premium support. Premium support runs 15–20% of annual contract value as an industry norm; enterprise implementation can reach six figures for organizations over 200 users. Both belong in the budget conversation, not the footnote.
• Annual escalation and bundle uplifts. Multi-year contracts in this market increasingly include "mostly increased" escalation language and content-bundle premiums; renewal uplifts have been projected at 30–40% as content tiers roll in. Insist on a numeric escalation cap.
• Forced migration risk. Recent acquisitions in the legal AI space have voided "lifetime" or "locked-in" pricing on legacy products; replacement subscriptions have come in at 50% or higher. Build acquisition-survives-pricing language into the contract.

The key insight: usage-based pricing is structurally hostile to AI adoption. The entire value proposition of AI in legal operations depends on broad, frequent use — asking more questions, running more analyses, surfacing more patterns. A pricing model that penalizes usage undermines the very behavior that generates value. The bundled model within existing Microsoft licensing removes this friction entirely.

---

Decision Four: Does the AI Grounding Cross the Engagement Boundary?

The first three decisions are about how AI works inside one organization's platform. The fourth is about the engagement itself — and it is the dimension the per-vendor evaluation tends to miss entirely.

Most legal AI tools today are sold to one side of the engagement, not both. The corporate legal department buys an in-house tool. The outside firms it engages buy firm-side tools — usually different ones. They work the same matter through two separate AI systems, with two separate operational memories, two separate model fine-tunings, and two separate sets of grounding data. Outputs can be exchanged. Drafts and analyses move back and forth. But the underlying intelligence layer — the patterns each AI has accumulated, the institutional context it has learned — stays siloed at the engagement boundary.

This is operationally significant. The matter the in-house team has been working for six months is the same matter the outside firm picked up last week — but the AI grounding the outside counsel's tool has none of the negotiation history, opposing-counsel patterns, or settlement-strategy rationale the in-house tool has been building. The reverse is also true. The intelligence each side accumulates is the intelligence each side keeps. There is no architecturally enforced way for AI to learn across the engagement, even though the work is the same work.

The consequence is fragmented intelligence on shared work. Two AI systems on opposite sides of the same matter is not the same as one AI system grounded in both perspectives. The output may look comparable; the depth of the grounding is not.

For legal departments evaluating AI tools, the question to add to procurement is whether the architecture allows in-house and outside counsel to operate against the same grounded intelligence layer for the matters they share. Most AI tools today do not — the platforms are sold to one side or the other, and even when the same vendor sells to both, the deployments do not share grounding. Spaarke's three-stakeholder coverage is built around this question — one matter record, one operational memory, one Legal IQ stack across the business stakeholder, in-house counsel, and outside firm. The broader point is structural: whatever platform a department chooses, the question is whether the AI grounding survives the engagement boundary or stops at it.

---

The Questions You Should Be Asking

Every legal department evaluating AI-enabled tools should be asking these questions. They are not technical questions. They are governance questions — and they belong in the room where procurement decisions are made.

• Where does AI processing occur — your environment or theirs? If theirs, what data leaves your boundary, and for how long?
• Is your data used to train models that serve other customers? Can you opt out, and how do you verify?
• What data sources ground the AI's outputs? Public data, your documents, or your structured operational data? The answer determines output reliability.
• Does AI grounding cross the engagement boundary? Will your in-house team's AI share grounding with the AI your outside counsel uses on the same matters, or will each side's operational memory stay siloed?
• What is the all-in three-year cost, and where does it scale faster than headcount? Ask the vendor for a three-year total-cost projection that names: (1) the per-seat price and any seat minimums, (2) any required companion subscriptions or content bundles, (3) the rate card for token, query, document, or agent-run overages, (4) AI processing and semantic-search costs as separately-attributed line items, (5) implementation and premium support as percentages of annual contract value, (6) the numeric annual escalation cap, and (7) a written commitment that the price terms survive a vendor acquisition. Where the vendor cannot answer in writing, treat the gap as a budgeted contingency.
• Can you audit AI interactions? Are AI queries and responses logged within your compliance and audit framework?
• What happens to AI-processed data after the query completes? Is it retained, cached, or used for model improvement?

You will be in the room when these decisions are made — or you should be. AI adoption in legal is not an IT project. It is a governance decision with direct implications for privilege, compliance, cost, and competitive position. These are the questions that separate informed adoption from risky experimentation.

---

Where to Go Next

For the data sovereignty foundation behind these architectural questions, read Your Legal Data Belongs to You. For the three-layer architecture that makes AI grounding work — how Data, Memory, and Inference compound to produce organizational intelligence — see The Legal IQ stack: Data, Memory, Inference. In our next article, we will explore how Spaarke implements these principles within the Microsoft 365 Copilot plane — delivering AI capabilities without requiring you to give away the keys to your data.