A first-level fighter walks into a dragon’s lair, and the dragon smiles. A lawyer takes the same risk when he opens an AI account on Monday and then asks it to take on a catastrophic injury case on Tuesday. The AI comes back with a very confident answer. The lawyer takes that answer and assumes it’s competent. Somewhere in that answer, a fact has been altered, an authority has been lost, or a prediction has been made more precise than the evidence would support. The lawyer takes that answer into the real world. The dragon has company.
I grew up playing Dungeons & Dragons. Now I spend time thinking about artificial intelligence and how lawyers can use it. I wanted to bring these two nerdy interests together to make learning AI more interesting. The game gives us a way to approach the subject. You begin with limited abilities. You face challenges, learn from the results, and take on harder assignments as your skills improve. You earn the next level through experience.
D&D rewards learning via adventure. You create a character and then make your way through the story as directed by the DM. The current rules generally take your character from level 1 to level 20, with abilities added as you progress. There was an Immortals rules set in the older editions, so I did, in fact, remember something about immortality after all. Level 20 in the current game does not make you immortal, though.
For this adventure, you are a fighter. Choose your name, appearance, and background. You need no spells or experience rolling dice. You need a willingness to practice. I have paired the levels to organize our work with AI. These stages provide a teaching framework; they do not reproduce the game’s advancement rules. A partner with 30 years of trial experience may start at the first level. A new claims professional may begin several levels ahead. Professional seniority and AI experience measure different things.
We aim to get better at defending claims. The path for defense lawyers, claims people, and in-house counsel is to go from playing with AI to using it. We want better questions, work product with a real purpose, and decisions properly based on the record. We also need to know when the tool lacks the information or authority to go any further. You learn by doing work that you can assess. Purchasing software doesn’t get you any further in the learning process.
Levels 1 and 2: Choose Your Equipment
Learn what your equipment does before you enter the dungeon. Your fighter starts with one weapon and one pack. You start with an AI tool. Generative AI produces text and other content in response to instructions and supplied material. Use it to organize an argument, compare documents, or write a report. A fluent answer does not establish that the system searched current law, read every attachment, or found the right passage. The tool’s access and capabilities determine what it can do.
Choose a Task You Can Evaluate
Start with one approved tool and one assignment you can evaluate. Give it a fictional paragraph describing a rear-end collision. Ask it to rewrite the paragraph for a claims professional deciding what to investigate next. Compare its version with yours. Did it preserve uncertainty about the collision? Did it turn a reported symptom into a diagnosis? Did it add a concession? Review each change. This exercise teaches you how the tool works and where its answer needs your attention.
Repeat the exercise with a change that tests your attention. Add a witness who disagrees about the traffic signal. Change an allegation to an admission. Remove the sentence that explains where the collision occurred. Ask for the same update and compare the responses. The goal is to see whether the tool tracks a change that matters. Keep your own answer beside its draft. You learn the tool by examining what it does with the facts you give it.
Protect the File Before You Open It
Choose the environment before you use a client’s file. Find out which products your firm, company, or insurer permits, what information those products may receive, and what restrictions the client imposes. Assign someone to review the provider’s terms, retention practices, access controls, and how it uses submitted information. A paid subscription or a product marketed to lawyers does not settle those questions. Removing a person’s name may still leave enough facts to identify the matter. Resolve how you will protect the information before you disclose it.
Florida Bar Ethics Opinion 24-1 addresses the duties governing this work. It examines confidentiality, competent work, oversight, and billing. It recommends obtaining the affected client’s informed consent before using a third-party generative AI program if that use would disclose confidential information. The opinion is advisory, and the underlying professional duties remain controlling. Resolve the applicable requirements before entering client information into the system.
Choose work whose quality you can judge. Editing a fictional client update gives you a place to begin. Asking for a reserve recommendation on an unfamiliar claim gives you little basis to assess the result. You complete these first levels when you can explain why the tool may receive the material, state the assignment, and identify an error in the response. Bring those habits to each new task. They equip you for the next encounter.
Levels 3 and 4: Give Instructions and Check the Result
Give the tool a defined assignment. At the next village, someone asks your fighter to investigate trouble on the road. You need to know where, when, and what will resolve the problem. AI needs that direction, too. A request to “analyze this case” leaves too many choices to the tool. State the decision the work must support, the audience, the permitted sources, and the treatment of missing information. Those instructions also establish the standard you will use to review the answer.
Give the Tool a Specific Assignment
Follow one fictional claim as you practice. A delivery truck collides with a passenger car at a warehouse entrance. The plaintiff alleges a back injury, lost earnings, and a need for future surgery. The trucking company disputes how the collision occurred and whether the crash explains the claimed condition. Witness accounts differ. Some records remain missing. I invented this claim for training. We can get through it without exposing a client’s information.
Begin with a question that serves the next decision: “Using only this fictional incident report, draft a 250-word update for the assigned claims professional—separate reported facts from disputed accounts. Identify information needed to decide whether to obtain an accident reconstruction opinion. Do not supply facts missing from the report.” This prompt names the audience and the purpose. It limits the sources. You can now check whether the answer gives the claims professional what the decision requires.
Show the tool an example when words alone can be confusing. Supply a fictional Update that uses your preferred organization and explain why it works. Perhaps you begin with the decision, summarize the supporting facts, and finish with the next assignment. Tell the system to adhere to that structure while using only the new record for facts. Check whether it copied a fact from the example. An example teaches a format; it must not become evidence in another claim.
Check the Answer Against the Source
Expect to revise the instructions. If the tool gives you a page of generic investigation suggestions, ask which suggestions address a specific uncertainty in this report. If it describes the truck driver as negligent, direct it to identify the source supporting that conclusion. Then read that source yourself. Requesting support can expose a weakness, but the tool’s explanation cannot validate its own answer. A citation may point to a real document that says something different.
Verify the answer through its sources. AI can propose wording, questions, and competing explanations. You must check that work through a method suited to the task. Read legal authority in a reliable research source and check its status. Check a calculation with a spreadsheet or another calculation tool. Return to the transcript to verify testimony. NIST identifies confidently stated false content as a generative AI risk. Build verification into the assignment. Telling the model to be accurate does not perform that work.
Save a prompt after you test it and understand its limits. Keep a brief note of the failure you corrected. Perhaps the system confused an allegation with an admission or omitted an unfavorable fact. That note becomes part of your experience. You advance when you can improve a weak answer by changing the assignment and explain why you trust the revision.
Ask for a list of unresolved questions before requesting a conclusion. In our truck case, the tool may need to know whether the incident report quotes the driver or summarizes someone else’s account. That distinction could change the update. Invite the system to identify what it needs, then decide whether the absent data warrants further work. You need not answer every question it generates. You do need to recognize which unanswered question prevents the assignment from producing a useful result.
Levels 5 and 6: Build a Map of the Record
Map the record before you plan the defense. Your fighter cannot choose a route around a ravine that the map omits. You cannot evaluate a claim without knowing what the file contains and what it lacks. Ask AI to extract events from records and connect each event to a source. This assignment moves the tool beyond editing prose. First, confirm that it received the documents and can read them. A chronology cannot account for a page the system never processed.
Know What You Have
Begin with an inventory. Identify the documents, their dates, and the pages in each file. Check whether scanned pages have usable text. Optical character recognition, which converts an image of text into searchable text, can misread a date, skip handwriting, or scramble columns. An upload confirmation does not establish that the system processed everything. Ask for a document inventory and compare it with the files you supplied. Resolve missing documents and unreadable pages before relying on a chronology.
Build the truck case chronology from a defined set of fictional medical records. Ask the tool to identify each consultation, the complaints, the examination findings, the treatment, and any work restrictions. Require the document name and a stable page locator for each entry. Keep the original pagination or connect each working page number to its original. The reviewer needs to move from the summary to the supporting page without searching the file. That connection enables review as the record grows.
Separate the source date from the event date. A physician may write a note on Thursday about an examination on Monday. An employer may produce a payroll report months after the period it covers. Ask the tool to preserve both dates when they differ. In the truck case, that practice helps you distinguish the timing of a complaint from the timing of its documentation. A chronology that sorts everything by the file’s creation date can send the investigation down the wrong road.
Preserve What the Records Say
Preserve what each record says. A patient’s denial of leg pain supports a different proposition from a note that says nothing about leg pain. A report of back pain before the collision does not establish that every later symptom shares that cause. Instruct AI to preserve those distinctions and identify inferences. If two records describe different onset dates, retain both accounts. A smoother story may conceal the very discrepancy that you need to investigate.
Now compare the medical chronology with the employment and incident records. The plaintiff’s reported last day of work may differ from the payroll record. Perhaps a treatment note refers to a prior examination you do not have. Ask the tool to identify the documents that would help resolve each discrepancy. Read the underlying entries before declaring a contradiction. A change in job duties, a delayed payroll entry, or a simple recording error may explain what looks like impeachment.
Turn Gaps Into Next Steps
Assign each defense team member a role in closing the gaps. Claims professionals can request records and flag developments for counsel. In-house counsel can locate company witnesses, records, and custodians. Outside counsel can connect missing information to the investigation and discovery. Give each person the material the assignment requires, subject to the matter’s access and confidentiality rules. Sharing a chronology does not require sharing every document with everyone who reads it.
As the file grows, maintain a record of additions and corrections. Ask AI to compare the new material with the existing chronology and propose changes, then have a reviewer approve those changes. Keep the earlier version where appropriate. A corrected injury date can affect several later documents. Knowing which versions used the old date helps you repair the work. You advance when you can trace a material statement back to its source and explain what the current record cannot establish.
Levels 7 and 8: Prepare for Discovery and Depositions
Turn gaps in the record into questions you can investigate. Your fighter reaches a guarded bridge. Before charging ahead, you ask why the guard stopped you. Defense work calls for the same pause. Once you have mapped the record, give AI a disputed issue and the materials that address it. Ask what evidence would distinguish the competing explanations and how you could obtain that evidence. Then decide which inquiry deserves your attention.
Target the Missing Evidence
In the truck case, the parties disagree about whether the passenger car had stopped before the impact. Ask AI to map the available accounts against the evidence that couldtest them. A nearby camera may help, but you need its location, field of view, and retention history. Vehicle data may matter, but you need qualified advice about what exists and what it can show. An AI suggestion identifies a potential inquiry. It does not establish that a device recorded the event or that a particular expert can retrieve it.
Use the resulting questions to draft targeted requests, then apply your legal judgment to scope and wording. AI often proposes discovery broad enough to overwhelm the file. Ask how each request connects to a disputed issue and whether a narrower request would obtain what you need. Counsel must evaluate proportionality, preservation, privilege, and the governing discovery rules. An AI draft can impose the same burden and provoke the same objection as a request you write yourself.
Give Each Question a Purpose
Build a deposition outline around the facts you need to explore. Connect each proposition to an exhibit or an unanswered question. Then develop a sequence. If the plaintiff claims that back symptoms first appeared after the collision, locate the earlier records before drafting questions. Establish what each record says and what the witness can explain. Give AI those limits when you ask it to propose questions. Your desire to find a contradiction cannot supply one.
Tell the tool how you ask questions. For an adverse witness, I favor short, leading questions that address one fact at a time. Ask for a progression from general facts to the specific point you need to establish. Review the outline aloud. A question may look efficient on the screen yet ask about three events or assume something the witness has never admitted. Delete questions whose purpose you cannot explain. Fifty useful questions can prepare you better than five hundred that obscure the decisive issue.
Prepare for the answer that disrupts your outline. Ask AI to suggest a plausible explanation for each apparent discrepancy, then check whether the record supports that possibility. The plaintiff may explain that an earlier back complaint resolved before the collision. Plan the follow-up questions about duration, treatment, and work activity. Leave room to listen. The outline gives you a route through the testimony, but the witness may identify a turn you need to take. Your judgment controls the next question.
Connect Testimony to the Next Decision
After the deposition, ask AI to connect testimony to the earlier issue map. Require page-and-line references and enough surrounding context to preserve qualifications. Distinguish an affirmative denial from a failure to recall. Keep objections and corrections where they affect meaning. Scrutinize each proposed impeachment passage because impeachment depends on the exact statements and their context. The tool can gather candidates. You decide whether the testimony supports the point and how to use it.
Explain how the testimony changes the next decision. Telling the claims professional that a deposition went well says little. Identify which factual dispute narrowed, what is still uncertain, and whether the testimony changes the investigation or settlement plan. Decide the substance before asking AI to draft the report. In-house counsel may identify a company record that answers a remaining question. Every completed task should help the defense team decide what to do next.
Levels 9 and 10 Test Opinions and Arguments
Test what supports an expert’s opinion. Farther along the road, a stranger offers your fighter advice about a danger ahead. You ask how the stranger knows. Bring that question to the expert’s report. Supply AI with the report, the testimony, and the materials the expert identifies. Ask it to separate the conclusion from the facts, assumptions, and methods that support it. That comparison helps you identify what to investigate and what to ask at deposition.
Test the Assumptions Behind the Opinion
In our fictional case, a physician attributes a proposed lumbar surgery to the collision. Ask AI to locate the passages addressing the patient’s prior condition, symptom onset, objective findings, and alternative explanations. Then ask which materials the report appears to rely on and which relevant materials the report does not discuss. An omission does not establish that the physician ignored a record. It identifies a question for investigation or deposition. A missing discussion and an unsound opinion require different responses.
Build a question sequence around the actual assumptions. If the physician assumes the plaintiff had no earlier back complaints, identify the source for that assumption and the earlier record that may challenge it. Ask whether the opinion would change under the alternative facts. Ask a defense medical expert to assess the medical significance. AI can prepare the comparison, but it cannot supply a qualified opinion on causation, surgical necessity, or prognosis. Those judgments require the appropriate professional and the actual clinical evidence.
The same method applies to an accident reconstruction or economic opinion. Ask what inputs drive the result, whether the report identifies them, and what happens when a disputed input is modified. Keep arithmetic in a dependable calculation tool. An expert’s spreadsheet may use a different work-life assumption from the one stated in the report. That discrepancy may warrant an explanation. It does not automatically establish misconduct or demandexclusion. Use AI to frame a question that gets the explanation you need.
Keep an opinion chart that shows what would change the analysis. For each disputed assumption, identify the supporting source, the competing evidence, and the person who can explain its significance. In the truck case, a reconstruction expert might need to address a different point of impact. An economist might need to address a return to work. Ask AI to organize those branches. Then bring the questions to the expert. A list of possible weaknesses does not tell you which weakness affects the conclusion.
Verify the Law Before You Use It
Read every authority you plan to use. Give the tool the jurisdiction, the procedural question, and the date through which you need current research. Ask it to distinguish binding authority from other authority and to identify adverse decisions. Then verify the cases, quotations, subsequent history, and legal propositions through reliable research sources. The tool may cite a real case for a proposition the court never adopted. It may state a rule but omit a controlling exception. Read beyond the passage that appears to support you.
Florida reinforced that responsibility in its 2026 amendment to Rule 2.515(d)(2). Effective June 15, 2026, the amendment requires a signer to represent that the legal authorities identified in a filing exist and that the signer cites them accurately. The Court also authorized sanctions after notice and an opportunity for the signer to respond. Its commentary describes a statewide replacement for varied circuit orders addressing AI disclosure and certification. Build the current rule into filing review, and check the requirements of the forum in which you practice.
Build the motion from verified facts and law. Provide AI with the authorities you have read and the supporting record. Ask it to identify where the proposed argument lacks a premise or source. Connect each proposition to the evidence you would present to the court. The draft may reveal a weakness by placing the steps of the argument next to each other. Use that result. Sometimes the work supports a stronger motion. Sometimes it shows why you should abandon one.
Levels 11 and 12 Test Strategy and Settlement Choices
Evaluate each step against the case goal. Your fighter may win an encounter and consume the resources needed to finish the adventure. Defense teams face that choice, too. A deposition may establish a point without changing whether another expert expense will improve the likely resolution. Ask AI to compare the available paths. Define the choices, supply the facts, and identify what each choice would cost or accomplish. Keep the whole matter in view.
Compare the Choices
Ask it to examine a specific decision, such as whether to obtain another expert opinion before mediation. Describe the disputed issue, the evidence already available, the proposed expense, and the decision the opinion could change. Ask for the strongest reason to proceed and the strongest reason to stop. Then ask what additional information would make the choice easier. This exercise gives the claims professional and counsel something concrete to discuss. It also challenges the habit of continuing discovery because you can.
Show What Drives the Numbers
Demand support for every proposed probability. A percentage in an AI reply does not become an empirical estimate because it includes a decimal point. Ask what data support it, whether those data concern comparable matters, and whether anyone has tested the method against outcomes. The number may reflect an assumption that the tool cannot substantiate. Label that assumption. Defense lawyers can discuss risk without presenting an untested guess as a measured result.
Suppose the team wants to explore three hypothetical trial outcomes of $250,000, $750,000, and $2 million. If the team assigns illustrative probabilities of 25 percent, 50 percent, and 25 percent, the weighted average is $937,500. A spreadsheet can calculate that figure. The calculation says nothing about whether the selected outcomes or probabilities are sound. It also leaves out additional expenses and other relevant consequences unless you include them. Use the exercise to expose the assumptions that drive the discussion. Do not present the result as a settlement value generated by AI.
Change one assumption at a time to see what drives the result. Hold the proposed outcomes constant and vary the probabilities. Then hold the probabilities constant and change the damages assumptions. Ask the team which change reflects evidence and which merely tests a possibility. Record that distinction beside the calculation. This exercise helps counsel explain why one missing medical opinion or employment record could matter more than another round of discovery. The numbers serve the discussion when everyone can see their limits.
Trace each damages figure to its support. Ask AI to organize the information supporting future care and identify missing evidence. Separate a proposed treatment from a recommendation the physician made. Distinguish a charge from an amount paid or another figure relevant under the governing law. Use a table to locate duplicate entries and inconsistent descriptions. Counsel and qualified experts must determine what those differences mean. Check the law that governs the claim and its dates before applying a damages framework.
Prepare for the Other Side
Ask AI to present the plaintiff’s strongest account of the evidence. In the truck case, that account might emphasize a driver’s opportunity to avoid the collision and the effect of persistent pain on work. Give the tool the unfavorable evidence along with the favorable evidence. Ask it to identify what a fair reader might find persuasive and what the defense account fails to explain. Resist the temptation to argue with the response until it agrees with you. The exercise works only if it can expose a weakness you would prefer to overlook.
Use role-play to test your case explanation. AI can act as a skeptical mediator, a demanding claims supervisor, or a client asking why another expense is necessary. Those conversations may reveal an unclear explanation. A panel of AI jurors supplies no observations from real people and cannot establish how a Florida jury will decide the case. County labels and detailed fictional biographies do not solve that problem. Use the exercise to generate questions you can investigate, and keep it separate from actual jury research.
Use AI to plan how you will explain the case at a hearing or trial. Ask it to propose a sequence for a timeline and a comparison of witness accounts. Tie every proposed entry to the record. Keep disputed accounts distinct, and inspect any visual for labels or details that could imply facts the evidence does not establish. In the truck case, an arrow on a diagram might suggest a vehicle path that no witness described. Decide what the exhibit communicates, consider the applicable evidentiary requirements, and prepare the foundation yourself. The tool can suggest a presentation. You must defend what it tells the audience.
Prepare the mediation memorandum around the decisions the team must make. Give AI the conclusions that counsel and the team have reviewed. Require the draft to identify uncertainties and the evidence that could resolve them. Name the person who holds settlement authority and specify what that person still needs. Counsel, claims, and the client may have different roles and interests. The memorandum should help each person carry out that role. Those with authority decide whether to settle and on what terms.
Write for the person who must act on the recommendation. The claims professional may need to explain the request to a supervisor. In-house counsel may need to connect it to an operational concern. Ask AI to adapt the team’s reviewed analysis for each audience without changing its conclusions. Check both versions against the same source memorandum. An explanation can become misleading when the tool removes a qualification to shorten the draft. Brevity helps only when the reader receives what the decision requires.
Levels 13 and 14: Build a Library Your Team Can Use
Organize your sources so the team can find what it needs. Your party has collected maps and reports from earlier adventures. Those records help only when you find the right one. A defense practice faces the same problem with briefs, deposition outlines, expert materials, and company records. At these levels, you move beyond supplying a few documents in a conversation. You design how an AI system will locate and use a collection. Start by learning how the system searches it.
Know What the System Searches
Three different features often sound alike in a sales presentation. A saved instruction tells a tool how you want it to work. A collection of uploaded documents supplies material it may consult. A system connected to a document repository may retrieve selected passages when you ask a question. None of these features show that every answer considers every relevant document. Ask the provider to explain which material the system searches, which material reaches the model, and how the answer identifies its sources.
Retrieval-augmented generation, or RAG, helps a model answer questions using information from a collection. The system searches for relevant material and uses the results to help produce an answer. An organization can use this approach with its own approved documents. The method still requires review. The search may miss the right document. The answer may misread a passage that the search found. Follow the source references and check the underlying text before relying on the result.
Keep Sources Current and Access Limited
Begin a firm pilot with a small collection of approved, current research memoranda on one recurring procedural issue. Require each memorandum to identify the jurisdiction, the research date, and the person responsible for maintaining it. Include the underlying authorities where access permits. Mark superseded material and decide whether to remove it from ordinary retrieval. A strong argument written under an earlier rule may become a poor starting point after the rule changes. The collection needs an owner who can make that distinction.
Limit access to the truck file to the people who may work on it. Preserve document versions and stable references within the collection. A broad question must not give a user access to another client’s file. Ask the technology team to test permissions when the system retrieves material, including attempts to reach restricted documents through indirect questions. A prompt that tells the system to respect confidentiality does not enforce access controls. The repository must restrict what the system can retrieve for each user.
Claims departments and in-house legal teams must decide which collections belong together. Company incident data, coverage analysis, defense strategy, and routine claims operations may serve different purposes and have different access requirements. Obtain advice about the treatment of privileged and work-product material before connecting those sources. Do not assume that labeling an entire folder privileged settles the issue. Design access around what people need while preserving the boundaries counsel identifies.
Test the collection with a question whose answer you already know. Ask for the governing company procedure on the date of the collision, then compare the response with the version you have verified. Include a later revision in the test collection. See whether the system distinguishes the two and explains which date controls the assignment. That exercise tests more than whether the search finds a document. It tests whether the answer uses the document that belongs in the analysis.
Test What the System Can Find
A model’s limited working context creates another practical problem. A system may store many documents but consider only selected material for a response. Ask it to identify which documents it used and what information it lacked. Test questions whose answers require connecting passages from different documents. For example, ask whether a work restriction appears in both a physician’s note and the employer’s account. Review both records yourself. A system that finds each document separately may still fail when the question requires comparing them.
Define what a vendor means by training your own model. Fine-tuning changes a model’s behavior through additional training on examples. Providing documents for retrieval is a different process. Before paying for either, identify the defect you need to correct. If the tool cannot find a current endorsement, the first problem may be document organization or retrieval. If the output varies from your required format, better instructions and examples may be enough. Ask vendors to demonstrate the proposed improvement on your approved test material.
Levels 15 and 16: Build and Test Workflows
Connect the tasks without letting one error travel through the file. Your party needs to assess its progress through each passage before entering the next. An AI workflow needs those checkpoints, too. Define the sequence, identify the material each step requires, and name the reviewer. Decide what allows the process to continue and what requires it to stop. Work that succeeds in a single conversation still needs these controls before other people depend on it.
Give Each Phase a Job
Start with new records arriving in the truck case. The first step inventories the files and flags unreadable pages. The second proposes additions to the chronology. A reviewer checks the new entries and settles discrepancies. The next step compares the approved additions with the existing issue map. Counsel decides whether any change affects discovery, expert work, or valuation. Only then does the system prepare a proposed claims update. Require the report to identify the development that matters and the action counsel recommends.
Break the work into steps you can inspect. Asking one prompt to read a file, resolve the medical issues, revise the strategy, and notify the client makes the result harder to check. If a report uses the wrong surgery date, trace the error to the scan, the extraction, or the drafting step. Correct the source of the error and any work that followed. Stop when the process lacks a required source. A gap calls for investigation, not a paragraph that hides it.
Specify what each step must return. A medical encounter entry might require a date, a document identifier, a page, a supporting quotation, and a proposed summary. A system can check whether those fields exist. A reviewer must decide whether the passage supports the summary. Correct formatting does not establish accuracy. Assign mechanical checks to the computer and substantive judgments to a qualified reviewer. Give each reviewer enough information to perform that task and authority to return deficient work.
Use code for a defined task when it improves a process you understand. AI might write a script that checks whether chronology entries contain page references or identifies duplicate file numbers. Have someone who understands the code inspect it before use. Run it on a copy of approved test material and compare its output with known results. Keep it away from the original file until the team verifies its behavior. You remain responsible for the assignment even when the tool writes the instructions that another computer executes.
Test Against Answers You Have Checked
Create a test set before expanding the project. Use fictional records or material your organization has approved for this purpose. Include ordinary examples and examples likely to cause mistakes. Put two patients with similar names in separate files. Include conflicting dates, a scanned page with poor text, a record that contains no answer, and testimony that qualifies an earlier statement. Have qualified reviewers establish the expected findings from the sources. Asking the same model to write the questions, decide the answers, and grade itself provides little independent assurance.
Measure what the assignment requires. For record extraction, count missed relevant encounters as well as incorrect entries. For source support, check whether the cited passage supports the statement. For a document comparison, include changes the system should find and passages it should leave alone. Examine serious errors individually. A system that gets ninety-nine routine dates right and misses the one date that controls the assignment may be unsuitable for that use, even if the overall percentage looks impressive.
Test the reviewer as well as the system. Give the reviewer a sample that contains a plausible but unsupported statement and ask for the source that supports it. If the reviewer accepts the statement because the report looks complete, another on-screen warning will accomplish little. Change the review process. Require the reviewer to open the cited page and resolve the discrepancy. A checkpoint protects the file only when someone performs the work that the checkpoint requires.
Measure the Whole Process
Review the report for what it omits as well as what it says. Accurate sentences can mislead when the system leaves out the strongest unfavorable evidence. Ask reviewers whether the draft addresses the decision and explains uncertainty. Compare it with the organization’s existing process. Count the time to prepare the input, check the result, and repair errors. A two-minute draft may save little if review takes longer than doing the original work. The benefit may come from finding an issue sooner rather than reducing hours.
For example, suppose a limited pilot reduces a reporting task from ninety minutes to sixty minutes, including review. Across twenty reports, that would save ten hours. These figures illustrate the calculation; measure your own results. Consider whether the reports became more consistent and whether the reviewer found the material facts more easily. Compare the total expense of the process with the value of the work it produces. An added subscription has to earn its place in the practice.
Retest when the model, instructions, source collection, or connected tools change. Keep a version of the test set that has not become part of the tool’s examples. That gives you a better chance to see whether the process handles unfamiliar material. Preserve a way to return to the earlier workflow and a person authorized to stop the pilot. A quiet change that makes answers less reliable is easier to address when someone owns the review and knows what performance had been acceptable.
Keep a record of why you changed the process. Identify the problem, the adjustment, and the test that showed whether it helped. A new instruction may fix one error while creating another. For example, telling the system to omit uncertain entries might remove an event that deserves investigation. Compare the revised output with the earlier version and examine what disappeared. Your fighter checks the map after choosing a different passage. Your team should check the whole result after changing a workflow.
Levels 17 and 18: Assign Work to Agents
Define the authority you give an AI agent. Your fighter now leads companions who can carry out assignments. To give them an objective, you must decide what they may do to achieve it. A conventional AI workflow follows a defined sequence. An agent can choose intermediate steps and use connected tools to pursue an objective. Along the way, it may search, retrieve, calculate, or modify something. Anthropic’s engineering guidance distinguishes these approaches and recommends adding complexity when the task warrants it.
Limit the Assignment and the Permissions
Give the agent a first assignment it can complete within your controls. For the truck file, the agent could inspect the approved folder, identify additions, retrieve relevant passages, and propose changes to the issue map. Require a weekly report that gives the assigned lawyer source references and pending questions. Limit the assignment to changes that the new records support. Specify the objective, sources, output, and person who will decide what happens next.
Set permissions according to the task. An agent reviewing records may need to read a matter folder without deleting files, contacting witnesses, or sending reports. A tool that can draft an email need not have permission to send it. Keep settlement communications, coverage decisions, court filings, and other consequential actions behind the appropriate human authority. Technical access should follow those choices. An instruction to ask permission provides a weaker boundary than withholding a tool the agent has no reason to use.
Increase an agent’s authority in stages. Begin with a task that produces a draft for review. If the pilot succeeds, consider whether the agent should place that draft in a designated folder or update a field after approval. Decide each permission on its own merits. Success at reading records does not establish competence to change the case plan. Before adding access, identify what the new permission accomplishes, what could go wrong, and how the responsible person would detect and reverse an error.
Control What the Agent Can Follow
Protect the agent from instructions hidden in the material it reads. A document may contain text that tells the system to ignore its assignment or transmit information elsewhere. Researchers call this problem prompt injection. Treat outside documents as evidence to analyze, not as instructions that can direct the system. Ask the technology team how the product separates the two, limits tool access, and tests attempts to cross that boundary. A document cannot grant the agent authority that your team withheld.
Set limits on time and expense. Require the agent to report a missing document after a defined search. Limit the folders it may search and the paid services it may call. Decide what happens when a tool fails, the system cannot read a page, or the agent finds conflicting source material. Those are normal conditions in a litigation file. A useful system returns the unresolved issue to the person responsible instead of inventing an answer to complete the assignment.
Prepare to Find and Correct Errors
Keep enough records of the agent’s work to reconstruct a material error. Record which documents it accessed, which tools it used, and what it changed or proposed. Apply the organization’s retention and legal-hold requirements to those records with counsel’s guidance. Avoid collecting unnecessary sensitive material because a tool can log it. The purpose is to understand and review the work while respecting the obligations that govern the file.
Make the handoff readable to the person receiving it. Require the agent to state what it completed, what it could not complete, and what needs a decision. In the truck file, “weekly review complete” tells counsel less than identifying three new records and an unresolved conflict about the surgery date. The recipient needs to know where the review begins. A clear handoff also prevents someone from treating an unfinished assignment as an approved conclusion and carrying that mistake into the next report.
Give separate AI roles separate jobs. One process can assemble proposed facts while another checks whether the cited sources support them. A third may examine whether the draft omits contrary evidence. But several AI responses do not become independent corroboration because you assign them different job titles. They may rely on the same incomplete collection or make the same mistake. Give each role a defined function, compare performance with a simpler process, and retain independent human review of the material conclusions.
You do not have to program an agent to understand what a vendor proposes. Ask for a demonstration in which the system lacks a source, encounters an access restriction, and attempts an action beyond its assignment. Watch whether the system stops and explains the problem. Ask how your team can disable it and recover from an unintended change. These questions tell you more about a product’s fitness for defense work than a demonstration that supplies perfect records and follows the easiest path to a polished report.
Levels 19 and 20 Lead the Campaign
Lead the team by choosing work that AI can improve. Your fighter now answers for decisions that affect the party. Your AI work reaches beyond your screen when you help a practice group, claims department, or legal team adopt a tool. Ask which recurring task needs improvement, who can judge the result, and what the team will do after a material error. Use the pilot to gather evidence for those decisions. Leadership requires knowing what the work needs.
Choose Work That Needs Improvement
Choose a small number of uses that solve recurring problems. A firm might begin with source-supported record summaries and deposition preparation. A claims department might focus on organizing incoming material and improving reports for supervisor review. An in-house team might start with locating relevant policies and company records for counsel. Give each pilot a responsible professional, approved sources, and a way to compare results with existing work. Expand after the pilot demonstrates value on representative assignments, including difficult ones.
Document the division of responsibility across the defense team. Outside counsel controls the legal work within the engagement and the client’s instructions. The claims professional carries out the responsibilities and authority assigned to that role. In-house counsel advises the organization and coordinates matters within the applicable structure. Use AI to clarify those responsibilities. For example, a proposed report can identify a recommendation awaiting counsel’s review and a settlement request awaiting the authorized person’s decision. Keep both decisions with the people who hold that authority.
Choose a person to own the process across the handoffs. The lawyer may understand the legal issue while the claims professional understands why the report arrives too late to support a reserve discussion. In-house counsel may know that the company stores a missing record elsewhere. Bring those observations together before buying another feature. Ask where the work stalls and who can remove the obstacle. AI can accelerate a task while leaving the delay around it untouched. Improve the whole assignment.
Explain the Work and the Bill
Discuss the arrangement with the client and insurer at the appropriate stage. Explain what the proposed use does, what information it receives, and how people will review the result. Resolve required disclosures, consent, and contractual restrictions before the workflow depends on the tool. Maintain the explanation specific. A blanket statement that the firm uses AI gives the reader little help in understanding whether the proposal entails editing an internal paragraph or giving a service access to a substantial matter collection.
Billing deserves the same clarity. If you charge by the hour, record the actual time spent, including appropriate work to check and revise the result. Do not charge the hours you would have spent without AI. Address any separate technology charges under the engagement terms and applicable requirements. ABA Formal Opinion 512 discusses these issues, including actual-time billing and the general prohibition on charging clients to learn a tool the lawyer will regularly use. Its guidance concerns the Model Rules; Florida lawyers must apply their governing rules and client arrangements.
Teach Judgment and Learn From Mistakes
Train people on the work they perform. Give a claims professional, a defense associate, and a supervising partner the same fictional record assignment. Compare where each finds value and where each identifies an error. The claims professional may notice that the report never explains why another expense is necessary. The associate may catch a missing page. The partner may see that a question assumes a disputed fact. Use those observations to improve the workflow and choose the reviewer for each step.
Teach judgment alongside the tool. Before asking AI to build an outline, have the lawyer identify the issues and explain the deposition’s purpose. After receiving a draft, require the lawyer to defend the question sequence with references to the record. Occasionally work without the tool so the supervisor can assess the lawyer’s own understanding. A system that produces a convincing answer can conceal a training gap. Design training to expose and address that gap.
Have the learner teach the assignment back to you. Ask why the prompt uses those sources, which error would matter most, and how the learner checked the answer. Then change one fact and ask what must change in the work product. This exercise reveals whether the lawyer understands the process or has memorized a prompt. It also gives the supervisor a way to coach the underlying skill. Advancement requires explaining the work well enough to help someone else perform it.
Make mistakes easy to report and hard to repeat. A missed medical entry, an unauthorized retrieval attempt, and an awkward sentence require different responses. Identify whether the cause lies in the input, the instructions, the system, or the review. Correct the affected work and decide whether other matters need attention. Add the example to the test set when it would help prevent a recurrence. Give the next user the benefit of the problem the first user found.
Advance When the Results Support It
If you are starting today, give one approved tool and one task a full month to evaluate. Use the next month to improve that task, document its limitations, and try it with a small group. In the third month, decide whether the evidence supports a wider pilot. These are suggested intervals, and the work may take longer. The sequence matters more than the calendar. Let the evidence determine whether you expand the work.
Keep measuring after the rollout. Track review time, material errors, and whether the work supports decisions. Ask users which parts they rewrite or ignore. Assign someone to update sources and retire uses that no longer perform well. Remove a complicated process when it repeatedly fails the assignment. A team can advance by simplifying its work. Higher levels require the judgment to stop using a tool, as well as the ability to start using one.
The Next Adventure
Start where you are. I brought Dungeons & Dragons into this discussion because learning AI can seem like standing outside a dungeon while everyone else knows the way through. People talk about models, agents, and integrations as if we all got a rulebook years ago. Choose a task you understand. Practice, examine the result, and correct the mistake. You can learn the game one encounter at a time.
Let each assignment prepare you for the next. Our truck claim began with an incident report. It grew into a record collection, factual questions, and decisions about discovery and settlement. We then built a process otherscould use and review. At each stage, we asked what the next decision required and whether the work supported it. That question gives the technology a purpose and gives you a way to judge its value.
Keep developing the judgment your clients hire you to exercise. Products will change. Vendors will add features. Clients will still need someone who understands the file, recognizes a missing premise, and explains a recommendation. Learn the tools while strengthening those abilities. Help colleagues begin, too. Answer the question you once had to ask. Share the error you learned to catch. The party advances when its members help one another improve.
My old game offered extraordinary powers and, in some versions, immortality. Our clients ask for something we can deliver: a defense team that understands their problem, uses its resources well, and takes responsibility for its advice. Bring AI to that work. Choose your first assignment. Learn from the result. Then enter the next room with more experience than you brought to the last one.
Frank Ramos is a partner at Goldberg Segalla in Miami, where he practices commercial litigation, products, and catastrophic personal injury. You can follow him on LinkedIn, where he has about 80,000 followers.
The post Leveling Up With AI In Civil Defense: A Dungeons And Dragons Adventure For The Defense Team appeared first on Above the Law.
Recent Comments