A Monthly Test of What AI Assistants Say About Your Firm
More people now ask an AI assistant a question they used to type into Google. "Who is a good DUI lawyer in Sacramento?" or "Do I need a lawyer for a first offense shoplifting charge in Phoenix?" The answer they get may name two or three firms, or none, or a firm that closed years ago. Most defense lawyers have no idea what these tools say about them.
You can find out, and you do not need special software to start. A simple monthly test, done the same way each time, tells you whether your firm shows up, what the assistants get right and wrong, and which sources they seem to lean on. That record is far more useful than a one-time check you do after reading a headline.
Pick a fixed set of questions
Write down 10 to 15 questions a real client might ask. Use the charges you most want and the cities or counties you serve. Mix the types:
- Direct requests for a lawyer: "best criminal defense lawyer in Fresno for drug charges"
- Situation questions: "my son was arrested for assault in Fresno, what should we do"
- Comparison questions: "is [your firm] or [competitor] better for DUI"
- Questions about you by name: "what do people say about [your name], criminal lawyer"
Keep the wording identical every month. If you change the question, you cannot tell whether the answer changed because of anything you did.
Run them in the tools people use
Run the set in ChatGPT, Gemini, Claude, Perplexity and Google's AI Mode, along with any AI Overview that appears in normal Google results. Use a logged-out or fresh session where you can, because assistants that remember your past chats may favor you after you have asked about yourself twenty times.
Answers vary from run to run, even with the same words. Run each question twice and note both answers. If a firm appears once out of two runs, record that honestly. Over several months the pattern matters more than any single response.
Some tools now show links next to their answers. Google in particular has been adding more visible links inside AI Mode, which changes what a firm should pay attention to, as covered in what Google's new AI Mode links mean for defense firms. When a tool shows sources, write those down too. They are your best clue about where the assistant learned what it said.
Record what you see in a plain spreadsheet
A shared sheet with one row per question, per tool, per month is enough. Useful columns:
- Date and tool
- The question, exactly as typed
- Firms named, in the order they appeared
- Whether your firm was named, and how it was described
- Any errors about your firm, such as a wrong address, a lawyer who left or a practice area you do not handle
- Sources or links shown
The errors column is often where the real value is. An assistant that says you handle family law, or lists a former partner, is repeating something it found online. Fixing the source usually fixes the answer within a few months.
Look at where the answers come from
After two or three months, sort the sources column. You will likely see the same handful of sites again and again: your own website, your Google Business Profile, a few legal directories, review sites, local news, bar association listings. Those are the places shaping how assistants describe you and your competitors.
Look at each one with a client's eyes. Is your profile on that directory complete and current? Does your own site say clearly, in plain text, which charges you defend and where? Do your lawyer bios state bar admissions and courts? Assistants pull facts that are stated plainly and repeated consistently across sources. Vague marketing language gives them little to work with.
Decide what to change, one thing at a time
The test is only useful if it leads to work. Common fixes after the first few months:
- Correcting outdated directory profiles and bar listings
- Adding clear charge pages for offenses the assistants never connect to you
- Making city and county coverage explicit on the site
- Asking satisfied clients for reviews that describe the type of case, within your state's ethics rules
Make one or two changes, note the date in the sheet, and watch the next few months of answers. Expect slow movement. These systems update on their own schedules, and some draw on training data that can be many months old.
Keep the ethics rules in view
If you later quote or describe AI answers in your own marketing, treat them like any other claim. An assistant calling you "the best DUI lawyer in town" does not make it a statement you can repeat. Most state bars restrict comparative and superlative claims, so check your state's advertising rules before using anything an AI tool says about you.
Questions about testing AI assistant answers
How long does the monthly test take?
Once the question set and sheet exist, about an hour or two per month for one person. The first month takes longer because you are writing the questions and setting up the columns.
What if no assistant names my firm?
That is common and it tells you where to start. Check whether your site, profile and directory listings clearly connect your firm to the charges and places in your questions.
Can I pay to be recommended?
The major assistants do not sell placement in their organic answers. Some platforms are testing ads, but those are labeled separately and are a different channel from being named in an answer.
For more on AI search and the technology behind it, see our guides to ChatGPT, AI search and legal marketing technology.
SBD Marketing works only with criminal defense law firms, one firm per city. Our work covers defense firm websites, SEO for the charges a firm defends, visibility in AI search tools such as ChatGPT, Gemini, Claude and Perplexity, and SEO audits.