How Counselors Can Use AI Without Losing the Human Core of Therapy

AI scribes, chatbots, and risk-scoring tools are moving into counseling practices faster than ethics codes, state boards, and insurers can address them. Illinois' WOPR Act, in effect since August 2025, makes the stakes concrete: clinicians now carry disclosure and consent duties for AI use, not best-practice suggestions.

That exposure sits in documentation, informed consent, and clinician accountability, not in the algorithm itself. The working definition, audit framework, and liability review that follow respond to the October 2026 announcement of Deborah Gray's forthcoming Counselor-Led AI, but the standard-of-care questions are already live in everyday practice.

What Human-Centered AI in Counseling Actually Means

AI tools usually arrive in counseling practices through note-taking apps or practice-management platforms, often before clinicians make a deliberate choice. Human-centered AI means using technology in counseling that supports the counselor's judgment and the therapeutic relationship, rather than replacing or steering either.

Human-centered vs. automation-first

A human-centered AI scribe drafts a progress note for the clinician to review, edit, and sign. An automation-first tool might auto-generate a risk score or a treatment plan that flows into the record without the same kind of review. The difference is not whether software makes suggestions, but whether a licensed human remains the decision-maker.

A two-question test

Before using any AI tool in a session or workflow, ask two things: Who is accountable for this output if it is wrong, and can the clinician override or reject it? If the answer is the clinician and yes, the tool fits a human-centered frame. If not, treat it as a higher-risk automation.

The tools marketed to counselors fall into recognizable categories: session scribes, note and billing assistants, client-facing AI therapist chatbots for intake or between-session support, and risk-flagging analytics. Treat every one of these as a candidate for evaluation, not a default part of care until accountability and override paths are clear. That is the practical meaning of human-centered AI: the clinician stays in the loop from start to finish.

The GRACE Model as a Pre-Deployment Audit for AI Tools

The GRACE Model appears in Deborah Gray's forthcoming book, Counselor-Led AI: Integrating Tools Into the Therapy Room (Grayson Strategies, LLC, scheduled for September 2, 2026), as described in an October 9, 2026 press release. GRACE is one author's framework, not a professional consensus or a regulatory standard, and no independent book reviews or professional association commentary were identified in the available coverage. Treat the questions below as a risk management checklist, not as a product endorsement.

What ACA, NASW, APA, and AAMFT Say About AI

Some professional bodies now publish AI-specific resource pages; others still ask clinicians to stretch older ethics codes over new tools. That difference shapes how counselors should document AI decisions.

What each organization says

• ACA: The 2014 ACA Code of Ethics remains the binding standard, but the AI resource hub, updated March 2026, applies it to AI. Counselors should learn AI essentials, evaluate tools before use, stay current, avoid over-reliance, protect the therapeutic relationship, inform clients, and obtain explicit consent. Client-facing use falls under A.2.b, H.2.a. AI is framed as an adjunct, not a replacement; it is not for crisis and not recommended for diagnosis "at this point."
• NASW: The Standards for Technology in Social Work Practice, updated October 2, 2026, require the Code to apply in AI-mediated practice as it does in person. Clinical standards flag privacy, confidentiality, medical-record risks, autonomy, transparency, algorithmic bias, and oversight for chatbots and predictive analytics. A 2026 NASW survey found two-thirds of social workers identify the need for clearer ethical AI guidance.
• APA Code of Ethics: A March 2026 practitioner article recommends including a clear AI-use statement in patient disclosures, and its July 25, 2026 consumer guide advises caution with AI coaching and discussing AI outside sessions. No standalone AI psychotherapy standard is finalized as of October 2026.
• AAMFT: As of October 2026, AAMFT has no standalone AI guidelines. Existing client welfare, confidentiality, informed consent, and competence duties apply.

What the codes have in common

Across all four, the same baseline appears: AI is an adjunct, the clinician retains responsibility, competence must come before use, disclosure and consent are required, and AI should not drive diagnosis or crisis decisions.

Where the codes go quiet

The guidance is thinner on specific AI scribes, client-facing chatbots, and vendor contracts. There, no rule tells you exactly what to click or sign; you are exercising professional judgment.

The ACA answer in plain terms

The ACA does not have an AI-specific amendment to its 2014 Code. Its March 2026 AI resource hub says counselors may use AI as an adjunct if they evaluate it, stay competent, avoid over-reliance, protect the relationship, disclose use, and obtain informed consent. It says AI is not a human-counselor replacement, not for crisis, and not recommended for diagnosis "at this point."

State Laws and Board Rules Are Moving Faster Than the Codes

Which states actually restrict AI in therapy right now, and what do those laws require of you?

Illinois and Nevada: Strictest rules

Illinois' Wellness and Oversight for Psychological Resources Act (WOPR Act), signed August 4, 2025 and effective immediately, bars unlicensed therapy offerings and prohibits AI from making independent therapeutic decisions, directly interacting in therapeutic communication, generating treatment plans without clinician review and approval, or detecting emotions or mental state.¹ Nevada's Assembly Bill 406, effective July 1, 2025, prohibits offering an AI system programmed to provide professional mental or behavioral health care or representing that AI can do so.¹ It blocks therapist-like titles for AI and restricts licensed providers from using AI in direct care, though administrative support like scheduling or records is allowed if the provider independently reviews AI-generated billing and session-note output for accuracy.¹ The law also bars public schools from using AI for school counselor, psychologist, or social worker mental-health functions.¹

Utah's disclosure approach

Utah's House Bill 452, effective May 2025, is narrower and targets mental health chatbots: they must disclose they are AI rather than human, cannot sell identifiable health data, and face advertising restrictions.² It permits chatbot use with guardrails instead of banning AI-assisted mental health interactions outright.²

Board guidance and next steps

Board guidance is thinner. Nevada's statute creates the note-review requirement, but Illinois and Utah have not issued separate board bulletins or rules specifically for AI-generated clinical notes as of this writing. Before adopting any AI note or chatbot tool, check your licensing board's current position and your employer's written policy, re-check periodically.

AI Notes, Transcription, and Billing: Where Documentation Goes Wrong

The appeal of AI documentation is straightforward: session notes shrink from hours to minutes, but the convenience transfers real legal risk onto the clinician, not the vendor. That tradeoff is the core of AI notes, transcription, and billing, especially as insurance changes mental health counselors 2026 put new pressure on reimbursement workflows.

HIPAA Starts With the BAA

Any AI scribe or transcription tool that hears session audio, sees prompts, or stores notes is handling protected health information. Under HIPAA, a vendor that creates, receives, maintains, or transmits PHI on your behalf is a business associate, and you need a signed business associate agreement before any data flows. The BAA should govern permitted uses, safeguards, breach notice, and what happens to data at termination.¹ HHS guidance treats cloud and AI vendors the same: the covered entity or practice remains responsible for the vendor's compliance.¹ Consumer-grade chatbots generally do not offer a BAA and should not be used for clinical documentation.

Audit Data Retention, Training, and Deletion

A BAA is only the start. Check whether the vendor retains session audio, interim transcripts, prompts, metadata, or logs, and whether deletion covers backups and subprocessors.² Ask for a written statement that client data will not be used for model training or fine-tuning unless you give explicit written authorization, since this use is not permitted by default.³ Confirm the vendor's subprocessors also have BAA coverage. Security evidence such as SOC 2 Type II, encryption, and audit logging are reasonable baseline expectations.²

Hallucinations and Billing Errors Are Your Risk

AI notes can invent statements, record the wrong medication, or write risk language that did not occur in session.⁴ Because the clinician signs the note, the review-and-correct step is non-negotiable. The same applies to billing: a note that overstates the session or supports the wrong code can trigger an audit or clawback, and the vendor is not responsible for the clinician's attestation.⁴

Whether clients must be told you are using AI for notes is a consent question covered in the next section, but the documentation workflow itself starts with the BAA and these vendor controls.

Vendor Evaluation Checklist for AI Scribes and Note Tools

Use this yes/no review before any AI documentation tool touches client information, each answer should be verifiable in writing and operationalizes the GRACE Accountability-Centered principle.

1. Signed BAA

   A signed business associate agreement is the baseline HIPAA duty; without it, the vendor has no legal obligation to protect client data.
2. Encryption in transit and at rest

   Notes and recordings must be protected during upload, storage, and backup, both states are required, not optional.
3. Retention period

   Vague or indefinite retention increases exposure if records are later subpoenaed or breached.
4. Training on client data

   If client data trains models, de-identification claims become a liability because outputs may later surface fragments.
5. Deletion on request and at contract end

   Confirms the vendor actually removes client data instead of archiving it in recoverable backups.
6. Audio recording handling

   Raw audio is more identifiable than a transcript; know when it is stored, transcribed, and destroyed.
7. Subprocessor disclosure

   Third-party processing chains can reintroduce risk the primary vendor claims to control.
8. Clinician edit and audit trail

   Every AI-generated note must be editable and traceable to your final clinical judgment, not a black-box draft.

Informed Consent and Client Disclosure for AI Use

Informed consent for AI means telling a client, before the first session where AI is used, exactly how technology will touch their care and giving them a real way to say no. It is not a generic clause buried in intake paperwork.

What to Disclose

• Tool: Name the specific AI product, not just "AI."
• Function: Explain what it does, such as transcribing, suggesting note language, or flagging risk words.
• Data: State what it touches, including session audio, transcript text, diagnoses, or billing codes.
• Access: Say who can see that data, including vendor staff or platform administrators.
• Right to decline: Confirm that refusing AI will not reduce access to care, even amid the behavioral health workforce shortage.

The Opt-Out Workflow

If a client declines, stop the AI feature immediately. Use manual notes, turn off recording or transcription, and document the choice in the record, for example: "Client declined AI-assisted documentation on [date]; manual notes used for this session." Repeat the offer only if the client reopens it rather than pressuring them.

Documenting Use and Errors

Record AI use both in consent paperwork and in the session note, and keep a simple AI concern log with the date, tool, what occurred, and what was corrected. Log errors, hallucinations, or client complaints even when no harm is obvious, this is the kind of record that can protect you in a counselor licensure board hearing.

When a Client Brings Chatbot Advice

Explore it with curiosity through a trauma informed care lens: ask what the chatbot said, compare it to the treatment plan, and share your clinical view. Do not dismiss it reflexively, and do not endorse it just because it sounds plausible.

Plain-language disclosure sample: "I use a note-taking tool that may transcribe our session to help me draft my progress note. It stores session audio and transcript text with the vendor, and only I and the vendor's authorized support staff can access it. You can decline, and I will take manual notes instead. Declining will not affect your care."

Counselor Liability When AI Gets It Wrong

Under NBCC's 2026 ethical principles, artificial intelligence may recommend, but "the counselor bears responsibility for clinical decisions and is accountable for client outcomes."¹ That makes the licensed clinician the default liability holder when a scribe, risk score, or draft note is wrong.

Who is responsible for AI mistakes

In counseling practice, AI generally does not shift responsibility to the software vendor or the tool. The licensed professional remains responsible for reviewing, correcting, and authorizing the final record. If a counselor relies on an automated risk score or AI-generated treatment plan without independent clinical judgment, that is the counselor's clinical decision, not the machine's. California Senate Bill 903 would extend this duty to psychotherapy, triage, and screening, and make the professional responsible for AI use by anyone under supervision. Supervisors and practicum sites therefore need clear policies , especially in supervision of mft trainees , because a supervisee's AI error can become a supervisor's exposure as well.²

Insurance and legal uncertainty

Malpractice carriers have not established a uniform national standard for AI-assisted documentation. Legal Issues for Therapists in 2026 warns of an "AI coverage gap" and advises counselors to get written confirmation that AI-assisted documentation is covered under their policy. Coverage approval is also separate from whether your use of the tool meets the standard of care. There is little case law yet, and licensing board discipline tied to AI is still emerging. Nothing in the sources establishes a universal appellate decision on therapist AI reliance; ordinary professional negligence theories would likely apply.

Two steps to reduce risk

• Ask your carrier in writing: Confirm whether AI-assisted notes, transcription, or risk tools are covered before you rely on them.
• Document your review: Keep a record showing you reviewed, corrected, and authorized the AI output, especially for treatment plans or risk-related content.

High-Stakes Populations: Where AI Cannot Replace Clinical Judgment

Who Is Most Exposed to Bad AI Output

Children, trauma survivors, justice-involved clients, and people in crisis are among the worst served by automated scores and incomplete data. These are not edge cases in counseling; they are routine forensic and clinical encounters where risk decisions carry legal weight. In these high-stakes cases, a tool's error can become a legal or clinical decision. Reviews of juvenile risk and needs assessments have found uneven predictive accuracy across populations, measurement bias, and a continuing risk of disproportionate minority contact , a cultural competence in counseling concern.¹ Criminal justice risk tools carry fairness trade-offs and can overestimate risk for some groups, including women.² Some juvenile recidivism models that improved overall accuracy also introduced group-fairness problems, so a better average score is not automatically a fairer one for the client in front of you.

Why a Wrong Score Follows the Client

A risk score or hallucinated summary that enters the record does not stay in the session. It may be read later in a court filing, a custody review, a reentry plan, or a crisis triage note. Because automated text can look objective, the original error can survive long after the clinician has corrected course. In forensic psychology, that is a liability problem, not just a documentation annoyance. A poorly validated tool used in the wrong population adds another layer of exposure, because error rates can differ by subgroup and risk level.⁴

A Working Rule for Clinicians

AI output may prompt a question, but it should never be the basis for a risk determination. Use a flagged score to trigger a fuller assessment, not to close one. When clinical judgment conflicts with the tool, override it and document what the tool said, what you observed, and why you departed. That documentation is the standard-of-care record that protects both the client and the counselor.

Related Articles

Can AI Replace Counselors? What the Evidence Says About AI Therapy vs. Human Therapy

The question of whether an AI chatbot can replace a licensed therapist has moved from speculation into controlled research, and the early evidence draws a clear line: not now, and not for the relational core of therapy.

Where the Evidence Is Encouraging

In a pilot randomized trial comparing AI-delivered and human-delivered internet CBT for depression in young adults, both reduced symptoms early, with no significant difference at week 2. But the AI condition plateaued after week 2, while human-delivered care continued improving through week 4. Recent meta-analyses show a similar pattern: CBT-oriented chatbots produce moderate reductions in depressive symptoms and small reductions in anxiety after intervention, though confidence intervals are wide and effects are mostly short-term. One 2026 study focused on college student mental health found a conversational AI agent improved anxiety, depression, well-being, and life satisfaction more than group therapy or a waitlist, but that result does not generalize to all clinical populations.¹

Why AI Alone Is Not a Therapist

Safety concerns separate purpose-built therapy chatbots from general-purpose tools. Reviews note that human-led therapy remains superior for deeper emotional engagement and clinical impact, with some data showing larger improvements with human-certified psychologists. AI companion chatbots have been associated with addiction-like attachment, worsening symptoms, and self-harm-related cases, though causal links are unclear.² General-purpose chatbots prompted as CBT therapists are not crisis-safe by default, especially for clients with suicidal ideation.

The Model That Has the Best Support

Three models matter in practice: AI alone, human alone, and human-led hybrid. Evidence currently favors human-led hybrid for symptom management and between-session support, while human therapy remains the standard for relational depth, crisis response, and clinical accountability.