Ethics in AI Coaching: A Practical Guide to the Risks and Rules

An AI coach is always awake, never tired, and remarkably persuasive. That combination is exactly why the ethics of AI coaching matter.

The question used to be whether software could help you change a habit. It can. The harder question is the one we should be asking now: should a system that knows your moods, your slip-ups, and your soft spots be allowed to nudge you — and who keeps it honest while it does?

This is a practical guide to the ethics of AI coaching: where the real risks sit, what the law and the professional bodies now require, and the handful of questions worth asking before you trust any AI tool with your goals or your private life.

Why ethics weighs more heavily here

Most software is transactional. Coaching is intimate. To help you build a habit or break one, an AI tool gathers a remarkably detailed picture of you — when you struggle, what you fear, what motivates you. That picture is what makes the coaching useful, and it is also what makes the stakes high. The same model that can gently hold you accountable could, with a different objective function, quietly steer you toward whatever keeps you engaged.

Two things follow. First, the data is sensitive in a way that step counts and shopping carts are not. Second, the influence is real — behaviour change is the whole point — so the line between helping you decide and deciding for you has to be drawn carefully. Most of the ethics of AI coaching comes down to defending that line.

The principles a trustworthy AI coach should follow

You do not have to invent these from scratch. The EU’s Ethics Guidelines for Trustworthy AI (2019) set out seven requirements that map neatly onto coaching: human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal well-being; and accountability.¹ Underneath them sit four ethical principles — respect for human autonomy, prevention of harm, fairness, and explicability.

For a coaching context specifically, the International Coaching Federation’s AI Coaching Framework and Standards (published November 2024) is the most relevant professional reference.² It covers six areas — foundational ethics, co-creating the relationship, communication, learning and growth, assurance and testing, and technical factors like privacy and accessibility — and frames AI as a complement to human coaching, not a replacement for it.

Distilled, four principles do most of the work:

• Autonomy. The tool advises; you decide. An ethical coach suggests a better time to train and waits for your yes — it does not quietly book your calendar.
• Transparency. You should be able to tell what the system is, what data it uses, and where its competence ends.
• Fairness. Guidance should be tailored to your goals, never skewed by your race, gender, or income.
• Accountability. Someone is responsible when it gets things wrong — and there is a way to find out that it did.

Influence versus manipulation

This is the heart of it. Coaching is influence by design; the ethical question is whether the influence serves you or the company behind the app.

The risk is not hypothetical. In a controlled study published in PNAS, researchers trained AI “adversaries” against human participants in lab decision-making tasks. In one task the AI steered people toward a target choice with roughly a 70% success rate; in another it pushed up participants’ error rate by about 25%.³ These were artificial laboratory games, not coaching apps — but they show that a system with enough behavioural data can learn to exploit the predictable ways humans decide.

Regulators have noticed. The EU AI Act, in Article 5, now prohibits AI systems that use “subliminal,” “purposefully manipulative or deceptive techniques” to materially distort someone’s behaviour in a way likely to cause significant harm — and bars exploiting vulnerabilities tied to age, disability, or economic situation.⁴ It is worth being precise about what that does and does not mean: it is not a ban on persuasion. Transparent, honest nudges are fine. What is prohibited is covert manipulation that harms — and breaching it sits in the law’s top penalty tier (up to €35 million or 7% of global turnover).

The practical test for any AI coach is simple: does it optimise for your stated goal, or for your screen time? A tool built to keep you engaged and a tool built to help you not need it as much will, over time, behave very differently.

Bias: the same advice is not fair to everyone

AI learns from data, and data carries the assumptions of the world that produced it. The cleanest evidence comes from healthcare: a 2019 study in Science found that a widely used algorithm — one affecting around 200 million people a year in the US — systematically underestimated the needs of Black patients, because it used past healthcare spending as a stand-in for health need. Correcting the flaw would have raised the share of Black patients flagged for extra care from 17.7% to 46.5%.⁵

The lesson transfers directly to coaching. If a habit-formation model is trained mostly on one demographic, its idea of “normal” progress, “realistic” goals, or “healthy” routines may quietly misfit everyone else. Guarding against this means diverse training data, regular bias testing, and human review — and resisting the temptation to treat a single average user as the template for all of them.

One cautionary note on a popular example: the 2019 Apple Card gender-bias story is often cited as proof of AI discrimination, but the New York Department of Financial Services investigated and found no unlawful discrimination after reviewing roughly 400,000 applicants.⁶ The honest takeaway is not “AI is always biased” but “bias is real, hard to see, and has to be tested for — not assumed in either direction.”

Privacy: who actually sees your private life

An AI coach can end up holding some of the most sensitive information you own — your emotional state, your setbacks, the goals you have not told anyone else. Under the EU’s GDPR, data about health and mental health is “special category” data, processed only under strict conditions, and the penalties for mishandling it reach €20 million or 4% of global turnover.⁷ California’s CCPA gives residents the right to limit how their sensitive personal information is used.⁸

A common misconception is worth correcting: HIPAA, the US health-privacy law, generally does not cover a direct-to-consumer wellness or coaching app. As the FTC itself states, “many companies that collect people’s health information … aren’t covered by HIPAA.”⁹ Instead, the FTC enforces against deceptive data-sharing — and it has. In 2023 it ordered GoodRx to pay $1.5 million for sharing health data with advertisers,¹⁰ and required the therapy platform BetterHelp to pay $7.8 million and stop sharing mental-health questionnaire data with Facebook and others for advertising.¹¹

So the questions to ask are concrete: Is my data encrypted? Is it ever sold or shared with advertisers? Can I delete all of it? Can I have a private session that is not stored at all? Good answers to those are not a luxury — increasingly, they are the law.

Over-reliance: a coach should make you less dependent, not more

A subtler risk is that a tool which always has the answer can erode the very capability it is meant to build. A 2025 study in Societies of 666 participants found a significant association between frequent AI-tool use and lower critical-thinking scores, with “cognitive offloading” — letting the tool think for you — as the likely mechanism.¹² The finding is correlational, not proof of cause, but the direction is intuitive: lean on something for every decision and your own decision muscle weakens.

For coaching, this is the whole game. The point of building a good habit is to reach the day you no longer need reminding. An ethical AI coach should be designed to fade — to hand capability back to you — not to make itself indispensable. If a tool seems engineered to keep you returning rather than to help you outgrow it, that is a values choice, and not a neutral one.

An AI coach is not a therapist — and should say so

The most important boundary is the one between coaching and clinical care — a line worth understanding in its own right (we explore it in whether AI can replace therapists). In November 2025 the American Psychological Association issued a health advisory warning that generative AI chatbots “were not created to deliver mental health care” and should not be used as a substitute for a qualified professional.¹³ The risk is concrete: in 2023, the eating-disorder charity NEDA pulled its chatbot “Tessa” offline after it gave actively harmful dieting advice to users seeking support — a vivid example of what happens when a wellness tool drifts into clinical territory it was never built for.¹⁴

This is also why transparency about what you are talking to matters. The EU AI Act’s Article 50 will require that people are told when they are interacting with an AI system rather than a human.¹⁵ A trustworthy AI coach states plainly what it is, names its limits, and points you toward a human professional — or, in a crisis, toward emergency help — rather than pretending to be something it is not.

What ethical AI coaching looks like in practice

Principles only matter when they show up in how a tool is built. At aidx.ai — an award-winning AI coaching and therapy service over chat and voice — those choices are deliberate. Conversations are fully encrypted in transit and at rest, with decryption kept isolated from storage; no human reads them; data is never sold or shared; and you can delete everything at any time. An Incognito toggle lets you turn off persistent storage for a conversation when you want a session that simply is not kept.

The coaching itself draws on established, evidence-based methods — CBT, ACT, DBT, and NLP — and adapts to the individual through a proprietary AI system (ATI) rather than nudging everyone the same way. And the boundary is held honestly: aidx.ai is built to support people under everyday strain and to help them build their own momentum, not to replace a human clinician or to handle acute crisis. The aim is a coach you eventually lean on less, not one engineered to keep you hooked.

If you are weighing up any AI tool for your goals — ours or anyone’s — the same short checklist applies: Does it advise or override? Can you see what it is and what it knows? Is your data protected and deletable? And is it built to help you grow past it, or just to keep you coming back?

The bottom line

AI coaching can be genuinely good for people — patient, available, and honest in a way that is hard to find at 2am. But “can be” is doing real work in that sentence. The technology is only as ethical as the choices baked into it: whether it guards your autonomy, protects your data, treats you fairly, knows its limits, and is built to serve your goals rather than its own engagement metrics. Ask those questions out loud. A tool worth trusting will have clear answers — and will not mind you asking.

Last reviewed: June 2026. This article is general information about the ethics of AI coaching, not legal or medical advice. If you are dealing with a mental-health condition, speak with a qualified professional; if you are in crisis, contact your local emergency services or a crisis line.

References

1. European Commission, High-Level Expert Group on AI. Ethics Guidelines for Trustworthy AI (2019).
2. International Coaching Federation. ICF Artificial Intelligence (AI) Coaching Framework and Standards (November 2024).
3. Dezfouli A, Nock R, Dayan P. Adversarial vulnerabilities of human decision-making. PNAS 117(46):29221–29228 (2020).
4. European Union. AI Act, Article 5 — Prohibited AI Practices (Regulation 2024/1689).
5. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366(6464):447–453 (2019).
6. New York State Department of Financial Services. Report on the Apple Card Investigation (March 2021).
7. European Union. GDPR Article 9 (special category data) and Article 83 (penalties).
8. California Office of the Attorney General. California Consumer Privacy Act (CCPA).
9. US Federal Trade Commission. Collecting, Using, or Sharing Consumer Health Information? Look to HIPAA, the FTC Act, and the Health Breach Notification Rule.
10. US Federal Trade Commission. FTC Enforcement Action to Bar GoodRx from Sharing Consumers’ Sensitive Health Info for Advertising (February 2023).
11. US Federal Trade Commission. FTC Final Order Banning BetterHelp from Sharing Sensitive Health Data for Advertising (July 2023).
12. Gerlich M. AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies 15(1):6 (2025).
13. American Psychological Association. Health Advisory on the Use of Generative AI Chatbots and Wellness Applications for Mental Health (November 2025).
14. NPR. An eating disorders chatbot offered dieting advice, raising fears about AI in health (2023).
15. European Union. AI Act, Article 50 — Transparency Obligations (Regulation 2024/1689).