If a hospital or a pharmacy customer about to close or renew sent you an AI security audit tomorrow, how confident are you in your answers? And if the deal stalls, are you prepared to explain this to your CEO?
Hospitals have moved past the basics. AI policies and training completion rates aren't going to cover you anymore. They're auditing how well you are protecting PHI from leaking into AI tools and this includes having visibility into things you probably haven't gotten to yet: which employees are using personal ChatGPT accounts for work, who's connected an AI tool to your customer data, which of your SaaS vendors quietly turned on embedded AI last quarter, and whether you can produce evidence for any of it on request.
TLDR: What healthtech CISOs need to know about shadow AI
To qualify for a BAA, your healthcare customers are quickly expecting more concrete answers around how your entire org uses AI. With shadow AI usage that ranges from using non-enterprise tiers to using any SaaS tool with AI functionality, you need to move past the idea that procuring enterprise licenses for a couple of AI tools will suffice. What's now required is AI visibility and policy enforcement across all of your workforce's devices (including personal devices) and interfaces, understanding not only what tools they use, but what plans, models, connectors, access, and prompts are being used. All of this needs to happen in realtime, in order to catch and prevent any data leakage.
Why you need to pay attention to shadow AI today (or yesterday, really)
The shadow AI conversation in every other industry is challenging, but the unique mix of increased regulation and tech affinity amplifies each risk in ways you may not have realized.
Your engineers are the highest-risk users. Verizon's 2026 DBIR analyzed over 850,000 AI-related data loss events. The largest category of data leaked, by a wide margin, was source code. For healthtech that's a double exposure. Your codebase touches PHI structures: FHIR mappings, EHR integration logic, claims processing, patient-matching algorithms. An engineer pasting a customer screenshot into Copilot's free tier or asking Claude to find a bug based on a log file can simultaneously leak the IP that took years to build, the schemas that show how PHI flows through your system, and real patient data sitting in a debug log.
Training isn’t the issue here. Your engineers aren't pasting PHI into ChatGPT because they're reckless. They're doing it because their job is hard and AI makes it easier. Until you give them something faster that's also approved for PHI, they'll keep finding their own way.
Coding assistants live inside the IDE. Copilot, Cursor, Windsurf, Codeium. They watch every keystroke. They may be using Cursor safely, but an added tool to test code for bugs may be a personal plan they added themselves without anyone knowing. Not because they’re reckless, but because otherwise, they will be falling behind.
The wrong-tier trap is everywhere. ChatGPT Free, Plus, Pro, and Team are not BAA-eligible. OpenAI says so in all caps in its own terms. Claude, Gemini, and Copilot all have the same split: consumer tiers can't legally process PHI. Most of your employees do not know this - and this distinction exists for every tool, across desktop mobile, browser, and basically every SaaS tool that they’re using. Even free tier SaaS tools that seem harmless can be leveraging AI in such a way that PII is being exposed.
The AI you didn't buy is the hardest to govern. Notion AI summarizes documents. Slack AI indexes channels. Zoom AI Companion transcribes calls. Salesforce Einstein writes follow-ups. Atlassian Intelligence searches Jira tickets. These features were turned on by default, governed by terms that don't match your BAAs, adopted before security got a chance to review them. Your AUP almost certainly doesn't name the AI now living inside dozens of other tools you already pay for.
Hospital CISOs aren't just asking anymore. HITRUST released a formal AI Security Assessment with Certification in November 2024 and The Health Sector Coordinating Council published a third-party AI risk guide in 2025. AI questions are now in vendor security questionnaires, RFPs, and annual reviews. The changes and demands are only accelerating and your customers are expecting you to not only be on top of it, but ahead of the curve.
What to actually do about shadow AI in healthtech
Forget banning AI. Samsung tried it back in 2023, before AI usage truly boomed, and it was an utter failure. On the other hand, healthcare organizations that provided sanctioned alternatives saw an 89% drop in unauthorized AI use. So the answer is governance and visibility, not prohibition.
Honest discovery. Every security vendor's playbook tells you to run a CASB sweep, pull proxy logs, and call it an inventory. It isn't. Your engineers route AI calls through personal accounts on personal phones when they're frustrated. They use AI inside their IDE and through command line execution, which never touches your egress. Real discovery runs across endpoint telemetry, IDE-level monitoring on engineering laptops, browser extension inventories, token-level visibility into provisioned AI tools, and SaaS posture monitoring for embedded AI. Microsoft's 2024 Work Trend Index found 52% of AI users won't admit to using it for their most important tasks.
Give people a safe tool for their needs. ChatGPT or Claude is not enough. On top of purchasing at least one BAA-eligible enterprise tier for ChatGPT Enterprise, Claude Enterprise, or Copilot, you’ll need at least a few other basics per team. For engineering, provision an enterprise-tier coding assistant with training opt-out confirmed in writing. Announce what's approved before you publish what's restricted.
Start with identity. SSO, MFA, role-based access. Every AI control sits on top of these. The moment AI agents start acting on behalf of users, and they will soon, every audit log needs to answer who authorized the action and which model executed it.
Write the AUP last, not first. Name the data classes that never enter any AI tool. Name the AI categories explicitly: chatbots, coding assistants, embedded AI in SaaS, browser extensions, voice and transcription. Don’t fall into the trap of spending months creating an AI policy only to find out that it’s completely irrelevant to how your employees are already exposed to AI.
Treat AI vendors like every other third party. Are they training on inputs? Sub-processors? Data residency? BAA on file, for which specific tier? Zero data retention configurable? If you can't answer those for every AI tool your company uses, you can't answer them for your hospital customers either.
Re-evaluate your existing SaaS tools as if they are new AI tools. Don’t assume that the existing evaluation you did on these tools years ago is applicable to the newly added AI components they have. Ask all the relevant questions again, as you may get very different answers this time around.
Get deep, absolute, and ongoing visibility into AI usage. It’s not enough to just see their surface level usage or prompt logs. You need deep visibility into each tool - who is using it, their prompt logs, which plan or model is being used, tools being connected, files being shared, and skills being used.
And with every SaaS tool now doubling as AI and hundreds of new tools popping up or updating daily Copilot, Claude, ChatGPT, and Cursor. You’ll also need to monitor a broader spectrum of tools and extensions. Your team is already using a wide range of AI tools. From voice assistants on their mobiles to AI within your email platforms and CRM, you may think you’re blocking AI usage, but unsanctioned tools are being used across devices, desktop and browser, by remote employees and contractors alike.
This is not a one-time audit either. AI is moving fast. You need real-time and ongoing visibility that covers both depth and breadth.
Tell your customers. Send a one-page AI governance summary to your top hospital BAA counterparts. Most vendors won't, because they're afraid of starting a conversation they can't finish. Do it anyway.
The three AI questions healthtech CISOs must be able to answer
When your next renewal call comes, the hospital CISO is going to ask some version of three questions. You can probably answer the structural ones like your policy, your tools, and your BAAs. The operational ones are where vendors get exposed.
- Tell us about an AI incident your team has handled in the last 12 months.
- How would you know, right now, if one of your engineers was using a personal ChatGPT account on their work laptop?
- Show us your AI tool inventory and the BAA status of every tier you use, including embedded AI in your SaaS stack.
"We haven't had any incidents" doesn't mean you haven't had any. It means you can't see them. The healthcare CISO already knows the difference.
.png)
