Your AI "Assistant" Has Access to Your Calendar, Email, and Files. Have You Actually Checked What It's Doing With That?

Somewhere in the last year, "AI assistant" quietly stopped meaning a chat window you type questions into. Now it means something that's been given standing permission to read your inbox, see your calendar, poke around your cloud drive, and sometimes take actions on your behalf without asking first. I've connected a handful of these tools to my own accounts over the past few months, and only recently did I sit down and actually look at what permissions I'd handed out. It was more than I remembered agreeing to.

This isn't a "delete all your AI tools" post. I still use several of them daily and they save me real time. But there's a gap between what these assistants can technically do and what most people realize they've authorized, and that gap is where things go wrong quietly rather than loudly.

Photo by Anna Shvets on Pexels

What "connected" actually means

When you link an AI assistant to Gmail, Google Calendar, Notion, or a file storage account, you're usually granting an OAuth token with a scope — a defined set of permissions like "read your calendar events" or "read and modify your email." The problem is that a lot of these integrations ask for broader scopes than the feature you wanted actually needs.

Say you connect an assistant so it can summarize your unread emails each morning. That's a read-only task. But plenty of these tools request read *and* write access by default, because it's easier to build one integration than two. Most people click "allow" without checking, because the alternative is reading a permissions dialog that looks like legal boilerplate.

I'm not saying every tool is doing something shady with that extra access. Most aren't. But "most aren't" is a bet, not a fact you've verified, and the honest answer for most of us is that we have no idea what's actually being read or stored.

The audit that takes ten minutes

Here's the thing that actually helped me: you don't need to distrust every tool to make this useful. You just need to look, once, at what's connected.

• Go to your Google Account's "Third-party apps with account access" page and read the list. You will probably not recognize a couple of names.
• Do the same for Microsoft if you use Outlook or OneDrive — it's under app permissions in your Microsoft account settings.
• Check your Slack or Notion workspace's connected apps list if you use AI tools tied to those.
• For each one, ask: do I still use this, and does the permission level match what I actually need it for?

I found two integrations I'd forgotten I set up — one from a productivity app I tried for about a week last spring and never uninstalled the connection for. It still had calendar read access. Nothing bad had happened, as far as I could tell, but "as far as I could tell" is exactly the problem. Revoking access I'm not using isn't paranoia, it's just closing doors I left open.

Read-only isn't nothing, but it's not the whole risk

A lot of privacy advice stops at "check if it's read-only." That's a fine first filter, but it undersells the actual risk with AI assistants specifically, because of what they do with what they read.

If an assistant reads your calendar to summarize your week, that's low-stakes. If it reads your email to draft replies, it's now processing the actual content of your conversations — names, numbers, whatever's in there — through whatever model is powering it. Most reputable tools state clearly whether your data is used to train models or just processed per-request and discarded. That distinction matters more than read vs. write, and it's usually a sentence or two you can find in the privacy policy if you search for "training" or "model improvement."

I'll say the unpopular part out loud: I think most people should read that one sentence before connecting anything to their inbox, and almost nobody does, including me until recently. It takes about ninety seconds and it's the single highest-value ninety seconds you can spend on this.

A worked example: the scheduling assistant

Photo by Thirdman on Pexels

Let's walk through a common one. Say you set up an AI scheduling tool that reads your calendar, finds open slots, and can send meeting invites on your behalf. That's genuinely useful — it's the kind of thing that saves you real back-and-forth every week.

But look at what that requires: read access to every event on your calendar (including ones with sensitive titles or attached notes), and write access to send invites as you. If that tool has a bug, or its own account gets compromised, someone else could see your full calendar or send emails that look like they're from you. That's not a hypothetical about the AI being malicious — it's a hypothetical about the AI being a normal piece of software with normal software bugs, sitting on top of your identity.

The fix isn't avoiding the tool. It's checking, once a quarter or so, whether it's still doing only the job you gave it, and whether the account behind it (yours, and the vendor's) has reasonable security — two-factor auth on your side, a track record of no major breaches on theirs.

The pattern I keep coming back to

I've written before about how the boring, well-integrated tool usually beats the flashy new one, and this is the privacy version of that same idea. The tools worth keeping long-term are the ones with clear, narrow permission requests and a plain-English privacy policy. The ones that ask for everything "just in case" are usually the ones you'll forget you connected, and forgetting is exactly how stale access sits around for a year doing nothing useful and everything risky.

None of this requires becoming a security expert. It requires doing the ten-minute audit once, and then actually disconnecting the things you don't use instead of leaving them there out of inertia.

FAQ

Is it safe to connect AI assistants to my email at all?

Generally yes, for reputable tools with clear privacy policies, but "safe" isn't binary — it depends on what scope you grant and how careful you are about which tools you trust. Read-only access for summarizing is lower risk than granting send/delete permissions. When in doubt, start with the narrowest access the tool allows and expand only if you actually need more.

How do I know if an AI tool is training on my data?

Check the tool's privacy policy for language about "model training," "model improvement," or "opt out." Most consumer AI products now offer a toggle to disable this, often in account or privacy settings rather than buried in a separate document. If you can't find a clear answer within a couple of minutes of searching, that's itself useful information.

What's the single best habit for managing this long-term?

Do a quarterly check of connected third-party apps on your major accounts (Google, Microsoft, Slack, whatever you use). It takes less time than a coffee break and catches the "installed it once, forgot about it" access before it becomes a real exposure.

Worth the ten minutes

None of this is about becoming suspicious of AI tools generally — I use several every day and I'm not stopping. It's about matching your actual permission grants to your actual usage, instead of running on whatever the default setup asked for. The tools that deserve to stay connected are the ones whose access still makes sense when you look at it directly. Go look.

Keep reading

• Your Phone and Laptop Are Basically Strangers — Here's How to Actually Sync Them
• The 20-Minute Weekly Reset That Keeps a Productivity System From Rotting
• Three Things Your Phone's Backup Doesn't Actually Save

#aitools #productivity #privacy #digitalassistants