Safe & Secure AI at Innovation Week Okanagon College
Gurinder Singh Mann
3 min read
Most organizations are being told to put AI in everything. Very few have decided what they would actually let it do without a human watching. That gap was the starting point for Safe & Secure AI, a session we delivered to IT staff at Okanagan College.
We built the talk for people who have to make adoption decisions and want to make them well.
Knowing what you're actually deploying
We walked the spectrum, because the word "AI" hides the thing that matters: how much autonomy you're handing over.
| What it is | What it does | What you're giving up |
|---|---|---|
| Generative AI | Produces output you review | Very little, if you read the output |
| Predictive AI | Traditional ML, using historical inputs to measure trends | Interpretability, if the model is opaque |
| Single-agent AI | Holds context, reasons, plans, adapts, executes tasks | Direct control over individual actions |
| Multi-agent AI | Agents coordinating with each other | Meaningful human oversight of the sequence |
Once an agent can take actions, the question shifts from "is the answer good" to "what can this thing reach." We also spent time on MCP, which is quietly becoming the connective tissue between agents and everything else in the environment. Its security posture is somewhere between early and improvised. If you are wiring MCP servers into internal systems, you are extending your attack surface into a protocol ecosystem that has yet to be hardened.
Run this as a coverage check
We went through the risks worth testing your own environment against. The goal is to find the one you haven't covered:
- Sensitive information exposure. PII, student records, research data. Education institutions are attractive targets and generous data holders.
- Supply chain. Open-source components, vendor data flows, where the data actually lands, and how your own team develops. A vendor scorecard that ignores data residency has a hole in it.
- Excessive agency and identity. The one that gets underestimated. Least privilege, blast radius, JIT/JEA access, zero trust boundaries around the agent. An agent inherits every permission you gave it, and it will use all of them.
- Misinformation and output trust. Public trust in what a system tells you is declining. Your controls have to earn that trust back.
The four places AI programs actually break
Map: you deployed more than you needed. Frontier models are the default answer to questions that don't require them. A small language model handles roughly 80% of what active agents get asked to do, at a fraction of the cost and with a far smaller blast radius. The EU AI Act's tiers (unacceptable, high, limited, minimal) work as a triage tool even for organizations it doesn't bind. Add non-determinism, edge devices reaching internal systems, digital sovereignty, and cost, and "one model to rule them all" stops looking like a strategy.
Measure: you can't see what it's doing. No AI inventory. Monitoring added after go-live. An orchestrator chosen before anyone defined the workload. No technical feedback loops. TEVV (testing, evaluation, verification, validation) treated as a launch gate rather than a continuous practice, alongside bias and fairness evaluation and risk tracked over time.
Govern: the human element. Trustworthy AI comes down to human agency and oversight, robustness and safety, privacy and data governance, transparency, fairness, societal and environmental wellbeing, and accountability. In practice that means deciding, explicitly, whether a human is in the loop, on the loop, or in command, and naming who is accountable when the answer changes. Culture, policies, AI inventory, supply-chain and stakeholder engagement cut across all of it.
Manage: no plan for when it's wrong. Assume it will be. Does your incident response plan mention AI systems? Does your insurance? Who monitors the third parties in the chain, and how do you prioritize and treat the risks once you find them?
What we'd want you to take away
Adoption discipline is the differentiator. Decide what the system is allowed to reach, prove you can see what it did, name who is accountable, and rehearse the day it gets something wrong. That work is the same work as building well, and it has to happen before the agent has credentials.