Episode summary
The Black Box Dilemma: Who Controls What AI Can Say? With CTGT’s Founder, Cyril Gorlla
Money Maze Podcast
· · Guests: Cyril Gorlla, Kristian West
Summary
CTGT’s founder argues that the core problem with today’s large language models is not just that they can be wrong, but that they are probabilistic systems operating in high-risk environments where banks, insurers, healthcare firms and governments need repeatable, auditable decisions. He says the key failure modes are hallucination, bias, and the inability to explain why a model produced a given answer, especially when one bad output can cascade through regulated workflows.
He contrasts simple guardrails with deeper model-level oversight. In his view, binary filters are too brittle for complex enterprise use, because models can route around constraints, omit important facts, or appear compliant while missing the real intent. The guest argues that organizations need independent evaluation, behavioral verification, and ways to inspect models directly, not just judge outputs after the fact.
The conversation also covers geopolitical competition and the growing split between U.S. and Chinese AI systems. The guest says censorship and policy shaping are already visible inside models, and that “open auditable instruments” will matter more than hype or race rhetoric. He also argues intelligence is becoming a commodity for many routine tasks, while the real value will shift to context, verification, and the ability to scale institutional judgment.
Key takeaways
- The guest says AI mistakes matter most when they create cascading downstream errors in regulated workflows, not just when a single sentence is wrong.
- He argues that guardrails are too rigid for many enterprise use cases and should be replaced or supplemented by remediation and behavioral verification.
- He says models should be inspected directly, because prompt-based filtering cannot enumerate all bad behavior or prove why a loan, email, or report was produced.
- He sees a real geopolitical divide in AI, with censorship and evaluation criteria already shaping model behavior in both China and the West.
- He thinks intelligence is becoming a commodity for routine synthesis tasks, while context, policy encoding, and independent oversight become the durable advantage.
Notable quotes
“The natural nature of LM to please the user may result in it giving you an answer that's not grounded in reality”
“they're often pretty rigid. They're binary, right? Like is there hate speech in this response? Um is there violent content in this response”
“the model, regardless of how you prompt it, it might not always match your intent”
“I think there is a very real problem in terms of that knowledge transfer from the junior to the senior roles”
How the episode opens
We fundamentally explore uh how AI models function in very high-risisk environments. Uh and so typically what that looks like is financial services, healthcare, insurance, etc. Places where getting the answer right matters a lot with just the things that have happened in in the past few weeks with the hugging face incident, um the near miss incident with with China from a hallucinated AI inventory.
The natural nature of LM to please the user may result in it giving you an answer that's not grounded in reality. This is one of the most fascinating and fast evolving um spaces of how AI is becoming the new diplomacy. What does a delearnized zone look like in the future between China and the US?
It probably looks like the intersection of how our AI models interact with their AI models, right? That'll be how uh agreements are being made. That'll be how like policy um is sort of interjected, critiqued, etc.
from both sides. If you are a San Francisco based tech startup profiled in the Wall Street Journal, Forbes and Business Insider asked to present at JP Morgan's Investor Summit alongside Jamie Diamond, backed by Google's AI fund and with a single mission to help organizations detect and correct biased or unreliable outputs from AI models and you're only 24. That is something worth exploring.
So today, the Moneym Podcast meets its youngest guest, Sir Gola, founder of CTGT. Sirill, welcome to the Moneym Podcast.
Thanks, Simon. Uh, great to see you and and Christian as well. — Well, because I am trying hard not to expose publicly my chasms of ignorance uh which define me. I have, if you'll excuse the pun, a co-pilot, JP Morgan Asset Management's head of the investment platform, Christian West. Christian, welcome back to the Money Podcast.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- llm non-determinism
- ai guardrails
- model vs output inspection
- hugging face incident
- us-china ai safety
Companies and tickers mentioned
- JPM
- GOOGL
- GOOG
- CTGT
- J.P. Morgan Asset Management
- JPMorgan
- Hugging Face
- Anthropic
Watch the episode
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Money Maze Podcast
-
Why, Where and How? CPP Investments' Heather Tobin on Building Large Hedge Fund Portfolios -
Fraud, Fees & Fake Degrees! Where's the Alternatives Allocation Headed? John Claisse, CEO, Albourne -
Come Fly with Me! Delta Air Lines, The World’s Biggest Airline - With CEO, Ed Bastian. -
AI and the Investment Edge: Enhancement, Replacement, or Reinvention? -
A New Era: Richard Oldfield on the Future of Schroders and the Asset Management industry -
The Golden Age of Absolute Returns? With Davidson Kempner’s Managing Partner and CIO, Tony Yosel...
Money Maze Podcast, this episode and its artwork belong to Money Maze Podcast. Octagon is not affiliated with the show; this page is an independent summary.
