Rajen Madan, Co-Founder and CEO, governr

Today we're delighted to speak with Rajen Madan, Co-Founder and CEO of governr, a company building AI operating and risk systems for highly regulated industries. As financial institutions rapidly deploy AI agents that can execute workflows and make decisions, Rajen argues that the critical question is shifting from "what can AI do?" to "how much are we prepared to let AI run?" - a fundamental risk management challenge that requires the same quantitative rigour banks apply to credit, market and operational risk.

My questions are in bold - over to you Rajen:

Who are you and what's your background?

I am the founder and CEO of governr, the risk management system for AI in regulated enterprises and an engineer by training. My early career spanned the designing of cars through to building critical risk and trading systems for global markets. I have worked in AI since 2017, founding a specialist technology practice at Leading Point that deploys data infrastructure, ML and AI for global banks, insurers, DLT and fintech firms. I am also an alumnus of the LSE and DCE and a mentor at Barclays Fintech and Tech London Advocates ecosystem.

I have spent most of my career at the intersection of global markets, quantitative risk, AI and the building of mission critical trading and risk systems in highly regulated environments.

What has stayed constant throughout my career is helping C-suite with business risk taking and managing it. Financial institutions have spent decades developing sophisticated ways of deciding how much risk they are prepared to take in credit, markets, liquidity and operations. AI is now entering those same institutions extraordinarily quickly, but the risk infrastructure around it is much less mature.

Financial institutions today need to be asking, 'how much are we prepared to let AI run?' That is ultimately a risk-management question, and I think it will become one of the defining questions for financial services, healthcare, defence and other regulated sectors over the next decade.

What is your job title and what are your general responsibilities?

I'm the CEO and co-founder of governr.

My role sits primarily around the problem we are solving which means working with regulated enterprises, AI innovators, risk leaders and AI teams to understand where AI is being deployed, what is preventing them from giving it more responsibility, and what risk management and control infrastructure they need as delegation increases.

I spend a lot of time with clients and investors because this market is developing extremely quickly. Six months ago, many conversations were about copilots and productivity. Increasingly, the conversation is about agents doing actual work accessing systems, making recommendations, executing workflows and potentially taking actions.

That is a fundamentally different risk problem. My job is to make sure governr is solving that problem rather than the problem the market had a year ago.

Can you give us an overview of your business?

governr is building an AI operating and risk system for enterprises in highly regulated industries, like financial services. The simplest way to understand it to tell governr what you want AI to do. We determine what could go wrong, how material that risk is, what controls reduce it, and therefore how much of that work you can safely give to AI.

Today, companies can build an agent remarkably quickly. What remains much harder is determining whether that agent should be allowed to perform a consequential business process. Security products can tell you whether the agent has the right credentials. Evaluation tools can tell you how well a model performs. Observability platforms can tell you what an agent did. Those are all important.

But the business still has to answer a different question: is this an acceptable risk for us to take?

We are building the infrastructure to answer that question continuously.

Underneath the product is a quantitative AI risk framework. Rather than stopping at a red, amber or green assessment, we identify measurable risk events, assess probability, severity, controls, vulnerability and dependencies, and ultimately translate material AI exposure into financial-loss terms. That means AI can ultimately be managed in a language that a CRO, CFO and board already understand.

Tell us how you are funded?

To date, governr has been funded privately by its founders and early strategic investors who have exited billion-dollar businesses and worked in high frequency risk management. We deliberately spent the initial phase building the underlying intellectual property, risk methodology and technology rather than trying to scale a conventional software product prematurely. We are now moving into the next phase, putting the system into live institutional environments and working with a select number of financial institutions on production deployments.

What's the origin story? Why did you start the company? To solve what problems?

The original observation was very simple. AI capability is moving much faster than enterprise permission controls can handle.

A bank might discover that an agent can perform 60% of a workflow, but that doesn't mean the bank is comfortable allowing it to do so. Why not? Because somebody has to understand what happens if it is wrong.

Once AI moves from producing information to doing work in the real world, the risk becomes the business process the AI is participating in, such as what it can access, what decisions it influences, what actions it can take, how autonomous it is, what controls sit around it and what financial consequences can follow.

That led us to a much bigger idea. Financial institutions need a Risk Book for AI. For every material AI-enabled workflow, they need to understand the risk being taken, the controls mitigating it, the financial exposure and whether that position remains within appetite. The deeper thesis is that as companies delegate more of themselves to machines, they will need an independent record demonstrating that they remain in control of what those machines are doing.

Who are your target customers? What's your revenue model?

Our initial focus is regulated and risk-sensitive institutions, particularly financial services and health care.

Banks, insurers, asset managers, wealth managers, payments businesses and other financial institutions are interesting because they already understand the concept of risk appetite. Their problem isn't convincing them that risk matters. Their problem is extending mature risk disciplines into a completely new technological environment.

If you had a magic wand, what one thing would you change in the banking and/or FinTech sector?

I would stop treating AI risk as a compliance exercise. This is a fool's errand, and one that adds even more risks and losses in the system. We have inherited a world of questionnaires, inventories, policies, heat maps and committees. They have a purpose, but they don't answer the fundamental economic question.

If a bank wants an AI agent to do substantially more tomorrow than it does today, what additional risk is it taking? If a bank has 1000 agents running, and one changed yesterday, they need to turn that into a business risk decision. That should be measurable. And that doesn't exist in most firms.

Financial institutions would never manage a trading book simply by labelling positions "low", "medium" and "high" risk. They developed sophisticated disciplines around exposure, concentration, expected loss, tail risk and capital. AI needs to move in the same direction.

Our own framework explicitly separates risk, loss and capital, because they are different things. Risk creates potential adverse events; those events can create financial losses; severe unexpected losses determine the financial resources an institution may need to absorb them.

My magic wand would be to bring that level of financial discipline to AI.

What is your message for the larger players in the Financial Services marketplace?

Don't confuse having controls around AI with knowing how much risk you are taking with AI. The next phase of AI in financial services isn't primarily about deploying more models. It is about giving machines more responsibility and scaling it without a drain on human expertise and judgement. An AI agent may remain completely within its technical permissions and still produce an outcome the institution never intended. It may combine information in an unexpected way, take an unforeseen path through a workflow, or use legitimate capabilities in a combination that creates material exposure.

So I would ask every CEO, CRO and CIO three questions:

What proportion of your business will AI be doing in three years?

How much financial risk will that create?

Can you answer the second question as confidently as the first?

If you can't, that gap is going to become increasingly important.

Where do you get your Financial Services/FinTech industry news from?

I read very broadly rather than relying on a single source. I also spend a lot of time with the builders and users of AI to stay ahead of the game.

Bloomberg, Financial Times, Barrons, NYT and WSJ remain important for understanding markets and institutions, but increasingly I go directly to research papers, regulatory publications and technical work from the organisations actually building AI.

The speed of development means that by the time something becomes conventional wisdom, it can already be several months out of date.

What FinTech services (and/or apps) do you personally use?

What interests me much more today is the convergence between financial infrastructure and AI infrastructure. We are moving from software that helps a person make a financial decision to software that can increasingly participate in making and executing that decision itself. That changes the architecture of financial services. Identity, payments, data, risk, compliance and AI will increasingly have to interact machine-to-machine rather than through a human operator. That's where I spend most of my attention.

Finally, let's talk predictions. What trends do you think are going to define the next few years in the FinTech sector?

The biggest change will be the transition from AI that advises to AI that acts. The first wave of generative AI was largely about producing things like text, code, research, summaries and analysis. The next wave is about doing things.

Agents will increasingly interact with customers, move through internal systems, make recommendations, initiate transactions, write and execute code, manage workflows and communicate with other agents. That means the limiting factor will gradually move from capability to permission. The technology will frequently be capable of doing more than the institution is prepared to allow it to do.

I believe AI risk management will evolve from today's predominantly qualitative governance approach toward something much closer to the disciplines financial institutions already use elsewhere like measurable risk events, financial exposure, dependencies, concentration, risk appetite and ultimately risk-adjusted return.

The winners will be the institutions able to trust their AI with the most consequential work. And that, ultimately, is what we're building governr to enable.

Many thanks to Rajen for taking the time to share his insights with FinTech Profile. You can learn more about governr on their website.