‘Humans are responsible for actions taken by AI agents’: Kore.ai CEO Raj Koneru

At a time when there are tensions on the autonomous behaviour and nature of AI agents, Kore.ar CEO and co-founder Raj Koneru has said that since it is humans that are developing AI agents, the accountability of those agents lie with humans and they are responsible for the decisions taken by these AI agents. AI agents are programs that are sophisticated enough to act autonomously, and there is an industry-wide fear that a small error can trigger a "cascading failure" across your digital ecosystem.

“The 'who's in charge' question is the right one to be asking, and I am glad the conversation has matured to this point. The humans are always in charge and accountable, and have to be. Humans are creating the agents, defining their role and responsibilities, set guardrails, and determined how much autonomy to be given. So, ultimately we humans are responsible for the decisions taken by these AI agents,” Koneru told The Times of India on the sidelines of the recently held India AI Impact Summit 2016.

He clarified that the “accountability does not transfer to the technology. If you built it, if you defined it, you own the outcome. It is that simple.”

Kore.ai is headquartered in Orlando, Florida, USA, and it has major offices and operations in Hyderabad (India), London, Seoul (South Korea), Tokyo (Japan), Frankfurt (Germany) and a presence in Dubai (Middle East). The company builds a platform and applications out of India and provide them to large enterprises in the US and globally.

‘Nature of the jobs will change’

While responding to a question on job cuts by the burgeoning use of AI chatbots and AI Agents, Koneru said that “the job fear is a moot point.”

“The jobs will not disappear but their nature will change. You see, humans always evolved with technology. Before emails we were communicating through paper mails, but we adopted to emails and now to WhatsApp. Same thing will be with AI. Going forward, the nature of jobs will split into two. Those who build agents for others to use, and those who use agents to get their work done. Either way, you will work with AI agents,” the CEO explained.

While talking about the ability of AI agents Koneru claimed that they are not matured enough to take decisions by themselves.

“Let us be honest about where we are. Agents have not yet matured to the point where they can take critical financial or legal decisions on their own. That is just a fact, and anyone building in this space knows it. We are on that path, but we are not there yet," Koneru added.

He explained that the AI agents created by Kore.ai are designed with a confidence threshold which means that when the agent acts, it does with confidence, when it is not, then the AI agent will stop and human will intervene .

“That intervention layer is non-negotiable in our design. And because the stakes are so different across industries, risk appetite has to drive where that threshold sits,” Koneru explained.

Raj Koneru on what is ‘India's greatest strength’

Koneru also argued that building infrastructure like massive data centres, or creating base models like GPT models is now a standard commodity but India’s strength is to scale the technology to higher levels. This means that India doesn't need to ‘win’ by outspending them on hardware but to build value on top of it.

“Infrastructure and models are table stakes, and the world is already building those at scale. That is not where India's opportunity lies. India's greatest strength has always been in taking technology and building extraordinary value on top of it. We thrived on the service economy that way. AI is no different,” Koneru argued.

He said that what India needs is education and direct talent towards building applications for the world, like the way the country does in the IT industry.

“So there are two things that will actually matter, including for rural India. First, educate and equip the workforce to build with AI technology. Second, direct that talent towards building sophisticated AI applications not just for India but for the world,” he said.

“Just like India exported IT services, it can now lead by building AI applications for India and for the world. Just like kore.ai built the enterprise AI technology from India for the world. The infrastructure is here, the talent is here. The question is whether we build on top of it with the same ambition. That is where India will thrive, and that is where the real transformation will happen,” he told The Times of India.