Industry leaders at Groq roundtable say India’s AI future hinges on speed and economics

Artificial intelligence has moved past the pilot stage in India. Enterprises across ecommerce, travel, food delivery, fintech, gaming, and real estate are deploying models that handle millions of daily transactions, conversations, and interactions. Yet, as panelists at the Groq × YourStory roundtable themed ‘Faster than now: Building India’s real-time AI future’ in Bengaluru pointed out, the country’s most ambitious use cases will not succeed without infrastructure that can respond in real time, at sustainable costs, and at national scale.

The closed-door roundtable, moderated by Sangeeta Bavi, COO of YourStory, brought together a curated group of India’s leading technology builders: Sanjay Mohan (CTO, MakeMyTrip), Madhusudhan Rao (CTO, Swiggy), Jithendra Vepa (Co-founder & CTO, Observe.AI), Manish Gupta (Founder & CEO, Rezo.ai), Zaher Abdulazeez (Head of Data Science, NoBroker), Ramesh Gururaja (SVP – Consumer Products & Growth, Flipkart), Samir Deepak Shaw (Head of Data Science, Flipkart), Dr. Arjun Jain (Founder & Chief Scientist, Fast Code AI), and Satish Rao (SVP – Semiconductor BU, Mistral Solutions).

They were joined by members of the Groq leadership team — Jonathan Ross, Founder and CEO, Scott Albin, GM – APAC, Mohsen Moazami, President International (Office of the CEO), and Mehul Patel, Sales – India, who shared global perspectives and demonstrated Groq’s ultra-low latency infrastructure in action.

Latency: the invisible deal-breaker

Across industries, participants agreed that latency is the biggest blocker to unlocking real-time AI at scale. Current commercial models often take many seconds to generate responses. For customer-facing applications — travel planning, ecommerce support, food delivery ETAs, or multiplayer gaming — such delays are deal-breakers.

The ambition, panelists stressed, is to move from seconds to sub-seconds. True adoption will come when AI interactions feel instant, natural, and seamless. Without that, customer trust and business value erode.

Economics: AI must beat human costs

Even if speed improves, leaders cautioned, economics will decide adoption. India’s market is unforgivingly price-sensitive. In areas like customer service, panelists noted that a human call center agent costs roughly Rs 9 a minute. For AI to replace or augment that role, the cost per interaction has to fall below that line.

The group emphasized that businesses cannot wait years for ROI. The expectation is that AI must justify itself within the current financial cycle. Latency without affordability is unsustainable; affordability without speed is irrelevant.

Language and voice: India’s unsolved frontier

Several participants pointed out that language diversity is as big a barrier as speed or cost. Models that perform well in English struggle with the code-mixed, accented, and noisy environments that dominate India.

Voice AI was called out as especially challenging. While progress has been made, most systems fail to handle the wide range of Indian dialects and accents in real-world conditions. Solving this frontier, panelists argued, is critical to taking AI beyond metros and into mass-market adoption.

Another theme that surfaced was the changing nature of consumer decision-making. Leaders noted that users are moving away from search-based discovery and toward conversational interfaces. If customers increasingly rely on AI assistants for shopping, travel, or financial advice, platforms risk losing the traditional top of the funnel.

The consensus: enterprises cannot simply bolt AI onto existing flows. They must design AI-native experiences where real-time intelligence is central to customer engagement.

Edge and constrained environments

Beyond consumer internet, voices from sectors like defense and semiconductors stressed that edge deployments come with unique constraints. In these contexts, workloads cannot rely on the cloud due to security, bandwidth, or latency requirements.

Panelists discussed the need for smaller, efficient models that can run on-device under strict power and heat limitations. These innovations, they argued, would not only benefit defense but also unlock opportunities in automotive, IoT, and rural connectivity.

Enterprise leaders also reflected on AI’s impact on developer productivity. Generative tools can accelerate coding, but they also introduce integration challenges and knowledge gaps. Without governance and evaluation frameworks, short-term gains may become long-term risks.

At the organizational level, panelists noted adoption gaps. Non-technical teams often approach LLMs as “magic wands” and lose momentum when they confront the effort required for integration and scaling. Clear frameworks and change management will be essential to close these gaps.

Data residency and local infrastructure

Infrastructure readiness was another recurring concern. Leaders pointed out that several industries, particularly BFSI and telecom, already require customer data to remain in-country. This makes local infrastructure not just a compliance factor, but a practical necessity for adoption.

While global models and cloud deployments remain important, the panel agreed that India’s AI ambitions depend on building local capacity — both to meet enterprise requirements and to enable large-scale real-time applications.

Groq’s demo: speed in action

The Groq team showcased its ultra-low latency capabilities, running models across text, vision, and video with sub-200 millisecond inference times. Demos included conversational itinerary planning, near-instant code generation, vision-based object recognition, and video rendering scaled to 4K within seconds.

Groq also shared its roadmap: to significantly accelerate the scale of its infrastructure globally, with pricing models that aim to bring down costs dramatically. The leadership confirmed that India is under active consideration for infrastructure deployment, contingent on visible demand signals.

Rethinking the global game board

The discussion closed on a forward-looking note. Panelists agreed that India is no longer a passive adopter of global AI. With its scale, data, and talent base, the country has the potential to set global benchmarks in real-time AI.

Groq’s Jonathan Ross left the group with a provocation: the global technology order is being rewritten faster than anyone anticipated. Could the next generation of top global tech companies be Indian? The roundtable agreed that the answer depends on whether India can combine speed, economics, and ecosystem readiness to make real-time AI inevitable.

The roundtable highlighted both the optimism and urgency surrounding India’s AI journey. Optimism, because the market has the scale, talent, and entrepreneurial drive to lead. Urgency, because latency, costs, and infrastructure gaps still stand in the way.