NeuralGarage seeks to take away audio-visual dissonance in dubbed content material utilizing generative AI

Note: Neural Garage was featured in YourStory in July. This is a truncated model of the profile, with up to date info on the startup’s entry into YourStory’s Tech30 record, as one of many 30 most promising Indian startups of 2023.

Today content material is dubbed in a number of languages to achieve a wider viewers. However, the dubbed content material doesn’t at all times present a cohesive viewing expertise because the lip and jaw actions of the actors don’t synchronise with the phrases popping out of their mouth.  

NeuralGarage goals to deal with this difficulty with its flagship AI-powered product VisualDub—in order that viewers have an immersive expertise and are in a position to respect the visible points totally.  

“I’ve a terrific fondness for Korean movies, and this was clearly at its peak in the course of the lockdown. But I had issues with the visible dissonance as a result of lack of audio-video synchronisation. This sparked the concept of whether or not this could possibly be solved,” says Anjan Banerjee, one of many founders of NeuralGarage. 

Driven by his need to bridge the hole between audio and video, Banerjee started finding out the potential of AI alongside along with his batchmates from IIT Kanpur, Subhabrata Debnath and Subhashish Saha. The three of them had earlier co-founded Visage Map, a facial recognition startup, which was later acquired by FaceFirst, a US-based facial tech firm. 

The trio reached out to Mandar Natekar, a media and leisure veteran, for mentorship, which in the end paved the best way for the start of the deeptech startup NeuralGarage in 2021. 

“Our imaginative and prescient at NeuralGarage is to make communication seamless throughout all obstacles of language visually by way of the facility of AI,” says Natekar, Co-founder and CEO, NeuralGarage.

The identify ‘NeuralGarage’ stands for neural networks—which is the core of AI—and pays tribute to tech corporations Apple, Microsoft, Google, and Meta, which had been supposedly began from a ‘storage’.

Eliminating audio-visual discord

NeuralGarage’s flagship product VisualDub reduces audio-visual disparity in dubbed content material by syncing the lip and jaw actions of actors with the audio—utilizing proprietary generative AI (GenAI) know-how.  

GenAI transforms facial elements utilizing audio activations, mixing them with the remainder of the scene. The lip actions are tweaked to match syllables, and the jaw and chin actions and smile strains are harmonised with this to make the dubbed content material visually reasonable and pure. This know-how is person- and language-agnostic. 

“Removing visible dissonance makes the dubbed content material look genuine and native, which helps viewers and customers join extra with the content material,” says Natekar. 

NeuralGarage gives this know-how by way of API integration, SaaS, and desktop software program. 

The know-how has been examined in additional than 30 languages internationally, together with many Indian languages and worldwide languages reminiscent of Italian, German, Spanish, Japanese, Korean, and Mandarin. 

Business and progress

NeuralGarage generates enterprise from promoting, influencer advertising, content material creation, OTT, and movies. Film and edtech content material account for greater than 90% of its revenues. All these initiatives are below course of. 

The startup has 10 shoppers together with Amazon India, Microsoft, Hippo Video, and Pixis. 

Recently, Amazon’s advert marketing campaign with actor Manoj Bajpayee was shot in Hindi and dubbed in Tamil, Telugu, Kannada, Malayalam, Bengali, Gujarati, and Marathi. 

VisualDub was then used to lip-sync the artistic within the dubbed languages to present the sensation that the artistic was truly shot in these many languages, thus creating an genuine reference to the patron, explains Natekar.  

NeuralGarage—which is backed by institutional VC fund Exfinity Venture Partners and angel buyers together with Amit Patni—has raised $1.45 million thus far in its seed spherical.

Also Read

Nawgati: An AI-powered platform that gives real-time updates on congestion and wait time at gasoline stations

There are over 60 startups within the generative AI panorama in India, throughout numerous industries, in line with NASSCOM. They embody TrueFoundry, which helps construct ChatGPT with proprietary knowledge; Cube, which allows AI-based buyer help on social media, and Gan.ai, a generative AI video platform. 

This 12 months, NeuralGarage goals to drive business testing and adoption of VisualDub throughout verticals and discover completely different use instances. The startup is concentrating on a income of $1 million and 50 shoppers by the tip of this 12 months. 

Participation at TechSparks

NeuralGarage was a part of TechSparks 2023, YourStory’s flagship tech startup occasion held just lately in Bengaluru, and made it to YourStory’s Tech30 record as one of many 30 most promising Indian startups of 2023.

“YourStory’s flagship occasion TechSparks is a invaluable useful resource for early-stage startups. The platform’s ‘Tech30’ occasion brings collectively high startup founders, offering invaluable insights and greatest practices,” says Natekar. 

“The workforce accountable for the pitches was glorious, offering the required help to current impactful tales. Alakh Pandey, Nitin Kamath, and Bhavish Aggarwal shared candid tales of their journeys and client success, that are invaluable for early-stage startups,” he provides. 

Edited by Swetha Kannan