Realtime AI video analysis app Lloyd will offer developer kit

Disclaimer: EndlessAI previously published a contributor piece on VentureBeat announcing the launch of Lloyd in early October.

Four-year-old AI startup EndlessAI isn’t a household name — yet.

But its founders and leaders believe they have a bonafide hit on their hands: their freemium iOS app Lloyd, which uses proprietary video streaming and encoding tech to feed the user’s live video view to underlying AI models including OpenAI’s GPT-4o for help with a wide variety of tasks from bicycle repair to telling bedtime stories, has achieved 50,000+ users three months after a stealth launch.

41% of those users engage with the app daily, according to data provided to VentureBeat by EndlessAI.

While it’s no ChatGPT — which became the fastest product in history to cross the 100 million user mark in January 2023, just two months after launch — it is nonetheless encouraging enough to EndlessAI CEO Roi Ginat and Executive Chairman Thomas Pompidou, who told VentureBeat in a recent video call interview they planned to open their platform up to third-party developers in early 2025 and launch a consumer-facing Android app in January.

Also, EndlessAI has already begun upgrading Lloyd with what it calls “powers,” or as Pompidou put describes them, “basically fine tuned, large language models (LLMs) that provide deep dive to consumer on specific use case.”

For example, the first Lloyd Power live now in the app is “Chef,” which provides a realtime, entirely AI coach for you that watches you as you cook (if you point your smartphone camera at your stove top or cooking area) and provides step-by-step guidance.

Another Lloyd Power planned to launch shortly is Tour Guide, which allows users to hold up their phone and see real-time contextual information about their surroundings. By capturing a video of a location, it identifies points of interest, provides relevant details, and can even recommend nearby attractions or activities.

Making realtime video analysis accessible at scale

While current LLMs have struggled to process live video efficiently due to high computational costs. EndlessAI’s technology overcomes this limitation, reducing the cost of video analysis by over 99%.

Pompidou underscored the app’s broader mission: “Our mission is to scale AI to the real world. The real world is visual and live, and today’s large language models, as they’re architected, face challenges in analyzing video accurately, at scale, and cost-effectively. That’s what we make possible.”

This enables real-time video analysis, allowing users to interact with their environment in novel ways, from diagnosing mechanical issues to creating personalized bedtime stories.

Lloyd’s core differentiation lies in its ability to process video data through LLMs at a fraction of the cost typically associated with such tasks. Traditional LLM architectures are not optimized for video, making real-time video analysis prohibitively expensive and slow.

“Analyzing video with ChatGPT, assuming it could, would cost over $300 per hour,” Pompidou said. “With Lloyd, we deliver the same level of accuracy for just 10s of cents per hour.”

This cost-efficiency is achieved without sacrificing accuracy, setting Lloyd apart from competitors that rely on reduced frame rates or lower resolutions to cut costs, often at the expense of reliability.

“Our communication layer is robust in ways other solutions aren’t. It lets developers integrate real-time AI services like speech-to-text, text-to-speech, and video analysis with unmatched reliability and performance.”

As Pompidou envisions the future, he offered a glimpse into the app’s potential: “Imagine a finely tuned LLM trained on every IKEA instruction manual, guiding customers step by step with video and recognizing errors in real-time. It’s just one example of how our technology can transform user experiences.”

Another big arena that EndlessAI plans to court through Lloyd and its underlying video encoding tech: law enforcement, specifically providing analysis of police body cam footage.

“If there is someone who has a heart attack, it is going to recognize that and provide the officer with instructions on what to do right away,” said Pompidou.

Privacy and security

Even though Lloyd itself sees exactly whatever you point your smartphone camera at, EndlessAI prioritizes user privacy.

“Data stays private to [user] accounts, and we only access it for support if users explicitly request assistance,” Ginat said.

This approach ensures robust safeguards while enabling seamless interactions.

But as a consequence, EndlessAI isn’t exactly sure what the most popular uses for Lloyd are among its users. Anecdotally, it says that its surveys and feedback forms have show interest in food preparation, household repairs, fashion and lifestyle coaching, and more.

While Lloyd’s consumer-facing features gain traction, EndlessAI is also building tools to empower developers and enterprises to harness its technology.

“Our long-term roadmap includes an SDK for developers, starting early next year,” Pompidou said. “It will empower them to create unique visual AI solutions with extreme simplicity.”

The SDK will allow developers to integrate AI vision capabilities into their own applications.

“The first offering for developers will be a robust platform for real-time API communication, connecting to OpenAI and other backends,” Ginat told VentureBeat. “Developers can pick and choose which components they want to use, such as audio services or speech-to-text.”

Applications for these tools span industries, from creating AI-enhanced chat applications to integrating video analysis into production lines and safety monitoring systems.

EndlessAI aims to offer scalable solutions that adapt to different performance and cost requirements.

“Our developer tools will allow on-the-fly adjustments—choosing between backend services or lightweight, on-device solutions depending on the use case and cost requirements,” Ginat added.

By combining robust APIs with an intuitive SDK, EndlessAI envisions a new wave of AI-driven applications that go beyond traditional text or image processing. “We’ll offer developers the ability to integrate various services, including side-processing video, enhancing their sessions with additional capabilities,” Ginat said.

Transforming consumer and enterprise AI

Lloyd’s ability to leverage existing smartphones—without requiring additional hardware—makes it uniquely accessible.

By reducing barriers to entry, EndlessAI is redefining what’s possible with AI in daily life and specialized industries alike.

With its rapid user adoption, versatile applications, and robust roadmap, Lloyd is poised to become a defining innovation in the AI space.

“Our long-term strategy is to stay complementary to LLMs,” Pompidou said. “Even when models can natively process video, we aim to remain the efficiency layer that makes these applications viable and cost-effective.”