European Mistral AI Challenges Everyone: The Robot Navigates with a Simple Camera and Surpasses LiDAR

Mistral AI

Mistral AI has announced Robostral Navigate, its first AI model specifically developed for the autonomous navigation of robots. The new solution represents a further step in the French company's strategy towards so-called Physical AI and aims to enable robots to move in complex environments using solely a standard RGB camera and instructions provided in natural language.

The model, consisting of 8 billion parameters, can interpret images from a single camera and follow instructions such as reaching a specific room, crossing a corridor, or stopping in front of a specific object. Unlike many competing technologies, Robostral Navigate does not require depth sensors, LiDAR systems, or multi-camera configurations.

According to data released by Mistral AI, the model achieved a 79.4% success rate in familiar environments and a 76.6% success rate in previously unseen scenarios during training in the R2R-CE benchmark. The company claims that these results surpass both the best solutions based on a single camera and those using additional sensors.

Further Details on Mistral AI's Technology

The technology is designed to be used in numerous contexts, including offices, commercial buildings, homes, industrial facilities, and outdoor environments. Additionally, the system is hardware-independent and can be implemented on wheeled robots, quadrupedal robots, and flying platforms. To determine the next movement, the model identifies the point in the image where the robot should head and also calculates the desired final orientation.

When the destination point is not within the camera's view, the system instead relies on movement instructions expressed in the robot's local coordinate system. Robostral Navigate was developed entirely by Mistral AI without using any pre-existing open-source models. Training occurred solely in simulation through approximately 400,000 trajectories distributed across 6,000 virtual scenarios.

The company also adopted a training technique based on so-called prefix-caching, which reduces the number of required tokens by about 22 times, significantly shortening development times. Subsequently, the model was further enhanced through an online reinforcement learning algorithm called CISPO, which increased the success rate by 3.2%, allowing the robot to also learn from mistakes during exploration.