Introduction to AI Mode
Google is taking a significant leap forward in its search-centric AI Mode chatbot by introducing multimodal capabilities. This update enables the chatbot to "see" and answer questions about images, making it a more comprehensive tool for users. The expansion of AI Mode to "millions more" users is a notable development, as it was initially exclusive to Google One AI Premium subscribers.
What’s New in AI Mode
The latest update combines a custom version of Gemini AI with Google’s Lens image recognition technology. This integration allows AI Mode Search users to take or upload a picture and receive a detailed response with links about the image’s contents. The multimodal update is available starting today and can be accessed through the Google app on Android and iOS devices.
Understanding Multimodal Capabilities
According to Robby Stein, VP of product for Google Search, "AI Mode builds on our years of work on visual search and takes it a step further." The multimodal capabilities of Gemini AI enable AI Mode to understand the entire scene in an image, including the context of how objects relate to each other and their unique materials, colors, shapes, and arrangements. This advanced understanding allows AI Mode to provide incredibly nuanced and contextually relevant responses.
How it Works
The update uses a "fan-out technique" that issues multiple queries about the image and any objects within it. This technique enables AI Mode to identify specific objects, such as books, and provide suggestions for similar titles with positive ratings. Additionally, AI Mode can answer follow-up questions to further curate recommendations, making it a highly interactive and helpful tool.
AI Mode and its Competitors
AI Mode for Search is Google’s response to other chatbot-like experiences, such as Perplexity and ChatGPT Search. These platforms provide AI-generated summaries pulled from their respective search indexes. AI Mode, however, has the added advantage of multimodal capabilities, making it a more versatile and user-friendly option.
Expansion and Availability
Initially, AI Mode was only available to Google One AI Premium subscribers within Labs. However, Google has now started to make AI Mode available to "millions more" Labs users in the US, beyond just paying AI Premium subscribers. This expansion marks a significant milestone in the development and rollout of AI Mode.
Conclusion
The introduction of multimodal capabilities to Google’s AI Mode chatbot is a significant development in the field of search-centric AI. With its advanced image recognition technology and ability to provide nuanced responses, AI Mode is poised to become a leading tool for users seeking comprehensive and interactive search experiences. As Google continues to expand access to AI Mode, it will be exciting to see how this technology evolves and improves in the future.
