- You can now bring Gemini's overlay in any app on your Android device.
- With contextual awareness, Gemini can see the contents on your screen and help you with it.
- Gemini is also getting more extensions that'll bring a deeper integration with various Google apps.
- Some features, like Gemini Live and Gemini overlay, are starting to roll out today, while others will launch in the coming weeks.
Google just hosted their Made by Google event where they launched the latest Pixel 9 smartphone series, Pixel Watch 3, and Pixel Buds Pro 2. In addition to the new hardware releases, the event was rife with AI updates.
Google is rolling out Gemini Live, the voice mode for Gemini, in the Gemini app on Android (with iOS in the queue).
Sakshi Garg
In addition to Gemini Live in the Gemini app, there are also other AI upgrades arriving on Android and they are free, unlike Gemini Live.
Gemini is getting contextual awareness. Users can now bring Gemini's overlay on top of any app with their power button or by saying "Hey Google" and Gemini can help with whatever's on your screen. For example, you can bring the Gemini overlay in the YouTube app while watching a video and ask it about the video. With its context-aware capabilities, it can answer questions or cater to your requests without needing any details about what video you're alluding to.
Gemini can not just read what's on your screen on your Android device but it can also interact with many apps on your phone. For example, if you use Gemini to generate an image in Google Messages or Gmail, you can also directly add those images to these platforms.
However, these capabilities of Gemini are restricted to Android only (since it enjoys a deeper integration with the OS). iOS users will only get access to the leveled-up Gemini Live via their Google app. These features will start rolling out today, just like Gemini Live in the Gemini app on Android.
But there are other updates as well that are not exclusive to Android. Google is deeply integrating Gemini with certain Google apps with more extensions for Keep, Tasks, Utilities, and more features for YouTube Music, which will roll over the coming weeks. There's also a new Calendar extension in the queue, but there's no information on when it will be launched.
These extensions will be available for the Gemini app on Android and the web and for Gemini in the iOS Google app.
With these extensions, Gemini will be able to undertake more tasks for you. For example, it can dig out ingredients from a recipe in your email from Gmail and add them to your shopping list on Keep. It can also make playlists for you in YouTube Music with vague prompts like, make a playlist of songs that remind me of the late ‘90s.
With the Calendar extension that will be available later sometime, Gemini can also access your Calendar. As shown in the demo (that failed twice) at the Made by Google event, you can snap a photo of a concert flier, and ask Gemini to see if you'll be free on the date the artist will be coming to your city.
Gemini will extract the relevant information from the flier, reference the date from your Calendar, and inform you if you have any prior commitments.
If with some help from Lady Luck, you turn out to be free, you can ask Gemini to add a reminder to check the tickets for the concert, and the AI assistant will be able to add it to the app. It'll all come together rather nicely for people deep in the Google ecosystem.
It's certainly better news that many Gemini updates are not going to be exclusive to the latest Android phones with powerful processors only. Certainly, there are some exclusive AI updates coming to the Pixel 9 series only and that's understandable; many AI features do need extreme processing power. But not all do. Maybe Apple should have taken a page out of Google's playbook and brought some Apple Intelligence features to older iPhones as well.
Member discussion