Whether it's Siri, Cortana, or Google Assistant, AI assistants have been present on smartphones for some time now. And with the rapid advancement of Artificial Intelligence technologies in every domain, these assistants are only bound to become better with time.
The first example we have of this is Google's Gemini, which has replaced Google Assistant on Android smartphones as the default AI assistant. While earlier Gemini was quite limited in what it could do compared to Google Assistant, Google recently announced major updates to it that will make the chatbot incredibly powerful and more useful.
This is especially true for Gemini Nano, which was announced on Pixel 8 phones and Samsung Galaxy S24 back in January. Here is how Google plans to update this model so it becomes even better.
Gemini's Contextual Awareness is Getting a Major Improvement
Like other chatbots, Gemini can usually understand the context in which you interact with it, though this is not always the case. That is why, one of the most important changes coming to it is a significant improvement in its contextual awareness capabilities.
Gemini can now easily understand what is going on on your screen and even the application you may be using at any given time. According to Google, users will be able to bring up the Gemini overlay and use the AI while using another app at the same time. For instance, you can generate images with Gemini and directly drag and drop them into Google Messages, Gmail, or another app.
Another example is when you're watching a YouTube video, you can use Gemini's 'Ask This Video' feature to get information from that video. This is similar to Gemini Advanced's 'Ask This PDF' feature which can help you get information quickly from a PDF without needing to read through it completely.
Finally, there are dynamic suggestions that Gemini will offer by checking your screen. This can even happen without you needing to type anything since Gemini will automatically understand the context of whatever you are working on.
For example, if you are having a conversation in Google Messages about going for a coffee, Gemini can automatically recommend the best coffee shops near you in Google Maps.
Users will soon be able to access Gemini from its overlay panel which will be present even while providing results, so you don't have to open it in full-screen mode every time.
Gemini Nano to Get Full Multimodal Capabilities
Till now, Gemini relied on text or voice-based inputs to understand a user's query and provide relevant answers. Now, Google is upgrading its capabilities and calling it Gemini Nano with Multimodality. This is because the updated model can also use visuals, spoken languages, and sounds to understand what a user wants.
This is a huge but very welcome upgrade and will drastically change what smartphones with AI assistants can do. You will be able to use your Pixel smartphone to get information from your surroundings through its camera and microphone instead of just relying on text.
Circle to Search Has Been Updated to Help Out Students
Back in January, Google announced a new feature for select Android smartphones – Circle To Search, which lets you use a simple gesture to search for information without leaving the app you are using. This works with multisearch, allowing you to use both images and text to ask Google a question about anything on your screen.
Since its launch, Circle To Launch was updated with features like full-screen translation, but at its I/O Developer Conference, Google announced yet another update. It can now help students with their homework by helping them solve math and physics problems easily.
Students just need to circle the problem they want help with, and Gemini will provide them with the steps they need to follow to get the answer.
Google also announced that Circle To Search will soon even be able to help solve complex problems involving graphs, diagrams, formulas, and more. Google is also expanding the feature to more Pixel and Samsung smartphones.
Talkback Will Offer Better Descriptions
Talkback is an Android feature designed to help people with vision disabilities by explaining what is being displayed on their smartphone screens. With Gemini Nano gaining multimodal capabilities, Talkback is also becoming much better at describing objects and elements.
What's most impressive about this is that this works even when your device is offline. The updated Gemini Nano model can offer descriptions that will be very similar to what a human would provide, making it much more useful compared to how Talkback works right now.
Gemini Can Now Alert You About Scam Calls
As scam protections have become more robust over the years, scam calls have also gotten more sophisticated. So, despite these protections, many people still fail to distinguish scam calls from genuine ones. That is why Google is testing a new feature that will let Gemini Nano provide real-time alerts for scam protection.
It will be able to detect patterns in conversations usually present in scam calls and alert you immediately. For instance, if you get a call from someone claiming to be a bank representative asking you to transfer funds or something along the same lines, you will get an alert. And since this is on-device protection, your privacy will not be compromised.
Gemini Nano was introduced with the Pixel 8 phone series and Samsung Galaxy S24 earlier this year. The updated model is expected to arrive on Pixel phones later this year, while features like scam call alerts will make their way to all Android devices.
With these updates, Google is making Gemini more powerful and better at being a virtual assistant that can help you out whenever you need it. However, most of these features will be limited to flagship phones, at least for now, so we may have to wait for some time to see them on other devices.
Member discussion