At Apple's 2024 event, the company announced numerous new features and updates for its devices, including OS updates for the iPhone, iPad, Mac, and VisionPro. In addition, it unveiled its own take on AI, labelling it Apple Intelligence, which will be powering all Apple devices soon.
However, one of the most significant announcements was that Siri, Apple's virtual assistant, will be getting a major upgrade. It will feature a new, more intuitive design, improvements to its conversational capabilities, and integration with OpenAI's ChatGPT.
This upgrade comes as a part of iOS 18, Apple's new OS for iPhones, and allows Siri to do a lot more and in a much better manner. This being a major upgrade, there's a lot to discuss, so let's take a look at Siri's new capabilities.
New Design
While there's a lot to talk about Siri's new and improved functionalities, what stole the show almost immediately was its redesign. The virtual assistant is getting a design overhaul, so whenever it activates, you will see a glowing light along the edges of your device instead of the symbol that used to appear earlier.
Siri now works everywhere
Apple's generative AI, which it is calling Apple Intelligence, is coming to Apple devices and will also be powering Siri. This will allow Siri to work both inside and outside apps, and it can also be activated throughout the system.
It can take actions like retrieving information from documents, such as your driver's license or emails, and extracting it to fill up forms.
On-Screen Awareness
Siri is also getting a new feature called 'on-screen awareness', which will allow it to understand what is going on on your phone screen, and take action on your behalf if required.
For instance, you can use a prompt like Show me pictures of [your friend] in a pink outfit, and it will show them to you.
Similarly, you can then select a photo and ask Siri, Make this photo pop and get it done easily.
Once Siri edits the photo, you can again give a command like Add this photo to Notes or something similar, and Siri will again open that app and perform the required action. According to Apple, Siri will be able to perform several actions in and across different apps.
These actions will rely on Apple's new App Intents framework, which will also be available as an API to developers. So, they can design their third-party apps to allow Siri to work within them.
Supports text interactions
Siri's upgrade also brings features that have been long overdue and have been present in other similar chatbots for some time now. Most important among them is the ability to interact with Siri using text, which users can now do by double tapping at the bottom to activate the chatbot, and then typing their questions or commands as needed.
You can now type to interact with Siri. Video Source: Apple
This makes Siri even more usable in situations where you do not wish to disturb others or may wish to keep your searches more private.
Improved Conversational Capabilities
Siri is also getting better at understanding conversations and users will now find it more natural to interact with the virtual assistant. Thanks to its new capabilities, it will now be able to understand your commands and questions even if you stumble while speaking them aloud. So, you might not have to repeat your questions and requests, as was the case earlier.
For instance, if you ask Siri what's the weather like in a specific location, but mispronounce the name of the place, the virtual assistant will still be able to understand what you mean and show you the relevant information.
Another improvement is that Siri can now retain the context of the conversation between requests, so you can ask it to do a larger variety of things at once.
For instance, after you've already asked Siri about the weather of a particular place, you can ask the chatbot to create an event, such as a hiking trip to that location. You won't have to mention the name of the location again, as Siri will automatically understand what you want.
Personal Context Awareness
Additionally, with its semantic index, Siri will be able to access your personal data in apps like Calendar, Files, and Photos, and provide information that you need, such as PDF documents. It can even find information when you're not even sure what you're looking for.
For example, you remember that your friend sent you book recommendations but don't remember whether they sent it in the mail or message. So, you can use requests like Show me the book recommendations [your friend] sent earlier and Siri can find the relevant information on its own.
You can also ask it to Get my ID from my driver's license and Siri will search through your data to fulfill that request and even add it to forms.
You can give Siri other commands and depending on those, it will switch between different apps to provide you with the information you are looking for. All this will take place while ensuring complete user privacy with most processes occurring on your device, with the rest relying on Apple's new Private Cloud Compute proprietary servers running on Apple silicon.
Device support comes to Siri
Siri now also has extensive product knowledge about features and settings for Apple Devices. So, it can also assist you in learning about your device as you can now ask questions about your iPhone, iPad, and Mac and it will answer them, providing device support wherever you go. It can help learn how to do things on your Apple device easily. Siri can even understand what feature you're referring to even if you don't know the exact name and only give it an explanation.
For example, you can ask Siri How do I send a message so it gets delivered tomorrow? and the virtual assistant will provide you with all the steps you need to follow to accomplish that task even though you didn't specifically request instructions for the 'Send Later' feature.
ChatGPT Integration
OpenAI recently showcased its latest GPT-4o flagship model, and Apple is partnering with the company to bring its capabilities to Siri. The assistant will be integrated with ChatGPT, allowing you to ask questions and get suggestions as and when you want.
When you ask Siri a complex question that it cannot answer on its own, it will ask you whether you want to use ChatGPT to complete the request. If you choose to do so, Siri will leverage ChatGPT's abilities to provide you with the required information. Apple stated that user requests and info will not be retained and you even do not need a ChatGPT account to use it via Siri.
Siri is getting its most significant upgrade in several years with Apple Intelligence and ChatGPT integration and is getting closer to becoming actually useful once more. Unfortunately, it looks like the upgraded version will only be available on a few select devices, which include the iPhone 15 Pro, iPhone 15 Pro Max, and iPads and Macs with the M1 or newer chips.
Pallav Pathak
As for when the new and upgraded Siri will be available, Apple says it is coming this fall. So, this gives users with non-supported devices plenty of time to decide whether they want to upgrade to a new one to access Siri's new features.
Member discussion