More details about iOS 18’s AI features emerge

Apple is expected to bring a plethora of AI features with iOS 18 under the internal codename “Project Greymatter”. A new report from AppleInsider sheds more details on the AI tools coming to Siri, Notes, and Photos.

Greymatter Catch Up is the new notification summary feature which was previously detailed as “smart recaps”. It is expected to offer summaries for missed notifications as well as web pages, notes, and documents and the new bit is that users will be able to ask Siri for a recap of their recent notifications.

Speaking of Siri, Apple’s digital assistant is getting Apple’s on-device LLM and a general overhaul with a new smart response framework. This should result in more cognitive and natural-sounding Siri responses with a deeper integration of contacts, companies, calendar events, and locations in responses. Apple is also allegedly working on cross-device media controls with Siri on multiple devices – allowing users to control media playback on their Apple TV from their Apple Watch for instance.

Siri and Notes are also expected to receive built-in text summarization and audio-to-text transcription.

Apple is also backing generative AI features in its Photos app across iOS 18, iPadOS 18 and macOS 15. Clean Up feature will allow users to remove unwanted objects from their photos – similar to Google’s Magic Eraser. Generative Playground is the more interesting feature which will allow for image generation and photo edits in the Photos app and alongside an app extension in iMessage.

Renders of the Photos app in macOS 15 (AppleInsider)

Alongside the transcript and summarization features, Notes is also getting in-app audio recordings. Users can add text, audio, and images in a single note.

Renders of the Notes app in iOS 18 (AppleInsider)

Math Notes is another big productivity addition inside the Notes app which will support mathematical notation and the ability to recognize text in the form of mathematical equations and provide solutions to problems.

Source