Additions to Apple Intelligence include English (India), English (Singapore), German, Italian, Korean, Portuguese, Vietnamese, and “others” yet to be announced.
The feature will launch in American English, when it arrives as part of the iOS 18.1 update. The company previously announced that localized English support for Australia, Canada, New Zealand, South Africa, and the U.K. will arrive later in 2024, with support for Chinese, French, Japanese, and Spanish coming in 2025.
For anyone counting, that’s 16 other countries besides the default U.S. English version.
On the iOS 18 page on their website, Apple has linked to a document titled “New features available in iOS 18.” This document provides an in-depth look at all the categories of new features in iOS 18, grouped by “Apple Intelligence,” “Key Features,” and “Even More.”
When I come across detailed guides like this, I like to import all of the information into Craft, then use its grouping and backlinking features to create a more-digestible version with a complete table of contents – something I’ve done for the Shortcuts developer sessions and the latest WWDC keynote, for example.
With this iOS 18 guide, I’ve imported and grouped all the new features into a single doc in Craft for my own personal use, which I’ve also made available for members of my website to browse and peruse.
However, I wanted to share the full outline of iOS 18 features here as a public reference – plus you can still find all the details for each feature in Apple’s PDF.
Here’s that full list:
What’s new in iOS 18
Apple Intelligence
Writing Tools
Rewrite
Style
Proofread
Summarize
Describe
Compose
Image Playground
Image Playground app
Create Images with Concepts
Create Images with People You Know
Create Images with Characters
Create Images with Photos
Create Images with Text
Suggested Concepts
Quick Previews
Preview history
Image Playground library
Animation style
Illustration style
Sketch style
Messages experience
Deep integrations
Siri
Personal context understanding
In-app actions
Onscreen awareness
More resilient request handling
Maintains conversational context
Product knowledge
Glowing edge light
Type to Siri
Predictive text in Siri keyboard
Suggestions in Siri keyboard
More natural voice
More visually rich responses
ChatGPT integration
Focus
Reduce interruptions
Intelligent Breakthrough & Silencing
Genmoji
Genmoji
Mail
Mail Priority messages
Preview summaries
Thread summaries
Smart Reply
Messages
Smart Reply
Summaries
Notes
Transcription summaries
Image Wand
Notifications
Priority notifications
Notification summaries
Announce Notifications support
Phone
Call summaries
Photos
Natural language photo and video search
Create your own Memories
Clean Up tool
Key Features
Customization
More flexible placement
Unique page layouts
Dark icons
Tinted icons
Larger icons
Control Center
Redesigned Control Center
Groups of controls
Controls gallery
Controls on your Lock Screen
Controls with the Action button
Photos
Biggest-ever design update
Collections
Recent Days
Groups in People & Pets
Trips
Wallpaper suggestions
Collection view options
Pinned Collections
Hide screenshots from your library
Months and Years highlights
Expanded content types in Utilities
Customize the Photos app
Clean Up tool*
Edit albums in folders
Album attribution
Edit in light or dark mode
Smart copy and paste edits
Video speed control
Creative stories
Smart suggestions
Messages
Text effects
Text formatting
Messages via satellite
iMessage over satellite
SMS over satellite
Satellite connection in Dynamic Island
Emoji and sticker Tapbacks
Send Later
RCS (Rich Communication Services)
Low Quality Image Mode improvements
High-quality Images
Mail
Transactions category
Updates category
Promotions category
Time sensitive
Grouped by sender
Quick action
Recategorization
Safari
Highlights
Location highlights
Summary highlights
Music highlights
Movie and TV show highlights
Person highlights
Redesigned Reader
Distraction Control
Passwords
Passwords app
Passwords and passkeys
Sign in with Apple
Wi-Fi passwords
Verifications codes
Sorting
Syncing
End-to-end encryption
Windows support
Maps
Hiking and walking
Hikes
Turn-by-turn navigation
Create and save walking and hiking routes
Topographic maps
Discover and save places
Gaming
Game Mode
Wallet
Event tickets
Pay with installments
Pay with Rewards
Apple Pay on Chrome and Windows PC
Tap to Cash
Tap to provision
AirPods
Siri Interactions
Voice Isolation
Personalized Spatial Audio for gaming
Reduced gaming audio latency
In-game voice quality
Notes
Audio recording
Live transcription
Phone call transcription
Math in Notes
Access Math Notes from Calculator
Collapsible sections
Highlighting
PDF inline search
Journal
State of Mind integration
Log mindful minutes
Insights view
Streaks
Calendar
Stats
Search and sort
Export and print
Widgets
Quick Actions
Audio transcription
More formatting options
Shortcuts
Apple TV
InSight
Enhance dialogue
More dynamic subtitles
Home
Hands-free unlock with home keys
Guest access
Pair Matter accessories without a hub
Control robot vacuum cleaners in the Home app
Electricity usage and rates
Privacy & Security
Improved Contacts preferences
Redesigned Privacy & Security settings
Accessory Setup Kit
Blocking sensitive content for children
Even More
Accessibility
Eye Tracking
Music Haptics
Vocal Shortcuts
Vehicle motion cues
AirPlay
Spatial Audio
Apple Account
Share Sign in with Apple credentials
Calculator
Math Notes
Typed math
Variables
Graphing
Adjust numbers
Works with Notes
Math Results
History
Editable expressions
Portrait scientific calculator
Unit conversions
Calendar
View reminders in Calendar
Edit reminders in Calendar
Create reminders in Calendar
Updated Month view
Refreshed look
Camera
5-second timer
Continue playing music when taking a photo
Adjust flash settings via touch and hold
Next-gen Portraits enhancements
CarPlay
Spatial Audio
Accessibility
Continuity
Continuity Camera with Apple TV 4K
Family Sharing
Family Sharing invitations
Recommended for your family
Files
Desktop and documents in the Cloud setup
Keep downloaded
Expanded external drive file formats
Erasing an external drive
Fitness+
For You in Summary
For You
Explore
Library
Search
Awards
Freeform
Scenes
Send a copy
Snap to Grid
Handwriting recognition
Improved diagramming
Math Results
iCloud
Settings redesign
iCloud Mail
iCloud Mail cleanup
Manage old email
Unsubscribe and delete
Keyboard
Keyboard Unified picker
Inline stickers and Memoji
Searchable stickers
Multilingual keyboard
Language detection
Locked and Hidden Apps
Locked apps
Hidden apps
Hidden apps folder
Seamless authentication
Move to iOS
Faster migration
Cabled migration
Improved Wi-Fi migration
Tips during migration
Dual SIM labels
Voice memo migration
Music
SharePlay on more devices
Phone
Call recordings
Call transcripts
Call history search
Keypad search
Live caller ID lookup
Live Voicemail
Dual SIM in Control Center
Automatic Mic Mode
Podcasts
Chapter scrubbing
Better playback control
Share from transcripts
Search suggestions as you type
Reminders
Calendar integration
Recently deleted list
Subtasks in Smart Lists
Multi-language grocery lists
SharePlay
Ask to share
Draw to highlight
Remote control
iPhone to iPhone, iPad to iPad
Privacy friendly
SharePlay on more devices
Tips
Tips sharing
Weather
Easier-to-find Feels Like temperature
Home and work locations
Prominent wind gusts and direction
261 Features Coming to iOS 18
Plus, for fun, I removed all the categories and groups to create a single-file list of all the features – assuming I didn’t miss anything, there’s 261 features coming to iOS 18 (according to the list Apple provided):
On Sunday, September 8, on the Mac Power Users podcast, I had the pleasure of joining hosts David Sparks and Stephen Hackett – here’s the show notes:
Matt Cassinelli joins the show to discuss his background with Workflow and Apple and how it led to him becoming the “Shortcuts Guy.” The group then discusses Apple’s automation tools across its platforms, and how those tools have extended to the web.
New in iOS and iPadOS 18.0, macOS 15.0, watchOS 11.0, and visionOS 2.0
This update includes enhancements to the Shortcuts app across all platforms, including dozens of updated or new actions as well as hundreds of new glyphs for personalizing your shortcuts.
New actions
Accessibility
“Set Switch Control Switch State” can manipulate switches on iOS and visionOS
“Set Hover Text Enabled” can now enable or disable displaying larger versions of text on macOS
Freeform
“Create Board” and “Open Board” are available on iOS, iPadOS, and macOS
Journal
“Create Entry,” “Create Audio Entry,” and “Search Entries” are available on iOS and iPadOS
Magnifier
“Describe This,” “Detect Items,” “Start Point & Speak,” “Open Reader,” and “Start Activity” are available on iOS and iPadOS
Music
“Set Music Focus Filter” is available on iOS and iPadOS
Wallet
“Open Card” is is available on iOS, iPadOS, and visionOS
Shortcuts
“Create Folder” and “Create iCloud Link for Shortcut” are available on all platforms
“Add Shortcut to Home Screen” is available on iOS and iPadOS
Updated Actions
For those building custom shortcuts, some actions have been updated:
“If” now supports evaluating multiple conditions at once, combining them with Any or All
“If” now supports selecting date and time values inline
“Open Smart List” can now open any Reminders list
“Match Text” now indicates invalid regular expressions above the keyboard
“Recognize Music” support has been expanded to watchOS
“Listen to Page” support has been expanded to visionOS
Shortcuts editor
For those building custom shortcuts, some changes have been made to the shortcut editor:
Faster search, improved categorization, and other enhancements to finding actions
Actions can be pinned for quick access
Keyboard navigation support has been expanded on macOS, including ⌘F for search, navigation with the tab and arrow keys, and more
Over 200 new icon glyphs allow greater expressivity in your shortcut icons, including animals, sports, home devices, and more
Improved performance and reliability
Control Center
Control Center now includes a Shortcuts Control that can be configured to run any available shortcut or open apps
Explore the latest shortcuts actions in iOS 18, including powerful new tools for Apple Journal, HomeKit, and Safari. Learn how to streamline your workflow with multi-conditional if statements, smart home toggles, and better organization for your Shortcuts.
In iOS 18.1 developer beta 2, Apple has updated how it organizes actions in the Shortcuts app to create a better experience for users, including a new ordering inside action and categories, as well as two new categories.
This a long overdue change to the Action Editor, which should make it easier for new users to learn how to use Shortcuts – categories are more distinct, important actions are easier to find, and there’s a logical sort order inside groups to help you progressively understand how to use them.
New Order
Throughout all categories and subcategories in the Shortcuts app, Apple has updated the order of actions from alphabetical to a custom order. Instead of actions being ordered by what letter happened to appear at the beginning of what verb happened to be chosen for that action, now the actions inside Shortcuts have a logical order that emphasizes important functions first and group similar functions near each other.
A great example of this new group ordering is in the Media category, which consists of Music, Podcasts, and Photos actions – previously there was were multiple small categories at the beginning (shown above), then photos, then music, then back to photos, then more music in the form of Playlists, and finally podcasts.
Now, all the entire category has been streamlined, music and podcast actions are near each other, and the photos actions are grouped logically as well.
Another great example of the new logical ordering of actions inside categories is the Scripting category, where the most commonly-used actions like Show Result and Ask for Input have been entirely moved up to the top, rather than buried in their respective “No-Ops” and “Notification” categories that deemphasized their critical use while Scripting.
Now, when looking through the Scripting group, it should be easier to find the actions you’re most likely to use near the top, and the more esoteric tools are lower down where advanced users can find them – or just Search for them.
New Groups
Going further, the Scripting category is also now dedicated more directly towards true scripting functions, and in the process Apple has moved a subset of actions out and into new groups for Controls and Device functions.
Over time, as Apple added more and more functionality to the Scripting category, it became bloated with lots of system functions that don’t necessarily relate to the act of scripting, per se.
Now, with the introduction of Controls as function in Control Center, Apple has broken out all Controls-related actions into a new Controls group. Similarly, all the actions that related to the specific Device details have been moved into the new Device category.
These new groups are a great addition, emphasizing the functionality found in Controls and Device categories that might’ve otherwise been lost on users who don’t do much Scripting, while also making the Scripting category more clear on its own.
I hope Apple continues to expand these groups and adds more categories. Apple could open these categories to third-party developers, letting developers add actions to these categories as well as creating more top-level groupings – this would further help users find what actions are available to utilize without going through app-by-app to check.
Welcome Improvements
These updated action grouping, ordering, and new categories are a very welcome addition to the Action Editor experience in Shortcuts – this automation platform has so much potential and little user experience tweaks like this can go a long way towards introducing newer users to such a complicated experience.
And even as a seasoned expert, I’ve wanted these for a long time – hence why I directly asked for these changes via Apple’s Feedback tool, which I’m extremely happy to see implemented.
Check out the developer betas to get early access to these changes, or you can look forward to these updates when iOS 18.1 releases this fall.
Here’s the full breakdown of the new categorization (note that some actions are not contained in a subcategory):
Actions
Scripting
Comment
Show Result
Show Alert
Ask for Input
Count
Control Flow
Choose from Menu
If
Repeat
Repeat with Each
Wait
Variables
Set Variable
Get Variable
Add to Variable
Lists
List
Choose from List
Get Item from List
Dictionaries
Dictionary
Get Dictionary Value
Set Dictionary Value
Get Dictionary from Input
Numbers
Number
Random Number
Round Number
Format Number
Get Numbers from Input
Math
Calculate
Calculate Expression
Calculate Statistics
Measurement
Measurement
Convert Measurement
Dates
Date
Format Date
Adjust Date
Get Time Between Dates
Get Dates from Input
Convert Time Zone
Text
Text
Get Text from Input
Show Definition
Get Name of Emoji
Text Editing
Change
Combine Text
Split Text
Replace Text
Match Text
Get Group from Matched Text
Correct Spelling
Audio
Dictate Text
Transcribe Audio
Speak Text
Make Spoken Audio from Text
Translation
Detect Language
Translate Text
Items
Get Name
Get Type
Set Name
Quick Look
View Content Graph
Shortcuts
Get My Shortcuts
Run Shortcut
Other
Get What’s On Screen
Base64 Encode
Generate Hash
URL Encode
Format File Size
Stop and Output
Stop This Shortcut
Wait to Return
Run Script over SSH
Open X-Callback URL
Nothing
Controls
Set Silent Mode
Set Focus
Set Volume
Set Flashlight
Set Low Power Mode
Shut Down
Connectivity
Set Airplane Mode
Set Wi-Fi
Set Bluetooth
Set Cellular Data
Set Personal Hotspot
Set VPN
Set AirDrop Receiving
Display
Set Appearance
Set Orientation Lock
Set Brightness
Set True Tone
Set Night Shift
Lock Screen
Set Stage Manager
Navigation
Open App
Go to Home Screen
Capture
Take Photo
Take Video
Record Audio
Take Screenshot
Device
Get Device Details
Get Battery Status
Get Orientation
Get Current Focus
Get Physical Activity
Clipboard
Copy to Clipboard
Get Clipboard
Notification
Show Notification
Play Sound
Network
Get Current IP Address
Get Network Details
Get Hotspot Password
Reset Cellular Data Statistics
Wallpaper
Set Wallpaper Photo
Get All Wallpapers
Switch Between Wallpapers
Location
Get Current Location
Find Places
Filter Locations
Open Directions
Open in Maps
Request Ride
Location
Get Details of Location
Location
Get Maps URL
Addresses
Street Address
Get Addresses from Input
Travel
Get Distance
Get Halfway Point
Get Travel Time
Media
Image
Convert Image
Filter Images
Get Details of Images
Get Images from Input
Make Image from PDF Page
Make Image Rich Text
Extract Text from Image
Image Editing
Markup
Combine Images
Crop Image
Flip Image
Mask Image
Overlay Image
Overlay Text
Resize Image
Rotate Image/Video
Remove Image Background
GIFs
Make GIF
Make Video from GIF
Add Frame to GIF
Get Frames from Image
Photos
Select Photos
Find Photos
Get Latest Photos
Get Latest Videos
Get Latest Screenshots
Get Latest Bursts
Get Latest Live Photos
Get Last Import
Remove from Photo Album
Delete Photos
Save to Photo Album
Video
Encode Media
Trim Media
Playback
Set Volume
Play/Pause
Seek
Skip Back
Skip Forward
Set Noise Control Mode
Hand Off Playback
Change Playback Destination
Music
Select Music
Find Music
Play Music
Add to Playing Next
Clear Playing Next
Get Current Song
Get Details of Music
Playing Next
Add to Playing Next
Clear Playing Next
Playlists
Add to Playlist
Create Playlist
Get Playlist
Podcasts
Find Podcasts
Follow Podcast
Play Podcast
Get Episodes of Podcast
Get Podcasts from Library
Get Details of Podcast
Get Details of Podcast Episode
Sharing
Share
Share with Apps
Send Message
Send Email
AirDrop
AirDrop
Clipboard
Copy to Clipboard
Get Clipboard
Photos
Post to Shared Album
QR Code
Generate QR Code
Scan QR or Barcode
Documents
Print
Markup
PDFs
Make PDF
Optimize File size of PDF
Split PDF into Pages
Add PDF to Books
Get Text from PDF
File Storage
File
Select File
Move File
Rename File
Save File
Delete Files
Get Link to File
Create Folder
Get Contents of Folder
Get File from Folder
Append to Text File
Archives
Make Archive
Extract Archive
Files
Open File
Get Details of Files
Filter Filters
Rich Text
Make HTML from Rich Text
Make Markdown from Rich Text
Make Rich Text from HTML
Make Rich Text from Markdown
Web
Search Web
Open URLs
Find Giphy GIFs
Safari
Show Web View
Add to Reading List
RSS
Get Items from RSS Feed
Get RSS Feeds from Page
Articles
Filter Articles
Get Details of Article
Get Article using Safari Reader
URLs
URL
Get URLs from Input
Expand URL
Get Component of URL
Web Requests
Get Contents of URL
Get Headers of URL
Web Pages
Get Contents of Web Page
Get Details of Safari Web Page
Run JavaScript on Web Page
* Save to Photo Album should be higher in this list, as it’s required for any images to be saved to Photos at all – this is often a case of confusion where people create GIFs or edit images, but don’t know to save them to Photos.
** The Playing Next category appears to be a duplicate, as the actions contained are also listed in the Music category.
These shortcuts come alongside new actions for Create Entry, Create Audio Entry, and Search Entries for the Journal app on iPhone, which are available in the iOS 18.1 developer beta (and possibly iOS 18, but I didn’t happen to come across them until my phone was updated).
Check out the new shortcuts:
Open my Journal: Opens the deep link into the Journal app for iPhone.
Search my Journal entries: Prompts you to enter a query, then opens to search results for your input text in the Journal app.
Create Journal entry: Prompts you to add enter text, then uses that text to create a simple entry in the Journal app.
Create audio entry: Opens the Journal app to the recording screen so you can create an Audio entry.
Create photo entry: Asks you to select an image from Photos or Files, then some associated text, then attaches it to a new entry in the Journal app.
Create bookmarked entry: Prompts you to enter text and a title for an entry, then adds it to the Journal app marked as “Bookmarked.”
Open Journal settings: Opens the deep link into Settings > Journal where you can adjust features like Journaling Suggestions, whether to lock your journal, and your journaling schedule.
From Jordan Morgan, developer advocate at Superwall:
Let’s add a new file, and call it GetCaffeineIntent.swift:
struct GetCaffeineIntent: AppIntent {
static var title = LocalizedStringResource("Get Caffeine Intake")
static var description = IntentDescription("Shows how much caffeine you've had today.")
func perform() async throws -> some IntentResult {
let store = CaffeineStore.shared
let amount = store.amountIngested
return amount
}
}
This has all of the three things we mentioned above:
It has a title (“Get Caffeine Intake”).
A description of what happens when we use it (“Shows much much caffeine you’ve had today.”)
And, an implementation of that action, vended via the perform function.
However, if we build and run — we’ll get a compiler error:
`Return type of instance method 'perform ()' requires that 'Double' conform to 'IntentResult'`
Looking at the return type, it’s some IntentResult. This is critical to understand to avoid a lot of undue frustration with App Intents. You always return some form of an IntentResult. For example, if your intent just does an action, and has nothing of value to say about that action — you can simply return .result(). You don’t ever return some primitive or domain specific type like we’ve done above.
Ours, though? It would be useful to tell the user how much caffeine they’ve had and return the actual amount, so change the return type to mark the intent to return two things:
An actual Double value of how much caffeine has been consumed.
And, some dialog to speak out their caffeine for the day.
So, instead of some IntentResult, here’s what we need:
func perform() async throws -> some IntentResult & ReturnsValue<Double> & ProvidesDialog {
let store = CaffeineStore.shared
let amount = store.amountIngested
return .result(value: amount,
dialog: .init("You've had \(store.formattedAmount(for: .dailyIntake))."))
}
Each intent’s return type needs to start with some Intent opaque return type, but from there we can also include more specific types. Here, we’ve noted that we return a double value and speak out dialog.
Developers should read the entire post, but I highlighted this portion because it’s fairly poorly documented and incredibly important.
Jordan also covers basic Intent setup, more on the Entity front, and using Siri Tips and Shortcuts Links to bring more visibility to your actions.
For quick access to the main developer video website, sessions by topic or year, and to search, these shortcuts will suit you well:
Show all Apple Developer videos: Opens to the “All Videos” page on the Apple Developer website so you browse the entire scope of what’s offered.
Browse WWDC videos by year: Presents a list of categories from Apple’s developer pages, then reformats the option of your choosing into the URL for that category and opens it.
Browse Apple Developer topics: Presents a list of categories from Apple’s developer pages, then reformats the option of your choosing into the URL for that category and opens it.
Search Apple Developer sessions: Asks you to enter a search query, then URL encodes the result, and opens the results on the web.
AppleScript commands
For folks wanting direct access to features in the Apple Developer app for the Mac, these shortcuts use AppleScript to open the app and trigger keyboard shortcuts for all the main functions – these are great for Stream Deck users:
Discover in the Developer app: Utilizes AppleScript to open the Developer and press Command + 1 to open to the Discover page, where you can see curated categories and recent articles.
Show Bookmarks in the Developer app: Utilizes AppleScript to open the Developer and press Command + 2 to open to the Bookmarks page, where you can see sessions you’ve saved for later.
Open Downloads in the Developer app: Utilizes AppleScript to open the Developer and press Command + 3 to open to the Downloads page, where you can find videos you’ve saved offline.
Continue Watching in the Developer app: Utilizes AppleScript to open the Developer and press Command + 4 to open to the Continue Watching page, where you can resume sessions you’ve already started.
Copy link from Apple Developer: Utilizes AppleScript to open the Developer and press Command + Option + C to copy the URL of the current session to your clipboard.
Copy link at timestamp: Utilizes AppleScript to open the Developer and press Command + Shift + Option + C to copy the URL of the current session at your specific timestamp to your clipboard, so you can jump back to that moment at any time.
Toggle Bookmark status: Utilizes AppleScript to open the Developer and press the Command + / keyboard command that adds a session to your bookmarks (or removes it).
Toggle Watched status: Utilizes AppleScript to open the Developer and press Command + Shift + U to mark the current video as watched (or unwatched).
Session setup
Once you’re ready to watch sessions, these shortcuts make it easy to get set up on your Apple TV or Mac for a first-run, then a second pass to screenshot relevant information, and finally a way to scrape the titles from all the sessions you’ve saved, for referencing later:
Open Developer TV: Opens the Apple Developer app for Apple TV so you can view bookmarks, browse presentations, and search for topics. When run from Mac, maximizes the window to Full Screen.
Prep for session screenshots: Resizes the Apple Developer window to the second screen at my preferred size for taking screenshots, then opens Craft and centers it on the main display.
Scrape session titles: For a given list of Apple Developer session URLs, this will iterate through each, grabbing the name of the webpage, cleaning it up, and reformatting it into a Markdown list before copying that to your clipboard.
Today I just happened to stumble across the Apple Developer page for “Shortcuts for developers”, which was first launched in July 2023, that Apple designed as a landing page for all things, well, Shortcuts and development for it.
Here’s how Apple pitches Shortcuts here:
Increase your app’s surface area and help users quickly access the most important views and actions in your app. With no user setup required, App Shortcuts are available as soon as your app is installed in iOS, iPadOS, visionOS, or watchOS and can be run from Spotlight, the Home Screen, the Shortcuts app, or even by using your voice with Siri.
Underneath that, Apple calls attention to App Intents, which they describe this way:
Enable shortcuts with App Intents, a Swift-only framework designed to make it faster and easier to build great actions that people can access throughout the system.
The page also links to the documentation to help you begin implementing App Intents with these starting points:
I’m glad Apple has made this resource for developers unfamiliar with Shortcuts and App Intents, as it’s a clear jumping-off point while emphasizing the value of Shortcuts and what apps can enable for their users.
I hope to see this page updated for App Intents in iOS 18 and Apple Intelligence, clarifying the connection between what you can do with Siri and how it’s all going to be available in the Shortcuts app as well.
In order to model this in an App Entity using parameterSummary, we would then need a different summary for every task type (“automatic”, “specific”, “next on page”, “next in category”), as well as accounting for each chart type. This is 8 combinations.
Now consider another of our widgets: the “Tasks” widget. This lets you choose up to 4 different tasks, and has the same options:
The structure of the Tasks widget, which shows different task selection options based on the task1type, task2type, task3type, task4type values.
In this case, there would be 16 combinations, which really doesn’t scale well. It’s extremely hard to maintain and is inflexible if future changes are needed.
To solve this, I introduced a new type called TaskTypeAppEntity, which encapsulates the four different types (automatic, specific, next on page, next in category) in a single entity:
struct TaskTypeAppEntity: AppEntity, Identifiable {
static let typeDisplayRepresentation: TypeDisplayRepresentation = "Task Type"
let id: TaskTypeAppEntityIdentifier
var title: String
var displayRepresentation: DisplayRepresentation {
.init(title: .init(stringLiteral: title))
}
static let defaultQuery = TaskTypeAppEntityQuery()
}
enum TaskTypeAppEntityIdentifier {
case automatic
case task(TaskID)
case category(CategoryID)
case page(PageID)
}
When building the defaultQuery, it’s just a case of including all of the options in that query. You can even group it into separate sections:
The new task type selection screen once the different options are flattened into a single App Entity.
This is more code than I’ve probably ever shared on my site, but this method is incredible for apps with more complex data models to build a clean App Intents experience for their users – all App Intents developers should take a look and see if they can use this.
I’ll definitely be recommending this to my clients – I had to save this here on my blog, as well as rewrite my own headline to clarify that this is useful beyond widget configurations as well as beyond moving from “INIntents” to App Intents.
If you’ve just purchased the updated Beats Pill from Apple, you’re probably looking for how to turn it off, since Apple oddly left it out of the packaging and user guide. Here’s how:
Press and hold the Power button for more than 1 second, but less than 3 seconds.
One of my main points of contention with the new Pill is how few buttons it has for its plethora of controls. For instance, the power button has six different functions.
Power on/off (hold for more than 0.8 seconds, but less than three)
Pairing (press and hold for over three seconds)
Voice Assistant (double tap)
Battery status (quick tap, less than 0.8 seconds)
Change charging direction (triple tap)
USB-C audio pass through (press and hold button while plugging in USB-C cable)
This is way too much for one button.
That’s not even including the six functions for the center button as well, which the linked story covers.
Further in the same piece, Brad also highlighted the same issue that I found – Apple never actually tells you how to power down your Beats Pill, anywhere. Here he is again talking about the physical manual:
When you unfold it, there are a few quick controls laid out, but it doesn’t even mention how to turn the speaker off, which as I found out, is more complex than it needs to be.
Instructions for how to actually turn off your Beats Pill don’t come in the manual that comes with the speaker, are not in the user guide linked on the QR code that’s on the manual, and are nowhere online except Brad’s article – hence why I’m writing a dedicated piece to help people discover this explicit.
To help further illustrate the power instructions, I created this custom graphic from an amalgamation of the images in the User Guide and SF Symbols.
Also, to illustrate the USB-C instructions, I created this second graphic as well with SF Symbols – Apple probably wouldn’t label this with the Losslessbadge or the USB-C image, but I find it helpful for remembering those functions.
After reading reviews on Tuesday about Apple’s new Beats Pill, I went and tried to pick one up – only to realize it was on sale on Thursday. So today, I went to the Apple Store and actually picked one up.
The Beats Pill’s streamlined design features a 20-degree upward tilt to help deliver sound waves towards your head, away from objects that might obstruct them.
Enjoy up to 24 hours of battery life for all-day continuous playback.³ The Beats Pill can even be used to charge your phone and other devices via USB-C cable.
IP67-rated for dust and water resistance for outstanding durability on the go.²
Designed for ultimate portability, including the removable lanyard and soft-grip silicone backing.
Dual compatibility for both iOS and Android enables one-touch Bluetooth® pairing, automatic pairing across your other devices, and Find My or Find My Device.
From Jared Newman at FastCompany, after interviewing yours truly last week:
Matthew Cassinelli, a writer and consultant on iOS automation who worked on the app that eventually became Shortcuts, believes Apple will succeed because of all the years it spent laying the ground work.
After all, App Intents aren’t just about Siri. They’re also an essential building block for other iOS features, such as widgets, Spotlight search, Live Activities, and iOS 18’s expanded Control Center and lock screen toggles. Developers that support these features will also be expanding Siri by extension.
“They’ve done a very good job of seeding this to developers in a way that’s key to their entire ecosystem and how it runs,” Cassinelli says.
Apple’s also going to benefit from the ongoing AI hype wave. Instead of just promoting a better version of Siri, Apple can wrap it in the broader “Apple Intelligence” branding, with a marketing push aimed at both users and app makers. Cassinelli says he’s already seeing a change in attitude toward App Intents from the latter group.
“They announced Apple Intelligence stuff, and since then, every app developer’s like, ‘Alright, I’m doing this now,’” he says.
And:
As Matthew Cassinelli points out, those companies must also still figure out how developers can monetize virtual assistant support. That’s not a problem for Apple, whose developers are simply building out Siri support atop their existing apps.
“Apple’s ability just to let you make money on the App Store alone, it’s very powerful compared to something like Google Assistant or Alexa, where they’d have to start a whole business just for that sort of thing,” he says.
Check out the rest of the piece for quotes from developers as well.
Developers, indies, and organizations – let’s work together to make your App Intents and Apple Intelligence implementation the best!
I’m now available for consultation work on Apple’s App Intents and Shortcuts implementations for app developer of all sizes, whether you’re a team, an indie dev, or part of a larger organization.
With my 10+ years of experience working in Shortcuts (as a former member of the Workflow team), I’ve built out a consulting process to educate, strategize, design, and build on top of a complete App Intents and Shortcuts implementation for your app.
I’ll explain the history of Shortcuts, how we got from Workflow to Apple Intelligence, walk you through the developer sessions, recommend intents specific to your needs, share personalized feedback on your implementations, provide extensive documentation and copywriting, and even create any number of custom workflows on top of what we build together, extending your app to the ecosystem with hand-built third-party integrations.
Plus, we can do everything mentioned above together, or just a few parts – let’s schedule a one-hour meeting and discuss what you need.
Pricing depends varies per engagement, but I’m happy to customize anything depending on your goals and timeline.
Now is the time – let’s build App Intents for the rest of us.
As discovered by Scotty Jackson, there are new Reminders actions in the Shortcuts app in iOS 18 developer beta 2. The new actions are “Create Reminder” and “Open Any List,” both of which still seem to be in testing.
Create Reminder and Open Any List are both updated versions—but distinct from—their respective counterparts “Add New Reminder” and “Open Reminders List” (and “Open Smart List).
New Reminders actions
Create Reminder has more options than Add New Reminder, including new fields for List Section (within the Target List) and Subtasks. Plus, the action changes the Alert field to separate All-Day and Due Date fields, losing the “When Messaging,” “When I Leave,” and “When I Arrive” options in the process as well – it remains to be seen if these will be restored somehow. There’s also a new “Assign Reminder” field that does not current return any results in my testing.
Open Any List provides a single variable for “Any List” which lets you pick a specific List from Reminders and, when run, open it – this appears to be a replacement for the separate “Open Reminders List” action (which came first) and the “Open Smart List” action (which came later), combining the two under a more-logical single action going forward. However, this action is also oddly-implemented in both its name and the fact that it includes an “Open When Run” toggle that’s not functional and should be built-in to the action, not available as a toggle.
Neither has the Parameter Summary feature implemented, so both actions do not read like a natural sentence with a parameter inline and instead hide it below the fold – this goes against Apple’s own recommendations, so it appears that both actions are an earlier implementation and hopefully will change in future betas.
App Intents for Apple teams too
Overall, seeing updates to these Reminders actions is a good sign for the Shortcuts ecosystem, as it’s the first signal that Apple is updating their native Shortcuts actions with App Intents-based replacements in iOS 18.
Since the inception of many of these actions in Workflow when Shortcuts was a third-party app, many actions have been built on longstanding external-facing developer APIs (hence actions like “Get Upcoming Reminders”) and then later custom intents from within teams at Apple – they either stayed the same as the Workflow actions, or got piece-by-piece updates for new features each year like Tags in Reminders.
However, as is the nature of intents development, Apple also has tried not to break anything or remove features that are being used in existing shortcuts – but rather than deprecating actions over time, they either have been updated-in-place, added as separate actions (like “Open Smart List”), or simply not implemented in Shortcuts at all.
Now, it appears that we’re seeing the first evidence of an Apple team seeding new actions in betas, hopefully testing and iterating on them, and then likely replacing the Workflow- and custom intents-era actions with modern App Intents actions that can be extended with new features more easily and updated going forward.
More to come
I hope Apple eventually replaces all the native Shortcuts actions from the Workflow days with full-featured, modern App Intents actions for all of their apps, as well as adopt their own recommendation that “Everything in your app should be an App Intent.”
That’s because developers will take absolutely notice when Apple leads the way, and, furthermore, many Shortcuts users have been waiting (somewhat) patiently for about 7 years for Apple to take the promise of Workflow and run with it for their own apps in a big way.
Apple Intelligence and App Intents for third-party developers will be fantastic, but Apple’s own teams need best-in-class integrations as well – hopefully these Reminders actions are a sign of more to come.
I’ll update this post if these Reminders get updates in future releases, and check back for more action coverage after each new developer beta this summer.
Here’s the description from Apple (with line breaks added for readability):
Learn about improvements and refinements to App Intents, and discover how this framework can help you expose your app’s functionality to Siri and all-new features.
We’ll show you how to make your entities more meaningful to the platform with the Transferable API, File Representations, new IntentFile APIs, and Spotlight Indexing, opening up powerful functionality in Siri and the Shortcuts app.
Empower your intents to take people deep into your app with URL Representable Entities.
Explore new techniques to model your entities and intents with new APIs for error handling and union values.
Before I begin, this video also also assumes you’re already aware of the basics of the App Intents API, building off the main sessions from 2022 and 2023 (which you should watch if you haven’t yet).
In many ways, the extended lifespan of this API is why Apple has released other new videos to explain why you should add App Intents and how to design them (which I’ll cover in future posts) in an updated context for 2024.
In reintroducing App Intents in 2024, developer Kenny York starts off the video emphasizing the variety of experience App Intents already powers, including Shortcuts, Spotlight, widgets, the Action button, and the new Apple Pencil squeeze (introduced in May 2024).
This year, those experiences now extend to includes Apple Intelligence and the new Controls experience in Control Center (both covered in future posts).
The App Intents framework itself is also expanding beyond intents and App Shortcuts to emphasize entities, helping the system understand and use them directly when invoked. And finally, developer improvements are designed at making it easier to make App Intents, so they’re not as hard to implement now that developers will be building a lot more.
By default, opening Spotlight on your device already creates a rich experience showing content you might want to see, from suggested apps and actions to a daily summary or recent searches.
Spotlight also helps you search for specific content inside an app (including a new preview experience that I’ve yet to discover how to design on your own).
Last year, Apple improved this experience by allowing developers to create App Shortcuts specifically for the Spotlight experience (as covered in their “Explore enhancement to App Intents” video) and emphasize the most important actions (and entities) from their apps.
IndexedEntity protocol
In order to take this even further, Apple has introduced a new IndexedEntity protocol that lets you index the entities from an app, making it available to the new powerful semantic search capabilities of your device new in iOS 18.
With this protocol, developers can create an index of all the items (or entities) from their app, giving each item a set of attributes, (including keywords), and even assign them priority to match features like favorites lists.
Then, by handing it off to Spotlight when the app is launched or updated, all that data is indexed and searchable, plus more easily matched when queried using natural language.
In order to add an extra layer of meaning beyond the App Entity and attributes, Apple is allowing developers to declare how their entities can be converted into new formats, which formats their App Intents can accept, and also securely provide files themselves in lieu of direct access.
Transferable AppEntity
Another issue with expanding the AppEntity protocol and making all the items in your apps accessible via AppIntents is what ends up being a format issue – how does data from one app properly convert into the right data for another app? In their example, a trail entity from their app could become a PDF in another when passed via App Intents.
To help with this, Apple now allows developers to extend their entities with Transferable, an API introduced in 2022 designed to convert formats from one type to another.
With Transferable, developers can actually specific which types of data they want their entity to convert to, including fallback options sorted by priority.
For anyone with a long history of Shortcuts and its original independent version Workflow, this acts like a modern-day version of the Content Graph engine, which is what powered Workflows inputs/outputs to intelligently convert data types when needed. Now, it’s built into App Intents, powered by a Swift API, and developers have control over which data types it gets converted into and when.
Improvements to IntentFile
On the flip side, when an App Intent is designed to receive an entity, developers can specify which of these content types it can support, what’s available from the other app, and which one to pick.
Plus, when that content type isn’t supported, it can give the app the original URL of the content and pass it to Transferable for conversion.
FileEntity
Finally, Apple has also addressed limitations for document-based apps where the file itself is the entity, and not a representation of an object from your database – in these cases, the FileEntity API is needed to operate on the actual file itself.
In these cases, FileEntity allows Siri and Shortcuts to securely access the files, perform the operation, and pass it along to the next step using an ID instead of the file itself.
With this method, apps can provide information about their files and other apps can operate on those files, but handled properly through App Intents instead of direct access.
Beyond making it easier to meaningfully search for content within your app, Apple is also improving deep linking to those specific entities using Universal Links.
While the slide above is in full developer-speak, what Apple has introduced this year is the ability to generate and open links to any item, page, or action within your app. In essence, every part of every app is now represented by a URL – which makes sense, since it is a universal resource locator.
Now, App Intents allows URL to redirect to everything section, every thing you see, and every action you can take in your app.
In practice, this will mostly be used by developers as the universal method of accessing said item from another action step in Shortcuts, another app when it is requesting Transferable operate on an entity, or specifying to open content after creating it anew.
However, I’m fascinated by the larger implications here – everything in every app is linkable now… I’ll have to do more research to really flesh out what this means for apps that are also web services.
Now that developers can use App Intents to index all of their entities, make them meaningful to Spotlight and Siri, and interlink between any entity, action, or section, there’s a lot of App Intents work to be done by developers.
Thankfully, Apple already designed App Intents from the ground up to make the entire framework functional in code only (compared to the Xcode UI for custom intents), extendable by only requiring you to add onto existing App Intents implementations, and, in most cases, already in-use for features like widgets.
This year, they’re building off the improvements of the last few years in two specific ways – a new UnionValue macro that makes multiple parameters or properties representable under one type, and the ability for Xcode 16.0 to generate title strings automatically.
UnionValue
For developers, UnionValue means they can let users choose between more complex data types in a union under a new, single value – Apple describes it as an “or” parameter.
Generated titles
And for developers creating lots of App Intents can avoid duplicating unnecessary work in designing titles for their intents that are already implied in the struct’s property – instead of adding the title “Location” to a location, Xcode can simply figure it out. And when developers still want that control to specify a unique title, they can add it back in.
Framework improvements
Finally, in order to make App Intents work better with the new changes to App Entities, Apple has removed limitations on the interaction between the two and lets you define app entities in a framework and reference them from your app and extension targets.
However, it should be noted that Apple hasn’t expanded this to libraries outside of a framework – for now, libraries are not supported.
Apple also explicitly mentioned that not all the changes to App Intents are covered in this video, as some changes are directly relevant to other sessions on Apple Intelligence and Controls, and those changes are only covered in those sessions.
Here’s the full list from the graphic above for any developers searching through the App Intents documentation for everything new:
This year, Apple has expanded the App Intents framework to act as an all-encompassing tool from defining intents and building App Shortcuts on top of them to fully indexing all the entities from your apps, making them meaningful, linkable, and transferable, and in general making it easier to develop, integrate, and maintain a balanced App Intents integration with less effort or roadblocks.
Developers For developers, this means making your app, its objects, capabilities, and interface accessible to the rest of the ecosystem in a thoughtful and secure way.
Users For users, this means all your apps have added a layer that Apple Intelligence can understand, you can control from within the Shortcuts app, and will continue to be developer against for years to come.
Apple For Apple, this means that Siri and your devices can finally understand what your apps can do, what you’ve created with them, and the connections between it all – and in a private way that doesn’t allow apps, users, or even themselves to abuse that access.
In many ways, App Intents is a new operating system for apps themselves, letting them talk to the system, tell it what’s possible, and be ready for anything when the time comes.
Apple is also pushing App Intents in a huge way – I’ll cover it in the next session, but they’re declaring “Everything in your app should be an App Intent.” That means this is the API for how your apps work now, and will be going forward.
For most Apple users, they’ll simply experience App Intents by way of better Spotlight, the enhanced Siri integrations, and via features like widgets or the Action button, never having heard of this technology powering it all.
But behind the scenes, App Intents will be working hard to make sure everything functions properly – Siri will have gotten better “overnight,” while Apple has been building up App Intents for years to get us to here.
Developers, are you looking for help with your App Intents implementation?
Today, Apple released Final Cut Pro for iPad 2, transforming iPad into an even more powerful production studio, and Final Cut Pro for Mac 10.8 with important artificial intelligence-driven organizational updates that make workflows more efficient.
On the Live Multicam feature and Final Cut Camera app:
With the all-new Live Multicam feature, users can wirelessly connect and preview up to four cameras at once, all in one place. Using a combination of iPhone and iPad devices, users will have a director’s view of each camera and manual control of their preferences using the all-new pro app Final Cut Camera. Final Cut Camera powers Live Multicam and also works as a standalone recording app on iPhone and iPad, with the ability to adjust settings like manual focus, shutter speed, ISO, and more.
On Final Cut Pro for iPad 2:
For more storage flexibility while working with large files, Final Cut Pro for iPad 2 now supports external projects, a top-requested feature.2 Users can create new projects or open an existing project without taking up internal storage space.
On Final Cut Pro for Mac 10.8:
Enhance Light and Color automatically adjusts video for color correction, while Smooth Slo-Mo intelligently blends video frames for the highest-quality movement. Also, new organizational tools improve efficiencies in post-production workflows, including custom names for color corrections and video effects in the inspector; the ability to search for and navigate to clips with missing media or effects in the timeline index; and text-based timeline search with information like reel, scene, camera angle, and more
From Julia Roggatz at The Browser Company (and formerly of Muse) in a thread on Twitter:
Arc on iPad is here! 🚀
(Well, sort of…)
We heard you loud and clear, and today’s update of Arc Search will finally be usable on your iPad.
Arc Search was designed for your phone, and we’ll continue to make it our core mission to bring you the best browsing experience on the go.
But so many of you have asked for a better experience on the iPad as well, and we don’t want to keep you waiting.
Arc Search now comes with an iPad version that has all your favorite features and a browsing experience that’s streamlined for a larger screen.
[…]
While we hope to eventually explore a version of “Arc on iPad” that’s reimagined for the tablet form factor from the ground up, we wanted to give you something that we can build fast, and that you can use today.
I honestly love this approach and wish more apps would do this. iPad users are willing to wait for better versions, but leaving us without anything now means your app can’t be part of our workflow at all.
