From 9to5Mac, covering a report from The Information:
According to the report, we should expect to see Siri gain deeper integration with Apple’s automation tool Shortcuts as part of iOS 18.
The Information says Apple’s Siri team “plans to incorporate language models to let users of the voice assistant automate complex tasks in ways they currently cannot,” like turning the last five photos taken into a GIF with a voice command.
“The new capability is related to Apple’s Shortcuts app, which lets users manually program a series of actions using different apps and is expected to be released alongside a new version of the iPhone’s operating system next year,” per The Information.
Interesting, might check this Shortcuts stuff out.
After the tragic passing of beloved developer Alex Hay last March, the question of what would happen to his apps Toolbox Pro, Nautomate for Notion, and Logger for Shortcuts remained uncertain – Alex was a fierce advocate for the Shortcuts developer community and his apps plus public repositories served as inspiration & a resource for many of the Shortcuts apps available today.
Today, I’m honored to share the news that my friend Rosemary Orchard and her development company Snailed It (with David Stephens and Dom Chester) have been entrusted by the Hay family to take over development of the late Alex’s apps. The Snailed It team is already responsible for WhenWorks, FocusCuts, and development of Pushcut, and will continue work on Toolbox Pro, Nautomate, and Logger after making the necessary development transitions.
I am confident that Rosemary and the team will honor Alex’s memory and his contributions to the Shortcuts community, and I am glad to know I can continue to rely on Alex’s work – often times his apps felt like they were built for exactly me, which I always greatly appreciated.
In many ways, Alex was the developer advocate I could not be, and I truly appreciate how he built up the Shortcuts community. I had the pleasure of video chatting with him a few times, but I am truly sad I didn’t get to spend time with him in person.
I see signs of Alex’s contributions every day in my work, and I’m endlessly grateful that he put so much of himself out into this world.
Secondly, App Shortcuts are now supported on HomePod.
This requires an App Shortcut enabled app to be installed on a companion iOS or iPadOS device. Unlike running on iOS or iPadOS, where App Shortcuts can result in launching the app, on HomePod, App Shortcuts may not launch the app at all.
For example, by simply having my Demo app on my iPhone, I can say to my HomePod, “Hey, Siri. Summarize my groceries list with Demo.” HomePod support for App Shortcuts is available from version 16.2 and newer.
When designing App Shortcuts for HomePod, consider that it is a voice-only device. Make sure you are returning clear and concise dialog to be spoken by Siri, since there won’t be any corresponding visuals.
The IntentDialog class has an initializer that takes both a “full” and “supporting” string. The HomePod will use the “full” dialog, while visual devices like iPhone will use the alternate “supporting” dialog. This allows you to be more brief when a user can reference a visual output, but remain fully descriptive if no screen is available.
I’m glad Apple (and developers) are putting in the work to match the dialog to the device – and that HomePod is getting even better Shortcuts support this year.
I’m pretty sure it’s new that watchOS-only apps can now enable shortcuts to the shortcuts app without having an iOS companion app ?✨
I can indeed confirm this is new – Apple mentions it in their “Spotlight your app with App Shortcuts” developer session from WWDC’23. Here’s what they say in the video:
App Shortcuts now work on Apple Watch, but you should be aware of some limitations to get the best possible experience. App Shortcuts on Apple Watch must come from a watchOS app installed on the device itself. App Shortcuts from a paired iOS device cannot be run on the Watch. This means that if you have an iOS app installed on iPhone, but not the corresponding watchOS app, your App Shortcuts will not work from watchOS.
Flexible matching with Siri is not available on Apple Watch, so phrases must be spoken exactly. Apple Watch support was introduced in watchOS 9.2 and is available on all devices running that version or later.
App Shortcuts are also now discoverable on Apple Watch. Just like on iOS, the Shortcuts app on watchOS prominently features App Shortcuts available on the device.
Just to be clear, App Shortcuts will work on both standalone watchOS apps and companion watchOS apps.
Plus, you will run into issues if you’re trying to run App Shortcuts from the watch and the corresponding iPhone app doesn’t have a Watch app – developers, get to building your watch apps!
New in the fourth developer beta and second public beta of iOS 17, Apple has added an “Open Camera” action for the Camera app that allows users to pick between camera modes and immediately open to any preset using Shortcuts.
The new “Open Camera” action can be set to switch between the following modes:
Photo
Selfie
Video
Portrait
Portrait Selfie
Cinematic
Slo-Mo
Time-Lapse
Pano
I’ve built a folder of shortcuts for each action, available now as part of my membership program in pre-release ahead of iOS 17.
These actions are highly welcome to the set of Media actions available by default in Shortcuts, as well as my collection of photography shortcuts built off of them.
In the betas for iOS and iPadOS 17, Apple has updated the Automations feature in Shortcuts so that triggers can “Run Immediately” rather than the default “Run After Confirmation.”
In effect, Automations are, well, actually automatic now, rather than triggers that require manual confirmation – here’s the breakdown:
Updated Automations
In Shortcuts, all Automation types—except Arrive, Leave, and Before I Commute in the current betas—now have a new “Run Immediately” option.
Update: As of iOS developer beta 5, Location automations also run immediately!
That means automations from Messages, Email, Wi-Fi, and Bluetooth now join the rest of the Automation triggers as true automations, alongside the new Transaction automation for Apple Wallet and the Display automation & Stage Manager automations on iPadOS.
For the Wi-Fi automation trigger, there’s also an additional dialog that lets you toggle whether to Run After Connection Interruption, with the following explainer text beneath:
If you reconnect to a Wi-Fi network within 3 minutes of being disconnected, this automation will not run again.
Subtle changes to Automations
To ensure user safety, Apple also requires that these automation types always notify you—the “Notify When Run” toggle does not appear if you switch to “Run Immediately”—so that you know for sure when something has happened on your device. Personally, I think this is a worthy trade-off – otherwise, how would you ever know that the Automation triggered successfully?
Further, the confusingly-named “Don’t Ask Before Running?” prompt has been renamed across all Automations to “Run After Confirmation,” making it much clearer what’s actually happening with each option.
Finally, you might also notice that Apple removed a step when initially creating an automation. Previously, you’d always have to pick between Personal Automations and Home Automations.
Now, the Add button starts you off in a Personal Automation by default—with a new Search field at the top as well.
Home Automations are simply found at the bottom of the list as an alternate option (you can still create Automations in the Home app as always).
Location automations
It’s not clear currently if the Location automations for Arrive, Leave, and Before I Commute will be exempt from true automation capabilities in the the final release of iOS and iPadOS 17.
That’s because, in the first developer beta, the Location automations were truly automatic – but, crucially, that option was removed and has not been available in any subsequent betas.
Hopefully we see the return of this capability in future betas, and Shortcuts can fully shed the misnomer status this feature has earned from being named “Automations” without having all of the triggers be truly automatic – otherwise, I’ll have to rename this piece.
True automation starts now
Automations actually being automatic will be a big deal for Shortcuts users in iOS 17 – in many ways, the feature has been hamstrung for its entire existence, because many users try it out, then get disappointed when it’s not truly automation.
Going forward, Automations will be a pillar for new users and pro users alike to get a lot of value from building in Shortcuts – all without even doing anything themselves, since everything happens automatically.
In the iOS and iPadOS 17 betas, Apple has introduced three new Automation triggers you can use in the Shortcuts app – Transaction, Display, and Stage Manager.
Here’s how Apple’s tip text describes each:
Transaction (Ex. When I tap a Wallet Card or Pass)
Display (Ex. When My Display Connects)
Stage Manager (Ex. When Stage Manager Is Turned On)
Transaction automations
Transactions work with Payment, Transit, Access, or Identity passes, plus you can select which card, which category of payment, or even specific Merchants.
The automation starts with an optional “Receive Transaction As Input”, allowing you to extract details about the transaction, including the card or pass, merchant, and amount – this is an incredibly powerful trigger when combined with things like expense tracking, budgeting, or travel apps.
Display automations
The Display automation works with the iPad Pro’s external display support, allowing actions to be kicked off when you plug in or disconnect a monitor cable.
The automation starts with an optional “Receive Display As Input” that allows you to extract the manufacturing Name of the device and adjust your flow accordingly — I have both Dell and LG monitors in separate rooms in my house, so I could use an If action to change my shortcut behavior depending on which room I’m in.
Stage Manager automations
Finally, the Stage Manager automation works with the system feature being toggled on or off, either from Settings, the Control Center widget, or the Shortcuts actions themselves that can Toggle Stage Manager in any shortcut.
I haven’t personally implemented this automation for my own setup yet, but I’m consider simply using Open App multiple times to “set the stage” each time I activate it with my most common apps – but I’ll have to test more during the beta season.
Automate away
Overall, these three new automations are highly welcome. The Transaction automation will surely get the most use and can apply to almost every iPhone user, and the iPad automations for Displays and Stage Manager are great additions for iPad productivity – there’s much value to be captured with all three of these.
Plus, each automation can Run Immediately (instead of requiring you to Run After Confirmation), meaning they’re true background automations – very nice.
I also think the presence of Display and Stage Manager are either a hint or justification for Automations in Shortcuts for Mac – I hope Mac users get the opportunity to automate their devices with these triggers in the future.
All-time superstar footballer Lionel Messi is making his debut for Inter Miami as part of Major League Soccer tonight, marking a historic moment for the sport in America and for fans worldwide — plus, it’s a big deal for Apple, having bought streaming rights for the next 10 years.
In the future, for matches where you won’t always have a dedicated URL ready, you can tune into MLS Season Pass using my shortcut that deep links to the in-app page. Or, for all the other leagues, you can use my Open Apple TV Sports shortcut – both of these work great on iPad in particular.
While I’m a Timbers fan myself having grown up in Portland, Oregon (?), it’s undoubtedly exciting to see such talent come to the United States – it’ll be huge for the sport after years of slow-but-steady growth here at home.
Apple Inc. is quietly working on artificial intelligence tools that could challenge those of OpenAI Inc., Alphabet Inc.’s Google and others, but the company has yet to devise a clear strategy for releasing the technology to consumers.
The iPhone maker has built its own framework to create large language models — the AI-based systems at the heart of new offerings like ChatGPT and Google’s Bard — according to people with knowledge of the efforts. With that foundation, known as “Ajax,” Apple also has created a chatbot service that some engineers call “Apple GPT.”
I hope Apple takes advantage of Shortcuts and the App Intents framework to make Siri actually do things for you, not just tell you information. If Ajax doesn’t utilize Shortcuts, they’re missing out on the biggest competitive advantage they already have built out.
Great for accessibility and I think will be popular with everyone. Lots of thanks to Apple for listening, and thanks for highlighting @zollotech https://t.co/Ake2MHsNbm
After mentioning that Siri’s “You need to unlock your iPhone first” was a big problem for quadriplegics like himself, Colin Hughes is now celebrating Apple’s update to Siri that lets AirPods act as authentication – when you’re wearing them, Siri requests will work without requiring you to unlock your device.
There’s a new Siri authentication system on AirPods where if your device is unlocked while wearing your AirPods, it will be considered authenticated as long as they are in range (or until you change the output audio device on the system).
So basically when my carer puts my AirPods in my ears, then as long as my iPhone is unlocked when she does so, even if it locks afterwards I can still access my messages, calendar events and more.
With iOS17 you can now set up recurrent AppleCash payments like your kid’s allowance – no more forgetting to pay and being charged a penalty by the kid ?
Once you select your contact to send or request money, there’s a “Recurring Payment” option that lets you pick a start date, repeating schedule, and memo for the payment.
One example where this is handy is scheduling your monthly portion of rent to a single account before sending to a landlord, as I do with my partner – or as Carolina mentioned, allowance for a kid. Very nice ?
Today, Apple released the first public beta of iOS 17, iPadOS 17, watchOS 10, and macOS Sonoma – and with it comes a new set of actions for the Shortcuts app, many of which I first discovered from u/iBanks3’s roundup.
On iOS 17, Shortcuts has new actions for Books, Clock, Scan Documents, Fitness, Freeform, Mail, Photos, Phone, Settings, Tips, and the new Print Center app. Plus, there are also hints of new Camera actions teased in WWDC videos (as seen by u/Portatort).
Here’s the full breakdown of all the actions – those marked with an asterisk (*) work on macOS as well:
For today July 11 and tomorrow July 12, folks who buy apps on Apple’s platforms can enjoy discounts on over 100 apps made by independent app developers for Indie Dev Sales, an alternate event put on during Amazon’s Prime Day that’s organized by Signals for HomeKit developer Matt Corey.
I love the idea of Indie Dev Sales because it celebrates the hard-working independent app developers who put time, love, and care into their apps in ways that we often don’t see from apps built by larger corporations.
This event helps raise the profile of indie apps, provides helpful discounts for users, and serves as inspiration for other would-be app developers – a great way to consider spending your money on a day like today.
Here’s the full list of 113 apps (as of writing) – make sure to check out the Indie Dev Sales page to see the app links and categories, prices and discounts, plus links back to the indie developers that you can follow online:
This update includes enhancements to existing actions and reliability improvements to editing and running shortcuts.
Updated Actions
Send Message now supports images and attachments when “Show When Run” is off
Make Rich Text From Markdown now displays a preview instead of an HTML file on macOS
Ask For Input now correctly uses the “Default Date” when asking for dates
Calculate now correctly computes Modulus operations for decimal numbers
Choose From List now supports “Select All Initially” when a dictionary is passed in as input, and works from Siri on watchOS when “Select Multiple” is on
Open URL correctly handles URLs that contain variables, and now works from the Action Button on watchOS
Take Photo runs more reliably when “Show Camera Preview” is off
This update includes new actions, enhancements to existing actions, and reliability improvements to the Shortcuts editor. A new option in the Shortcut editor allows selecting an app icon as an alternative to selecting a color and glyph for the shortcut.
New Actions
Find Books searches your library for books or PDFs matching a given title, author, genre, purchase date, or content type on iOS and macOS
Intercom broadcasts announcements to your home on iOS and watchOS
Set VPN connects, disconnects, or sets the On Demand setting for any VPN on iOS and macOS
Set AirDrop Receiving sets the AirDrop Receiving preference to Everyone, Contacts Only, or Nobody on iOS and macOS
Set Stage Manager enables or disables and changes the settings for Stage Manager on iPadOS and macOS
Set Announce Notifications toggles the Announce Notifications setting on iOS and watchOS
Set Silence Unknown Callers toggles the Silence Unknown Callers setting on iOS
Set Night Shift and Set True Tone toggle the display settings on iOS and macOS
Set Always-On Display toggles the Always-On Display on supported iOS devices
Find Apps finds and filters apps installed on macOS
Find Displays finds and filters displays on macOS
Shut Down, Restart, and Lock Display are now available on iOS and macOS
Log Out, Sleep and Put Display to Sleep are now available on macOS
Updated Actions
For those building custom shortcuts, some actions have been updated:
Move Window now allows moving a window to a display on macOS
Ask for Input has a new “Allow Multiple Lines” option, which can be disabled for single-line text input
Find Photos can now find screen recordings
Find Contacts now filters by birthday more accurately
Get Device Details can now get the OS build number
Get Device Details now rounds numbers, including the current battery level, volume, and brightness, to remove unnecessary decimal digits
Set Playback Destination now works when run from Siri
Play Music once again resumes currently playing music if no music is passed as input
Run AppleScript can now produce dictionaries as output from AppleScript records
Nothing now produces a variable with zero items, instead of producing no variable at all
Along with the first beta release of iOS 16.4, Apple has published a new beta protocol for the App Intents APIs that let developers foreground their process if user interaction is required.
Here’s the description of the protocol from Apple’s website:
The ForegroundContinuableIntent protocol represents intents which begin their work with the app in the background but may request to continue in the foreground.
This protocol will be in beta for the iOS 16.4 developer cycle, so app developers can begin building with this now – everyday users won’t see the effects of this in their apps until iOS 16.4 releases in full after Apple’s testing cycle.
Today, Apple posted the first developer beta for iOS 16.4, iPadOS 16.4, macOS 13.4, and watchOS 9.4 — as is tradition, the Shortcuts community rushed to download the new betas and checked for new actions available in the app.
This update includes the ability to disable a shortcut from being run while the device is locked, enhancements to existing actions, and reliability improvements to editing and running shortcuts.
Updated Actions
For those building custom shortcuts, some actions have been updated:
Find Reminders can now filter reminders due in the next or previous week
Log Workout now supports Ask Each Time for workout duration
Log Health Sample now supports Ask Each Time for values on watchOS
Find Health Samples now supports searching for health types in the editor
Get Contents of URL can now use variables for the file parameter for POST or PUT requests
Configure Focus Filter actions now correctly set the Focus Filter on iPhone when run from watchOS
Show Notes Folder action now correctly appears in the editor
Following the release of iOS 16.2 on Tuesday, Apple published an update in their Support knowledge base detailing new features and issues fixed in Shortcuts in the last few releases since iOS 16, iPadOS 16, watchOS 9, and macOS Ventura launched.
The main headline features for iOS 16.2 are new actions for Books and changes to the Wallpaper actions:
This 9to5Mac article explains “What you can assign to the Action button, secondary actions, third-party apps” with a list of “Native options for the Action button can start/launch”:
Workout
Stopwatch
Waypoint
Backtrack
Dive
Flashlight
Shortcut (from Shortcuts app)
None
However, being able to use “Shortcut (from the Shortcuts app)” really means a lot more than that one line.
Back at WWDC ’22 during the keynote presentation, Apple’s Senior Director of Siri and Language Technologies, Robby Walker, said this line about the Shortcuts app (at ~0:16:30 in):
“There are over 15,000 apps that work with Siri using SiriKit and Siri Shortcuts.”
That means the Action button can be set to any shortcut using any of the:
• 15,000+ apps with Shortcuts support,
• 300+ native actions available for custom shortcuts,
• automatically-generated App Shortcuts from your apps
As I’ll explain in an upcoming post, the Scripting actions in Shortcuts can also be used to make your chosen Action button shortcut fully dynamic. With a bit of setup in Shortcuts, you can program one shortcut to serve multiple purposes depending on the time of day, day of week, Focus mode, or any other condition that can be detected — sign up here for my newsletter to get updated once that post goes live.
I’m super excited to test this out and get my hands on an Apple Watch Ultra — I missed the preorder, however, so I suppose I’ll be outside my local Apple Store this Friday…
