Apple Intelligence has not had the best year so far, but if you think Apple is giving up, you’re wrong. It has big plans and is moving forward with new model training strategies that could vastly improve its AI performance. However, the changes do involve a closer look at your data – if you opt-in.
In a new technical paper from Apple’s Machine Learning Research, “Understanding Aggregate Trends for Apple Intelligence Using Differential Privacy,” Apple outlined new plans for combining data analytics with user data and synthetic data generation to better train the models behind many of Apple Intelligence features.
Some real data
Up to now, Apple’s been training its models on purely synthetic data, which tries to mimic what real data might be like, but there are limitations. In Genmoji’s, for instance, Apple’s use of synthetic data doesn’t always point to how real users engage with the system. From the paper:
“For example, understanding how our models perform when a user requests Genmoji that contain multiple entities (like “dinosaur in a cowboy hat”) helps us improve the responses to those kinds of requests.”
Essentially, if users opt-in, the system can poll the device to see if it has seen a data segment. However, your phone doesn’t respond with the data; instead, it sends back a noisy and anonymized signal, which is apparently enough for Apple’s model to learn.
The process is somewhat different for models that work with longer texts like Writing tools and Summarizations. In this case, Apple uses synthetic models, and then they send a representation of these synthetic models to users who have opted into data analytics.
On the device, the system then performs a comparison that seems to compare these representations against samples of recent emails.
“These most-frequently selected synthetic embeddings can then be used to generate training or testing data, or we can run additional curation steps to further refine the dataset.”
A better result
It’s complicated stuff. The key, though, is that Apple applies differential privacy to all the user data, which is the process of adding noise that makes it impossible to connect that data to a real user.
Still, none of this works if you don’t opt into Apple’s Data Analytics, which usually happens when you first set up your iPhone, iPad, or MacBook.
Doing so does not put your data or privacy at risk, but that training should lead to better models and, hopefully, a better Apple Intelligence experience on your iPhone and other Apple devices.
It might also mean smarter and more sensible rewrites and summaries.