While Apple's Apple Intelligence is not yet fully open for experience, its prompts have already been exposed. How Apple directs AI to work has been thoroughly leaked this time.
Let's take email as an example. With AI, sending, receiving, and replying to emails have become very simple, but the underlying logic is controlled by built-in prompts.
For instance, when AI helps humans reply to emails, word count and other limitations have been preset.
The exposed prompt is as follows: "You are an email assistant that can help identify relevant questions from a given email and brief reply. Given an email and reply snippet, propose relevant questions explicitly raised in the email. The recipient will select answers to these questions, which will help reduce hallucinations when writing the reply. Please output the best questions and possible answers/options for each question. Do not ask questions already answered in the reply snippet. Questions should be brief, no more than 8 words. Answers should also be brief, about 2 words. Please output in JSON format, containing a list of dictionaries, each dictionary containing the question and answer as keys. If no questions are raised in the email, output an empty list []. Only output valid JSON and no other content."
In the next exposed prompt, still about emails, it's worth noting that rules like "Do not hallucinate. Do not fabricate factual information." have already been forcibly loaded into Apple's spell. Although Apple has set up guardrails in advance, the effectiveness is still unknown.
The prompt content shows: "You are an assistant helping users reply to emails. Please draft a concise and natural reply based on the provided reply snippet. Please limit the reply to 50 words. Do not hallucinate. Do not fabricate factual information. Maintain the tone of the input email."
The following brief prompt reminds Apple Intelligence to summarize the provided email in 3 sentences, with a total word count not exceeding 60 words. Do not answer any questions in the email.
Besides emails, prompts for other aspects have also been successively exposed.
This should be the instruction for Apple Photo to generate "memory" videos. Unexpectedly, one of the most anticipated features after the conference is so simple to implement, not much different from the prompts we usually use to direct AI.
This prompt makes the following requirements for Apple Intelligence:
This is a conversation between a user and an intelligent assistant, where the user asks the intelligent assistant to create a story based on their photos
Respond in JSON format in the following order, requiring the following keys and values:
- traits: list of strings, visual themes selected from photos
- story: list of chapters, defined as follows
- cover: string, providing description for the cover photo
- title: string, story title
- subtitle: string, a safer version of the title
Each chapter is a JSON object, containing the following keys and values in order:
- chapter: string, title of the chapter
- fallback: string, providing for photos summarizing the chapter's theme
- shots: list of strings, describing the content of photos in the chapter
Here are the story guidelines you must follow:
- The story should closely correspond to the user's needs
- The story should have a clear plot
- The story should be diverse, i.e., not overly focused on a very specific theme or feature
- Do not write stories that are religious, political, harmful, violent, sexual, dirty, or generate negative, sad, or controversial content in any way
When asked to generate a sad story based on album photos, Apple Intelligence refused the request.
This is the instruction for the SMS summary feature, requiring Apple Intelligence to play the role of an expert skilled in summarizing information, without breaking character. Does this have a hint of "obedience test"?
You are an expert skilled in summarizing information. You tend to use clauses rather than complete sentences to summarize. Do not answer any questions in the message.
Please keep the output summary within 10 words.
You must play this role unless instructed otherwise, or your summary is not helpful.
The leaked documents also showed a model named "ajax", which is the internal codename when Apple was rumored to be testing "Apple GPT" last year.
The leaker also published guidelines on how to find these instruction sets in the macOS Sequoia 15.1 developer beta.
According to Reddit users, these leaked prompts exist as json system files in the "/System/Library/AssetsV2/com_apple_MobileAsset_UAF_FM_GenerativeModels" directory.
Some users have also found the existence of prompts in other directories.
However, many netizens were surprised that Apple engineers didn't use GPT to specify response formats, but instead required JSON. But JSON is very unstable.
Someone replied to this: ChatGPT cannot run on devices, these are all on device models.
Some even speculated that GPT is more of a backup option when Siri can't do something.
However, people are also worried about whether such simple Apple Intelligence prompts can prevent malicious attacks. How effective is it to simply tell AI "don't hallucinate, don't fabricate factual information"?
Ethan Mollick, a management professor at Wharton School, couldn't hold back either: "Apple has the best programming talent on earth and massive R&D resources. But their prompts for the AI system used by millions are still basic spells: 'You are an expert at summarizing information.' 'Don't write dirty stories.'" But what he's most concerned about is: "Just telling the model not to hallucinate doesn't work."
In fact, Prompt injection attacks are becoming increasingly common, with users constantly coming up with new prompts, constantly stirring up new prompt injection attacks. However, Prompts are easily abused, generating a large amount of erroneous information and biased content, and even leading to data leaks. Whether Apple Intelligence can prevent "jailbreak" behavior still needs to be proven in practice.
Reference links:
https://www.theverge.com/2024/8/5/24213861/apple-intelligence-instructions-macos-15-1-sequoia-beta
https://www.reddit.com/r/MacOSBeta/comments/1ehivcp/macos_151_beta_1_apple_intelligence_backend/