Their keynotes are irrelevant, their official privacy policies and legal disclosures take precedence over marketing claims or statements made in keynotes or presentations. Apple’s privacy policy states that the company collects data necessary to provide and improve its products and services. The OS-level AI would fall under this category, allowing Apple to collect data processed by the AI for improving its functionality and models. Apple’s keynotes and marketing materials do not carry legal weight when it comes to their data practices. With the AI system operating at the OS level, it likely has access to a wide range of user data, including text inputs, conversations, and potentially other sensitive information.

Running FOSS and taking control of your network will do a far better trick of privacy vs convenience than most people can imagine

I don’t use either of those thankfully:).

• Malicious actors could potentially exploit vulnerabilities in the AI system to gain unauthorized access or control over device functions and data, potentially leading to severe privacy breaches, unauthorized data access, or even the ability to inject malicious content or commands through the AI system.
• Privacy breaches are possible if the AI system is compromised, exposing user data, activities, and conversations processed by the AI.
• Integrating AI functionality deeply into the operating system increases the overall attack surface, providing more potential entry points for malicious actors to exploit vulnerabilities and gain unauthorized access or control.
• Human reviewers have access to annotate and process user conversations for improving the AI models. To effectively train and improve the AI models powering the OS-level integration, Apple would likely need to collect and process user data, such as text inputs, conversations, and interactions with the AI.
• Apple’s privacy policy states that the company collects data necessary to provide and improve its products and services. The OS-level AI would fall under this category, allowing Apple to collect data processed by the AI for improving its functionality and models.
• Despite privacy claims, Apple has a history of collecting various types of user data, including device usage, location, health data, and more, as outlined in their privacy policies.
• If Apple partners with third-party AI providers, there is a possibility of user data being shared or accessed by those entities, as permitted by Apple’s privacy policy.
• With the AI system operating at the OS level, it likely has access to a wide range of user data, including text inputs, conversations, and potentially other sensitive information. This raises privacy concerns about how this data is handled, stored, and potentially shared or accessed by the AI provider or other parties.
• Lack of transparency for users about when and how their data is being processed by the AI system & users not being fully informed about data collection related to the AI. Additionally, if the AI integration is controlled solely at the OS level, users may have limited control over enabling or disabling this functionality.

I’m imagining a world where advertisers have to try to raise engagement from AIs in their ads

I would not give the right of anyone deciding what is good for my privacy, including Apple. This should be a judgement made by myself.

That’s great. But that’s not how it’s being marketed and sold to the public. It’s being sold as an oracle (as in crystal ball, not database). And it’s misleading and hurting people as a result.

I’ll reiterate: An LLM has no comprehension of what it says.

It’s a matter of engineering ethics, on multiple levels:

• the training data in the vast majority of cases is outright stolen
• it’s being sold as something that it’s not, and the result is causing real damage to people and society in a ton of ways we’re still discovering
• most people deeply involved in developing LLMs, and basically all of the technical leadership, are categorically ignoring and abrogating any and all responsibility around this “magical” new system they’ve made. We’ve seen this before with social networking. We know where this road leads.

I’m not saying the tech should be banned. That’s obviously idiotic. Neural nets can - and are - used for tons of fascinating and excellent applications. It’s just that my staunch opinion is that LLMs are a terrible application of that the tech at this stage of development, and it’s particularly terrible that OpenAI/Microsoft/etc are aggressively foisting this technology on the public, and simultaneously refusing to take any ethical responsibility for it.