Add Apple Intelligence provider — fully local on-device AI for macOS

README.md

@@ -22,7 +22,7 @@ Battle tested at [<picture><source media="(prefers-color-scheme: dark)" srcset="
 
 ---
 
-Build chatbots, AI agents, RAG applications. Works with OpenAI, xAI, Anthropic, Google, AWS, local models, and any OpenAI-compatible API.
+Build chatbots, AI agents, RAG applications. Works with OpenAI, xAI, Anthropic, Google, AWS, Apple Intelligence, local models, and any OpenAI-compatible API.
 
 ## From zero to AI chat app in under two minutes
 
@@ -123,6 +123,12 @@ end
 response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "product.txt"
 ```
 
+```ruby
+# Totally local AI with Apple Intelligence — no API keys, no cloud
+chat = RubyLLM.chat(model: "apple-intelligence", provider: :apple_intelligence)
+chat.ask "Explain this code", with: "app.rb"
+```
+
 ## Features
 
 * **Chat:** Conversational AI with `RubyLLM.chat`
@@ -140,7 +146,7 @@ response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "pr
 * **Async:** Fiber-based concurrency
 * **Model registry:** 800+ models with capability detection and pricing
 * **Extended thinking:** Control, view, and persist model deliberation
-* **Providers:** OpenAI, xAI, Anthropic, Gemini, VertexAI, Bedrock, DeepSeek, Mistral, Ollama, OpenRouter, Perplexity, GPUStack, and any OpenAI-compatible API
+* **Providers:** OpenAI, xAI, Anthropic, Gemini, VertexAI, Bedrock, DeepSeek, Mistral, Ollama, OpenRouter, Perplexity, GPUStack, Apple Intelligence, and any OpenAI-compatible API
 
 ## Installation
 

docs/_getting_started/apple-intelligence.md

@@ -0,0 +1,164 @@
+---
+layout: default
+title: Apple Intelligence
+nav_order: 4
+description: Run AI completely on-device with Apple Intelligence — no API keys, no cloud, fully private.
+---
+
+# {{ page.title }}
+{: .no_toc }
+
+{{ page.description }}
+{: .fs-6 .fw-300 }
+
+## Table of contents
+{: .no_toc .text-delta }
+
+1. TOC
+{:toc}
+
+---
+
+After reading this guide, you will know:
+
+* How to use Apple Intelligence for completely local, on-device AI
+* The system requirements and how to verify your setup
+* How to configure and customize the provider
+* How it works under the hood
+* Current limitations and troubleshooting tips
+
+## What is Apple Intelligence?
+
+Apple Intelligence brings on-device AI to RubyLLM through Apple's Foundation Models. Your prompts and responses never leave your Mac — no API keys, no cloud services, no data sharing. It's the most private way to use AI with RubyLLM.
+
+Under the hood, RubyLLM communicates with the `osx-ai-inloop` binary, which pipes JSON requests to Apple's on-device language model via stdin/stdout.
+
+## Requirements
+
+* **macOS 26** (Tahoe) or later
+* **Apple Silicon** (M1 or later)
+* **Apple Intelligence** enabled in System Settings > Apple Intelligence & Siri
+
+> Apple Intelligence is not available on Intel Macs or older macOS versions. RubyLLM will raise an error if the requirements aren't met.
+{: .note }
+
+## Quick Start
+
+No configuration needed. Just use it:
+
+```ruby
+chat = RubyLLM.chat(model: "apple-intelligence", provider: :apple_intelligence)
+chat.ask "Explain Ruby's block syntax"
+```
+
+That's it. No API keys, no environment variables, no account setup. The `osx-ai-inloop` binary is automatically downloaded and cached on first use.
+
+## Conversation History
+
+Apple Intelligence supports multi-turn conversations, just like any other provider:
+
+```ruby
+chat = RubyLLM.chat(model: "apple-intelligence", provider: :apple_intelligence)
+chat.ask "What is a Ruby module?"
+chat.ask "How is that different from a class?"
+chat.ask "When should I use one over the other?"
+```
+
+Each follow-up includes the full conversation history, so the model maintains context across turns.
+
+## Configuration
+
+### Zero Config (Default)
+
+Apple Intelligence works out of the box with no configuration. RubyLLM automatically downloads the `osx-ai-inloop` binary to `~/.ruby_llm/bin/osx-ai-inloop` on first use.
+
+### Custom Binary Path
+
+If you prefer to manage the binary location yourself:
+
+```ruby
+RubyLLM.configure do |config|
+  config.apple_intelligence_binary_path = "/opt/bin/osx-ai-inloop"
+end
+```
+
+### Setting as Default Model
+
+To use Apple Intelligence as your default chat model:
+
+```ruby
+RubyLLM.configure do |config|
+  config.default_model = "apple-intelligence"
+end
+
+# Now RubyLLM.chat uses Apple Intelligence automatically
+chat = RubyLLM.chat(provider: :apple_intelligence)
+chat.ask "Hello!"
+```
+
+## How It Works
+
+1. RubyLLM formats your conversation as a JSON payload
+2. The payload is piped to the `osx-ai-inloop` binary via stdin
+3. The binary communicates with Apple's Foundation Models on-device
+4. The response is read from stdout and parsed back into RubyLLM's standard format
+
+The binary is sourced from the [osx-ai-inloop](https://github.com/inloopstudio-team/apple-intelligence-inloop) project and cached at `~/.ruby_llm/bin/osx-ai-inloop`.
+
+## Limitations
+
+Apple Intelligence is text-only and runs entirely on-device. This means:
+
+* **No streaming** — responses are returned all at once
+* **No vision** — image analysis is not supported
+* **No tool calling** — function/tool use is not available
+* **No embeddings** — use another provider for `RubyLLM.embed`
+* **No image generation** — use another provider for `RubyLLM.paint`
+* **macOS only** — requires Apple Silicon and macOS 26+
+
+For capabilities that Apple Intelligence doesn't support, you can use another provider alongside it:
+
+```ruby
+# Local AI for chat
+local_chat = RubyLLM.chat(model: "apple-intelligence", provider: :apple_intelligence)
+local_chat.ask "Summarize this concept"
+
+# Cloud provider for embeddings
+RubyLLM.embed "Ruby is elegant and expressive"
+```
+
+## Troubleshooting
+
+### "Platform not supported" error
+
+Apple Intelligence requires macOS 26+ on Apple Silicon. Verify your setup:
+
+* Check macOS version: Apple menu > About This Mac
+* Ensure Apple Intelligence is enabled: System Settings > Apple Intelligence & Siri
+
+### Binary download fails
+
+If the automatic download fails (network issues, firewall, etc.), download manually:
+
+```bash
+wget -O ~/.ruby_llm/bin/osx-ai-inloop \
+  https://github.com/inloopstudio-team/apple-intelligence-inloop/raw/refs/heads/main/bin/osx-ai-inloop-arm64
+chmod +x ~/.ruby_llm/bin/osx-ai-inloop
+```
+
+### Binary not found at custom path
+
+If you configured a custom binary path, ensure the file exists and is executable:
+
+```bash
+ls -la /your/custom/path/osx-ai-inloop
+chmod +x /your/custom/path/osx-ai-inloop
+```
+
+## Next Steps
+
+Now that you have local AI running, explore other RubyLLM features:
+
+- [Chat with AI models]({% link _core_features/chat.md %}) for more conversation features
+- [Configuration]({% link _getting_started/configuration.md %}) for multi-provider setups
+- [Tools and function calling]({% link _core_features/tools.md %}) with cloud providers

docs/_getting_started/configuration.md

@@ -107,6 +107,26 @@ end
 > Attempting to use an unconfigured provider will raise `RubyLLM::ConfigurationError`. Only configure what you need.
 {: .note }
 
+### Apple Intelligence (On-Device)
+
+Apple Intelligence requires no API keys — it runs entirely on your Mac. Just use it:
+
+```ruby
+chat = RubyLLM.chat(model: "apple-intelligence", provider: :apple_intelligence)
+chat.ask "Hello from on-device AI!"
+```
+
+The `osx-ai-inloop` binary is automatically downloaded on first use. To customize its location:
+
+```ruby
+RubyLLM.configure do |config|
+  config.apple_intelligence_binary_path = "/opt/bin/osx-ai-inloop"
+end
+```
+
+> Apple Intelligence requires macOS 26+ (Tahoe) on Apple Silicon with Apple Intelligence enabled. See the [Apple Intelligence guide]({% link _getting_started/apple-intelligence.md %}) for full details.
+{: .note }
+
 ### OpenAI Organization & Project Headers
 
 For OpenAI users with multiple organizations or projects:
@@ -450,6 +470,9 @@ Here's a complete reference of all configuration options:
 
 ```ruby
 RubyLLM.configure do |config|
+  # Apple Intelligence (on-device, no API key needed)
+  config.apple_intelligence_binary_path = String  # Optional: custom binary path
+
   # Anthropic
   config.anthropic_api_key = String
   config.anthropic_api_base = String  # v1.13.0+

docs/_getting_started/overview.md

@@ -149,6 +149,9 @@ chat = RubyLLM.chat(
   model: "{{ site.models.local_llama }}",
   provider: :ollama,
 )
+
+# On-device AI with Apple Intelligence — no API keys, no cloud
+chat = RubyLLM.chat(model: "apple-intelligence", provider: :apple_intelligence)
 ```
 
 ### Capability Management

lib/ruby_llm.rb

@@ -15,6 +15,7 @@
 
 loader = Zeitwerk::Loader.for_gem
 loader.inflector.inflect(
+  'apple_intelligence' => 'AppleIntelligence',
   'azure' => 'Azure',
   'UI' => 'UI',
   'api' => 'API',
@@ -93,6 +94,7 @@ def logger
   end
 end
 
+RubyLLM::Provider.register :apple_intelligence, RubyLLM::Providers::AppleIntelligence
 RubyLLM::Provider.register :anthropic, RubyLLM::Providers::Anthropic
 RubyLLM::Provider.register :azure, RubyLLM::Providers::Azure
 RubyLLM::Provider.register :bedrock, RubyLLM::Providers::Bedrock

lib/ruby_llm/providers/apple_intelligence.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+module RubyLLM
+  module Providers
+    # Apple Intelligence provider — pipes requests through the osx-ai-inloop
+    # binary via stdin/stdout, completely bypassing HTTP/Faraday.
+    class AppleIntelligence < Provider
+      include AppleIntelligence::Chat
+      include AppleIntelligence::Models
+
+      def initialize(config)
+        super
+        @config = config
+        @connection = nil
+      end
+
+      def api_base
+        nil
+      end
+
+      # rubocop:disable Metrics/ParameterLists,Metrics/PerceivedComplexity
+      def complete(messages, tools: nil, temperature: nil, model: nil, params: {}, headers: {}, schema: nil,
+                   thinking: nil, tool_prefs: nil, &)
+        _ = [temperature, model, params, headers, schema, thinking, tool_prefs] # not used for local provider
+
+        # Two-pass tool calling: if tools are registered and we haven't already
+        # executed a tool (no :tool messages yet), extract arguments and call.
+        if tools&.any? && messages.none? { |m| m.role == :tool }
+          last_user = messages.reverse.find { |m| m.role == :user }
+          tool_msg = try_tool_call(tools, last_user, @config) if last_user
+          return tool_msg if tool_msg
+        end
+
+        payload = build_payload(messages)
+        execute_binary(payload, @config)
+      end
+      # rubocop:enable Metrics/ParameterLists,Metrics/PerceivedComplexity
+
+      class << self
+        def configuration_options
+          %i[apple_intelligence_binary_path]
+        end
+
+        def configuration_requirements
+          []
+        end
+
+        def local?
+          true
+        end
+
+        def assume_models_exist?
+          true
+        end
+
+        def capabilities
+          AppleIntelligence::Capabilities
+        end
+      end
+
+      private
+
+      def try_tool_call(tools, last_user, config)
+        user_text = case last_user.content
+                    when String then last_user.content
+                    when Content then last_user.content.text || ''
+                    else last_user.content.to_s
+                    end
+        tool_result = resolve_tool_call(tools, user_text, config)
+        return unless tool_result
+
+        Message.new(
+          role: :assistant,
+          content: '',
+          tool_calls: tool_result,
+          model_id: 'apple-intelligence',
+          input_tokens: 0,
+          output_tokens: 0
+        )
+      end
+    end
+  end
+end

lib/ruby_llm/providers/apple_intelligence/binary_manager.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+require 'open-uri'
+require 'fileutils'
+
+module RubyLLM
+  module Providers
+    class AppleIntelligence
+      # Manages downloading, caching, and locating the osx-ai-inloop binary
+      module BinaryManager
+        BINARY_URL = 'https://github.com/inloopstudio-team/apple-intelligence-inloop/raw/refs/heads/main/bin/osx-ai-inloop-arm64'
+        DEFAULT_CACHE_DIR = File.join(Dir.home, '.ruby_llm', 'bin')
+        DEFAULT_BINARY_NAME = 'osx-ai-inloop'
+
+        module_function
+
+        def binary_path(config = nil)
+          custom = config&.apple_intelligence_binary_path
+          return custom if custom && File.executable?(custom)
+
+          default_path = File.join(DEFAULT_CACHE_DIR, DEFAULT_BINARY_NAME)
+          ensure_binary!(default_path) unless File.executable?(default_path)
+          default_path
+        end
+
+        def ensure_binary!(path)
+          check_platform!
+          download_binary!(path)
+          File.chmod(0o755, path)
+        end
+
+        def check_platform!
+          raise RubyLLM::Error, 'Apple Intelligence provider requires macOS' unless RUBY_PLATFORM.include?('darwin')
+
+          return if RUBY_PLATFORM.include?('arm64')
+
+          RubyLLM.logger.warn('Apple Intelligence binary is built for arm64. ' \
+                              'It may not work on this architecture.')
+        end
+
+        def download_binary!(path)
+          FileUtils.mkdir_p(File.dirname(path))
+          RubyLLM.logger.info("Downloading osx-ai-inloop binary to #{path}...")
+
+          URI.open(BINARY_URL, 'rb') do |remote| # rubocop:disable Security/Open
+            File.binwrite(path, remote.read)
+          end
+
+          RubyLLM.logger.info('Binary downloaded successfully.')
+        rescue OpenURI::HTTPError, SocketError, Errno::ECONNREFUSED => e
+          raise RubyLLM::Error, "Failed to download Apple Intelligence binary: #{e.message}"
+        end
+      end
+    end
+  end
+end