ART-MLX: Train AI Agents on Your Mac (First GRPO on Apple Silicon)

By Prahlad Menon Published 2026-07-12 3 min read

We built ART-MLX — the first reinforcement learning framework for AI agents that runs natively on Apple Silicon. It’s now verified and working.

Project page: menonpg.github.io/art-mlx
Test results: Tic Tac Toe GRPO results
Repository: github.com/menonpg/art-mlx

The Simple Version

Imagine you want to teach an AI to get better at a task by practicing — like how you get better at video games by playing more. This is called reinforcement learning (RL).

There’s a tool called ART from OpenPipe that does exactly this for AI agents. The problem? It only works on expensive NVIDIA graphics cards — the kind that cost $1,000-$10,000 and run on Linux servers.

You have a $3,000 MacBook Pro with a powerful M1/M2/M3/M4 chip… and it just sits there useless for this kind of AI training.

We took ART and rewrote the parts that need NVIDIA to use Apple’s MLX instead — a framework that runs natively on your Mac’s GPU.

Think of it like: the original game only ran on PlayStation. We ported it to run on Nintendo Switch.

What We Proved

We trained an AI to play Tic Tac Toe using our new code:

Before training: Won 53% of games (basically random)
After training: Won 57% of games
Training time: ~90 seconds on M1 Max

That +3% might sound small, but the point isn’t Tic Tac Toe — it’s that the training actually worked. Gradients flowed, weights updated, the AI learned. On a Mac. For free.

Before ART-MLX	After ART-MLX
Need to rent cloud GPU ($0.50-3/hr)	Use your Mac ($0)
Upload code, wait, download results	Run locally in seconds
Only works on Linux + NVIDIA	Works on any M1/M2/M3/M4 Mac

TL;DR: We made expensive AI training tech work for free on Macs. And proved it works.

The Technical Details

OpenPipe’s ART lets you train AI agents using GRPO (Group Relative Policy Optimization) — the same algorithm behind reasoning models like DeepSeek-R1. Their claim: a Qwen 2.5 14B model trained with ART outperforms o3 on email retrieval tasks.

But the original stack requires NVIDIA:

vLLM for inference (CUDA only)
Unsloth for LoRA training (CUDA only)
PyTorch with CUDA tensors

We replaced all of it with native MLX.

What We’re Building

A drop-in MLXBackend class that replaces LocalBackend:

# Before (requires NVIDIA GPU)
from art.local.backend import LocalBackend
backend = LocalBackend()

# After (runs on Apple Silicon)
from art.mlx import MLXBackend
backend = MLXBackend()

Same API, different hardware.

The Replacement Stack

Component	Original ART	ART-MLX
Inference	vLLM (CUDA)	mlx-lm (Metal)
LoRA Training	Unsloth (CUDA)	MLX native
GRPO Algorithm	PyTorch CUDA	MLX native
Hardware	NVIDIA GPU	Apple Silicon

What is GRPO?

GRPO (Group Relative Policy Optimization) is an RL algorithm designed for training agents. The key insight: instead of needing a separate “critic” network to estimate value, you compare multiple attempts at the same task directly.

Task: "Find the email from John about the meeting"

Rollout A: Searches inbox, finds email → reward = 1
Rollout B: Gets confused, returns wrong email → reward = 0  
Rollout C: Searches, finds email correctly → reward = 1

Group mean = 0.67
Advantage A = +0.33 (better than average)
Advantage B = -0.67 (worse than average)
Advantage C = +0.33 (better than average)

→ Update policy to increase P(A), P(C) and decrease P(B)

Run enough rollouts, compute advantages, update the model. Repeat. The agent gets better at the task.

This is simpler than PPO (no value function to train) and has proven effective for multi-step agent workflows.

Current Status

What’s implemented:

✅ MLXBackend class structure
✅ GRPO algorithm in native MLX
✅ LoRA layer implementation
✅ Basic inference with mlx-lm
✅ Tic Tac Toe training example

What’s in progress:

⏳ Full gradient updates in training loop
⏳ Checkpoint save/load

What’s planned:

2048 game example
Benchmarks vs ServerlessBackend
OpenClaw-RL integration (directive signals)
PR to upstream OpenPipe/ART

Quick Start (For Testing)

If you have an Apple Silicon Mac:

git clone https://github.com/menonpg/art-mlx
cd art-mlx
git checkout mlx-backend
pip install -e ".[mlx]"

# Verify environment
python examples/mlx_quicktest.py

# Run training example
python examples/mlx_tictactoe.py

Requirements:

Apple Silicon Mac (M1/M2/M3/M4)
macOS 13.5+
Python 3.12+
16GB+ RAM (64GB recommended for 7B+ models)

The Code

MLXBackend

The main backend class that replaces LocalBackend:

from art.mlx import MLXBackend
from art import TrainableModel

backend = MLXBackend()

model = TrainableModel(
    name="my-agent",
    project="email-task",
    base_model="mlx-community/Qwen2.5-7B-Instruct-4bit"
)

await model.register(backend)

GRPO Training

The core algorithm:

# Compute advantages relative to group mean
rewards = [t.reward for t in trajectories]
mean_reward = np.mean(rewards)
advantages = [(r - mean_reward) / np.std(rewards) for r in rewards]

# GRPO loss: weighted policy gradient
for trajectory, advantage in zip(trajectories, advantages):
    log_prob = compute_log_prob(model, trajectory)
    loss = -advantage * log_prob  # Increase prob if advantage > 0

LoRA Layers

Efficient fine-tuning with low-rank adapters:

class LoRALinear(nn.Module):
    """
    output = W @ x + (B @ A) @ x * scale
    
    Where W is frozen, A and B are trainable low-rank matrices.
    """
    def __init__(self, in_features, out_features, rank=8, alpha=16.0):
        self.lora_A = mx.random.normal((rank, in_features)) * 0.01
        self.lora_B = mx.zeros((out_features, rank))
        self.scale = alpha / rank

Future: OpenClaw-RL Integration

We’re planning to integrate concepts from OpenClaw-RL — a framework we covered previously that extracts learning signals from every agent interaction.

The key addition: directive signals.

Current ART only captures evaluative signals — did the task succeed or fail? OpenClaw-RL also captures directive signals — specifically what should have been different.

# Current: just reward
trajectory.set_reward(1.0 if success else -1.0)

# Future: reward + directive hint
trajectory.set_reward(reward)
trajectory.set_directive_hint(
    extract_correction_from_error(error_message)
)

When an agent fails with TypeError: expected int, got str, the error message itself tells you what to fix. OpenClaw-RL’s “Hindsight-Guided OPD” extracts these hints and uses them for richer token-level supervision.

Why This Matters

The MLX ecosystem has:

✅ Inference (mlx-lm)
✅ LoRA fine-tuning (mlx-lm/tuner)
❌ Reinforcement learning for agents

That last gap is what we’re filling. If this works, you’ll be able to:

Train agent behavior locally — no cloud GPU for iteration
Use your Mac’s unified memory — 64GB is a lot of model
Iterate faster — no upload/download, no network latency
Experiment for free — your hardware, your time

Updates

This post will be updated as development progresses.

July 12, 2026 (Verified!):

✅ Tested and working on M1 Max — 53.3% → 56.7% win rate after 90 games
PR merged with fixes for current mlx-lm API (0.31.3)
QLoRA working: 1.47M trainable LoRA params, base model frozen
Full test results with logs

July 12, 2026 (Gradient Updates):

✅ Gradient updates now working — GRPOTrainer computes gradients via mx.value_and_grad and updates weights with AdamW
Added checkpoint save/load utilities
Tic Tac Toe example now performs real training

July 12, 2026 (Initial):

Forked OpenPipe/ART
Created src/art/mlx/ module with backend, GRPO, and LoRA
Added quicktest and Tic Tac Toe examples
GitHub Pages site live: menonpg.github.io/art-mlx