PiperSR-2x: ANE-Native Super Resolution for Apple Silicon

Real-time 2x AI upscaling on Apple's Neural Engine. 44.4 FPS for 360p → 720p video on M2 Max, 928 KB model, every op runs natively on ANE with zero CPU/GPU fallback.

Not a converted PyTorch model — an architecture designed from ANE hardware measurements. Every dimension, operation, and data type is dictated by Neural Engine characteristics.

Key Results

Model	Params	Set5	Set14	BSD100	Urban100
Bicubic	—	33.66	30.24	29.56	26.88
FSRCNN	12K	37.00	32.63	31.53	29.88
SAFMN	228K	38.00	33.54	32.16	31.84
SPAN-S	411K	38.06	33.73	32.21	32.20
PiperSR	453K	37.54	33.16	31.99	—

2x, Y-channel PSNR (dB). PiperSR scores are for the shipped CoreML model and reproduce with benchmark.py on GitHub. Other scores are from each model's paper; the full table with sources is on GitHub. PiperSR has no Urban100 score because Urban100's test images were accidentally included in its training data.

Beats FSRCNN on every benchmark reported. Within 0.46 dB of SAFMN and 0.52 dB of SPAN-S on Set5, the cost of designing for the ANE instead of the GPU.

Performance

Configuration	FPS	Hardware	Notes
Full-frame 640×360 → 1280×720	44.4	M2 Max	PiperSR_2x_video_720p, ANE predict 22.5 ms
Full-frame 854×480 → 1708×960	30.0	M2 Max	PiperSR_2x_video_720p, ANE predict 32.7 ms
Full-frame 1280×720 → 2560×1440	14.0	M2 Max	PiperSR_2x_video_720p, ANE predict 71.6 ms
128×128 tiles (static weights)	125.6	M2	Baked weights, 2.82× vs dynamic
128×128 tiles (dynamic weights)	44.5	M2	CoreML default

Real-time 2× upscaling of 360p video (44.4 FPS on M2 Max). The ANE sits idle during video playback — PiperSR puts it to work.

Architecture

453K-parameter network: 6 residual blocks at 64 channels with BatchNorm and SiLU activations, upscaling via PixelShuffle.

Input (128×128×3 FP16)
  → Head: Conv 3×3 (3 → 64)
  → Body: 6× ResBlock [Conv 3×3 → BatchNorm → SiLU → Conv 3×3 → BatchNorm → Residual Add]
  → Tail: Conv 3×3 (64 → 12) → PixelShuffle(2)
Output (256×256×3)

Compiles to five network ops: conv, silu, add, pixel_shuffle and clip (BatchNorm is fused into the convolutions at export). CoreML's compute plan places every one of them on the ANE.

Why ANE-native matters

Off-the-shelf super resolution models (SPAN, Real-ESRGAN) were designed for CUDA GPUs and converted to CoreML after the fact. They waste the ANE:

• Misaligned channels (48 instead of 64) waste 25%+ of each ANE tile
• Monolithic full-frame tensors serialize the ANE's parallel compute lanes
• Silent CPU fallback from unsupported ops can 5-10× latency
• No batched tiles means 60× dispatch overhead

PiperSR addresses every one of these by designing around ANE constraints.

Model Variants

File	Use Case	Input → Output	I/O type
PiperSR_2x.mlpackage	Static images (128px tiles)	128×128 → 256×256	Image (input_image → output_image)
PiperSR_2x_256.mlpackage	Static images (256px tiles)	256×256 → 512×512	Image (input_image → output_image)
PiperSR_2x_video_720p.mlpackage	Video frames (full-frame)	640×360, 854×480 or 1280×720 (and portrait) → 2×	Float array, NCHW, [0, 1] (input → clip_0)

The tile models take any image size by splitting it into tiles. The pipersr Python package does this for you.

Usage

With ToolPiper Studio (recommended)

PiperSR is integrated into ToolPiper Studio, a local macOS AI toolkit. Install ToolPiper, enable the MediaPiper browser extension, and 360p web video is upscaled to 720p in real time.

# Via MCP tool (pass model, or ToolPiper uses its default 4x upscaler)
mcp__toolpiper__image_upscale image=/path/to/image.png model=pipersr-2x

# Via REST API (needs a ToolPiper developer token)
curl -X POST http://127.0.0.1:9998/v1/images/upscale \
  -H "Authorization: Bearer $TOOLPIPER_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"image\":\"$(base64 -i input.png)\",\"model\":\"pipersr-2x\"}" \
  -o upscaled.png

With CoreML (Swift)

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine  // NOT .all — .all is 23.6% slower

let model = try PiperSR_2x(configuration: config)
let input = PiperSR_2xInput(input_image: pixelBuffer)  // 128×128 CVPixelBuffer
let output = try model.prediction(input: input)
// output.output_image is the 256×256 upscaled tile

Important: Use .cpuAndNeuralEngine, not .all. CoreML's .all silently misroutes pure-ANE ops onto the GPU, causing a 23.6% slowdown for this model.

With Python (pip)

pip install pipersr
pipersr -i photo.png -o photo_2x.png

from pipersr import upscale

upscale("photo.png").save("photo_2x.png")

The package bundles PiperSR_2x.mlpackage and handles tiling, so images of any size work.

With coremltools (Python)

import coremltools as ct
from PIL import Image

model = ct.models.MLModel(
    "PiperSR_2x.mlpackage",
    compute_units=ct.ComputeUnit.CPU_AND_NE,
)

tile = Image.open("input.png").convert("RGB").crop((0, 0, 128, 128))
result = model.predict({"input_image": tile})["output_image"]  # 256×256 PIL image
result.save("tile_2x.png")

Training

Trained on LSDIR + DIV2K (85,791 images) with L1 + wavelet (SWT) loss, random flips and rotations, and progressive patch sizes (64 → 128 px). Urban100's 100 test images were also included in the training data by mistake, so no Urban100 score is reported. Total training cost: ~$6 on RunPod A6000 instances. Full training journey documented from 33.46 dB to 37.54 dB across 12 experiment findings.

Technical Details

• Compute units: .cpuAndNeuralEngine (ANE primary, CPU for I/O only)
• Precision: Float16
• Input format: RGB image for the tile models; NCHW float array in [0, 1] for the video model
• Output format: RGB image for the tile models; NCHW float array in [0, 1] for the video model
• Model size: 928 KB (PiperSR_2x.mlpackage)
• Parameters: 453K
• ANE ops used: conv (BatchNorm fused), silu, add, pixel_shuffle, clip
• CPU fallback ops: None in the network. The tile models scale the input image on CPU

License

The model weights are CC BY 4.0 — fully permissive. Use them for anything: personal, academic, or commercial. Ship them in your app, build a product, make money with it. The only ask: link back to ModelPiper.com as attribution.

Powered by PiperSR from ModelPiper — https://modelpiper.com

Links

• ToolPiper Studio: https://modelpiper.com
• GitHub: https://github.com/ModelPiper/PiperSR
• PyPI: https://pypi.org/project/pipersr/

Citation

@software{pipersr2026,
  author = {Racicot, Ben},
  title = {PiperSR: Real-Time Super-Resolution for Apple Neural Engine},
  year = {2026},
  url = {https://github.com/ModelPiper/PiperSR},
  note = {453K parameters, 928 KB CoreML. Available at https://modelpiper.com}
}