OpenExplorer/e2e_planner

E2E Diffusion Planner + HENet-tiny

An end-to-end driving model for camera-based perception and short-horizon ego trajectory planning on NAVSIM.

E2E Diffusion Planner + HENet-tiny combines an 8-camera BEV perception front-end with a diffusion planner conditioned on BEV features, ego state, and route. Perception and planning are trained and quantized as one graph for Horizon J6M / J6P deployment.

---

Deployment Metrics

Model Parameters

Model	Model Input	Backbone	Neck	Model Output
E2EPlanner (BEVFusionMultitask + Diffusion_Planner)	8-camera multi-view images (B,8,3,512,960) + ego state + route lanes	HENet-tiny	FPN + SingleBevFormerViewTransformer	future ego trajectory (1,8,4) via diffusion

Accuracy Metrics

March	Metric	float	calibration	hbm
J6M	PDMS	0.8224	0.8139	0.8162

Measured on NAVSIM navtest with HAT_USE_NEIGHBORS_FUTURE=0 and dequantize_bev_embed=False. No QAT stage.

HEAL version: heal 0.6.0 / hbdk4-compiler 4.13.3 / horizon_plugin_pytorch 3.4.5 (board UCP 3.16.2).

Performance Metrics

Performance benchmark: FPS is measured with single-core 8 threads; latency is single-core single-thread; memory is peak DDR usage. Data source: tools/perf.py (--perf-time 1; DDR sampled by hrt_ucp_monitor -d 200). Measured 2026-10-19 (dequantize_bev_embed=False). Toolchain: hbdk4-compiler 4.13.3 / horizon_plugin_pytorch 3.4.5. No native:: CPU ops.

March	latency (ms)	fps	Memory Usage (MB)
J6M	104.66 (BPU 104.56)	9.75	376.90
J6P	53.25 (BPU 53.19)	66.26	353.70

---

Model Overview

Core Design

Think of two stages glued on one BEV map. Cameras are encoded by HENet-tiny and lifted with a single-frame BevFormer into a forward-looking BEV grid; the same feature feeds detection, occupancy, and map heads for a consistent scene. A compact diffusion planner then reads that BEV, listens to the route, and denoises a few future ego waypoints—so planning stays conditioned on what perception actually sees, without a separate hand-off stack.

• Task type: End-to-end planning (perception + planning joint).
• Perception backbone: HENet-tiny (block_nums=[4,3,8,6], embed_dims=[64,128,192,384]), initialized from the ImageNet-pretrained henet_tinym float checkpoint.
• Perception neck: FPN + SingleBevFormerViewTransformer (128×192 BEV, [-51.2m, 102.4m] × [-51.2m, 51.2m]).
• Dynamic det head: DenseBEVSparseHead (3 classes; num_anchor=384, MemoryBank 128 temporal instances).
• Static det head: DenseBEVSparseHead (4 classes; num_anchor=200).
• Occupancy head: BevformerOccDetDecoder + BEVOCCHead2D (Dz=16, 9 classes).
• Discrete map head: BEVFormerDet2DDecoder (2 classes: stop_line/crosswalk).
• Online map head: MaptrDetDecoder + MapInstanceDetectorHead (1 class lanes, SD-map fusion, 20 points per polyline).
• Planner encoder: Diffusion_Planner_Encoder (BEVFusionEncoder ×8 downsample → 384 tokens, 3-layer FusionEncoder transformer).
• Planner decoder: 3-layer DiT (hidden_dim=256, 8 heads, RouteEncoder), model_type="x_start", diffusion_pred_steps=5, linear VP noise schedule.
• Loss functions: perception — FocalLoss + L1Loss + CrossEntropyLoss + GaussianFocalLoss / CrossEntropyLoss / MapTRCriterion / BevFormer2DCriterion per head; planner — DiffusionLoss (ego 2.0 + collision 1.0).
• Model input: 8-camera multi-view images (B,8,3,512,960) (YUV444 + Normalize(128,128); camera-only) + ego state + route lanes.
• Model output: 8-step future ego trajectory (1,8,4).

Official Repo and Paper

1. Horizon reference algorithm blog (Diffusion-Planner_v3.0): https://developer.horizon.auto/blog/14190
2. Diffusion-Planner: https://github.com/ZhengYinan-AIR/Diffusion-Planner (paper: https://arxiv.org/abs/2501.15564)
3. NAVSIM benchmark: https://github.com/autonomousvision/navsim (paper: https://arxiv.org/abs/2406.15349)
4. SparseBEV-style det head repo: https://github.com/MCG-NJU/SparseBEV (paper: https://arxiv.org/abs/2308.09244)
5. MapTR repo: https://github.com/hustvl/MapTR (paper: https://arxiv.org/abs/2208.14437)

Note: The camera backbone HENet and the E2E assembly are HEAL in-house implementations; the official repos use different backbones and pipelines.