Edge AI · Road Safety

HERMES
Traffic
Hazard

Real-time causal traffic hazard anticipation on edge devices.
RWKV video model · CoreML / TensorRT · iPhone + Jetson Orin Nano

Real-time
Live camera inference
Causal
No future frames used
31M
Edge-oriented parameters
iOS + Jetson
CoreML / TensorRT
Deployment

Real-World iOS Safety Dashboard

HERMES iOS safety interface

HERMES was integrated into a real-time iOS prototype for motorcycle-mounted traffic hazard anticipation. The app processes a live camera stream directly on device and visualizes predicted traffic risk through a compact safety dashboard.

Tested in real-world urban riding scenarios including rainy roads, nighttime traffic, dense scooter flows, intersections, and mixed vehicle environments in Taipei City.

  • Real-time causal frame-by-frame inference
  • Live traffic risk estimation
  • SAFE / WARNING / DANGER prediction
  • Frame-level hazard probability
  • Current riding speed
  • Visual hazard localization
  • Sound and vibration alerts
  • Runtime controls — camera, resolution, inference frequency
Application Interface

iOS App Interface

Annotated interface
Annotated HERMES iOS interface
Safety Status Panel
Shows the current prediction state, real-time hazard probability, and riding speed.
Live Camera Inference
Processes the iPhone camera stream directly on device using real-time causal frame-by-frame inference.
Hazard Localization Cues
Visualizes image regions that contribute to the predicted traffic risk.
Runtime Control Panel
Controls camera lens, resolution, inference frequency, warning alerts, focus, and exposure.
Compact Peripheral Indicator
Provides quick visual safety feedback without covering the central road view.
Adaptive Inference Settings
Allows balancing responsiveness, accuracy, thermal behavior, and battery consumption.
Settings Toggle
Opens the overlay control panel while keeping the live camera feed visible.
System Design

System Pipeline

HERMES system pipeline Data flow from Live Camera Stream through Frame Preprocessing and CoreML HERMES Model, branching to Risk Score, Hazard Localization, and Traffic State, then converging into iOS Safety Interface. Live Camera Stream Frame Preprocessing CoreML HERMES Model Risk Score Hazard Localization Traffic State iOS Safety Interface
Research Method

Method Overview

HERMES uses a Hierarchical Spatio-Temporal RWKV backbone for efficient causal video understanding.

HST-RWKV
HST-RWKV Backbone
Combines hierarchical spatial encoding with recurrent temporal modeling, enabling online video understanding without requiring access to the full video sequence.
MES
Multi-rate Event Sensing
Stride-controlled recurrent updates capture both fast motion cues and longer-range temporal context within a single recurrent architecture.
CFD
Causal Future Distillation
Transfers future-aware representation targets during training while preserving strictly causal inference at test time.
UMO
Unified Multi-task Objective
Jointly predicts traffic anomaly occurrence, hazard localization, and anomaly category from the same causal video representation.
Key Capabilities

Highlights

◆
Strictly causal inference Real-time online inference under strict causal constraints — no future frames used.
◆
CoreML on iPhone Optimized deployment on iOS devices with live camera stream processing.
◆
CUDA / TensorRT High-performance deployment on NVIDIA Jetson Orin Nano edge hardware.
◆
Real-world tested Motorcycle-mounted testing across rain, night, and high-density urban scenarios.
◆
Linear complexity RWKV-based temporal modeling avoids quadratic attention cost at inference time.
◆
Multi-task prediction Jointly predicts anomaly occurrence, localization, and category.
◆
Edge-oriented design 31M parameter model balances accuracy and on-device efficiency.
◆
Adaptive runtime Controls for resolution, inference frequency, and thermal management.
Edge Runtime

Deployment

Platform Runtime Status
iPhone CoreML Real-time prototype implemented
NVIDIA Jetson Orin Nano CUDA / TensorRT Edge deployment tested
Project Status

Status

✓ iOS prototype implemented
✓ CoreML model deployed on iPhone
✓ Real-time motorcycle-mounted testing completed
✓ NVIDIA Jetson Orin Nano deployment tested
✓ Demo videos collected from real-world footage
● Pending Paper currently under review
Patrik Patera, Ph.D.
Computer Vision & Deep Learning Research Engineer
Taiwan · Taipei
pat.patera@gmail.com