A drone sees the road far better than a car does, and can compute far less. Griffin is a middleware layer that splits each video frame between the two, deciding partition by partition where the work should run — minimising drone energy while holding latency and accuracy targets.

← All projects Paper (PDF) Poster Code

What it does

Griffin is a middleware framework for real-time road event detection using a UAV and a ground vehicle working together. The drone captures aerial video; Griffin partitions each frame and decides, per partition, whether it is processed on the drone or offloaded to the ground vehicle. An optimiser makes that assignment continuously, trading energy against latency and accuracy. Two applications are demonstrated on top of it: collision detection and lane-change detection.

Why — the motivation

The two platforms have exactly complementary deficits.

A ground vehicle has compute but not perspective. Car-mounted sensors see from roughly a metre off the ground, in traffic, with vehicles occluding each other. Events that matter — a car drifting across lanes four vehicles ahead, a developing collision geometry — are frequently outside what that vantage point can resolve.

A drone has perspective but not compute. From above, the scene is essentially unoccluded and the geometry of an event is legible. But as the repository puts it, “UAVs have limited computing and energy resources, making them unsuitable for computationally intensive image processing tasks.” Running a detector on every frame onboard drains the battery that is also keeping the aircraft airborne.

The naive fixes both fail. Process everything onboard and flight time collapses. Stream every frame to the ground and you pay a communication cost that is itself energy-expensive and adds latency to a task where latency is the whole point. The interesting design space is in between, and that is what Griffin occupies.

Intuition behind the approach

The observation that makes this tractable: not all of a frame is equally worth computing.

An aerial road frame is mostly not where the event is. Empty asphalt, verge, rooftops, sky — large regions contribute nothing to detecting a collision or a lane change. The vehicles, and specifically the vehicles whose trajectories are converging, carry nearly all the information.

So the frame should not be treated as an atomic unit to run or offload. Partition it, and decide per partition — because the partitions differ enormously in how much they matter and how much they cost.

Once you accept that framing, the offload decision becomes an explicit optimisation rather than a heuristic. Each partition has an estimated criticalness to the event being detected, a compute cost, a transmission cost, and a latency implication. Choosing an assignment across all partitions that minimises drone energy subject to latency and accuracy constraints is a solvable problem — and it can be re-solved as conditions change, which matters because the drone’s speed, the link quality, and where the vehicles are all move continuously.

How it works

The offload decision is driven by four inputs, per the system description:

1. Partition criticalness to event detection — how much this region is likely to matter for the event being watched for.
2. Real-time system status — current energy budget, link conditions, and load on both endpoints.
3. Previous processing results — what was found in prior frames, which informs where the action currently is.
4. Energy–latency–accuracy trade-offs — the constraint surface the optimiser works within.

The optimiser then “solves the partition distribution problem to minimize UAV energy consumption while meeting application requirements for latency and accuracy.” Energy is the objective; latency and accuracy are constraints. That ordering is the design statement — the system is not trying to be maximally accurate, it is trying to be as cheap as possible while still being good enough and fast enough, because flight time is the scarce resource.

Supporting components handle the fact that both platforms are moving:

• Ground vehicle speed detection via YOLOv8 tracking.
• Drone motion estimation via Lucas–Kanade optical flow, so apparent motion in the frame can be separated from the drone’s own movement — without this, every object appears to move and trajectory reasoning breaks down.
• Energy consumption monitoring using Jetson Power tooling, which closes the loop by measuring the quantity being optimised rather than modelling it.

Architecture

Technical specifications

Optimizations and results

What is being optimised. Unusually for a perception system, accuracy is a constraint rather than the objective. The optimiser minimises drone energy consumption subject to meeting the application’s latency and accuracy requirements. The claim being made is therefore not “more accurate than a baseline detector” but “the same task, at materially lower UAV energy, within the same real-time budget” — which is the metric that determines whether the drone can stay in the air long enough to be useful.

The baselines this is measured against are the two endpoints of the design space it sits between:

• All-onboard processing — no offload. Highest energy on the drone, no transmission cost, latency bounded by TX2 throughput.
• All-offload — every frame streamed to the ground vehicle. Minimal compute energy on the drone but maximal communication energy and latency, and dependent on link quality.

Griffin’s partition-level assignment is the middle path, and the contribution is showing that the middle is meaningfully better than either extreme because partitions genuinely differ in criticalness.

Two engineering choices that make the numbers real rather than simulated: energy is measured on the Jetson rather than estimated from a model, and the evaluation runs on the actual heterogeneous hardware pair rather than in emulation. Both matter, because offloading research is easy to make look good in simulation where communication cost is assumed rather than paid.

Limitations and honest caveats

• Two moving platforms means the geometry is never static. Optical flow compensation handles drone egomotion, but sustained accuracy depends on that estimate staying good through manoeuvres and wind.
• The link is a hard dependency. Offload assumes a usable channel to the ground vehicle; degradation pushes the system toward the all-onboard extreme it was designed to avoid.
• Criticalness estimation is the weak point of any partition-level scheme. If a partition is misjudged as unimportant, the event in it is missed — and that failure is silent, which is the worst kind.

Work with Prof. Zheng Song, University of Michigan. Back to all projects.