HTS: Sub-50µs Time Sync for PX4 Multi-Sensor Fusion — No Extra Hardware Required

Originally published by 码农实验室 (CoderLab) on WeChat

Multi-sensor fusion on drones is hard. You have an IMU, a GPS, a camera, and a LiDAR — all running on independent clocks — and you expect the Extended Kalman Filter to treat their data as if it came from the same instant. It doesn’t. The result? Trajectory drift, fusion jumps, and outright instability at high speeds.

After two months of hitting walls with PTP and GPS PPS hardware synchronization, one PX4 developer tried a counterintuitive approach: instead of making sensors agree on time, teach the system to estimate time error as a state variable. The result — dubbed HTS (Hybrid Time Synchronization) — delivers 10–50µs timestamp accuracy with zero additional hardware. At Aomway, we break down exactly how it works.

1. The Problem: 5ms Time Error = 83cm Spatial Blind Zone

Here’s the math that motivated this work:

Drone at 60 km/h, IMU and camera timestamps differ by 5ms → 83cm spatial misalignment.

When your EKF thinks four sensors saw “the same moment,” they actually disagreed by nearly a meter. Two traditional approaches fail:

• Hard sync (GPS PPS): Requires extra wiring, fails indoors, and demands PPS-compatible sensors — cost prohibitive.
• Soft sync (PTP/NTP): Network latency is unknown, dynamic, and varies with temperature and CPU load. Tuning for weeks still leaves 1–2ms error.

2. The Core Idea: Time Error as a State Variable

The breakthrough insight:

Time error isn’t noise to eliminate — it’s a state to estimate.

Just as IMU gyro bias is estimated and compensated within the EKF, clock bias can be treated the same way.

State Vector Extension

Traditional EKF state:

x = [position, velocity, quaternion, bias_gyro, bias_acc]

HTS extended state:

x = [position, velocity, quaternion, bias_gyro, bias_acc, δt, δṫ]

Symbol	Meaning
δt	Time offset (clock bias)
δṫ	Time drift rate (clock drift)

Time State Prediction

The time error follows a random walk model — it’s not constant, it evolves:

// Time state prediction
δt(k+1) = δt(k) + δṫ(k) × dt + w_t
δṫ(k+1) = δṫ(k) + w_d

3. Four Types of Time Observations

With the state defined, HTS requires observation models. The developer designed four distinct “time sensors”:

3.1 GPS PPS Observation (When Available)

PPS provides an absolute time anchor. The observation equation is straightforward:

z_pps = t_system + δt + v_pps

The residual feeds directly into the EKF update. In outdoor tests, this achieves 10–30µs accuracy after convergence.

3.2 IMU Time Residual (The Clever Part)

This is where the method gets elegant. IMU pre-integration residuals inherently contain timing information:

When the time offset estimate is wrong, IMU integration produces systematic position errors relative to GPS or visual odometry. The EKF can back-calculate time error from these position mismatches.

Vector3f velocity_error = _velocity × dt_error;
return _position + velocity_error;

3.3 Camera Exposure Delay Model

The camera presents the most complex timing challenge:

t_effective = trigger_time + δt + exposure_delay/2 + readout_delay

Exposure and readout delays are calibrated offline. The clock bias is estimated online by the EKF.

3.4 ROS2 Companion Computer Observation

If a companion computer is present (e.g., Jetson Nano), its clock provides yet another time observation:

z_ros = t_ros - t_px4 = δt + v

4. PX4 EKF2 Implementation

The implementation extends Ekf.cpp and ekf2_main.cpp in PX4:

• Two new state variables: _delta_t and _delta_t_dot
• Covariance matrix extended with _P_time (2×2)
• predictTimeState(dt) for the prediction step
• updateTimeFromPPS(), updateTimeFromIMU(), updateTimeFromROS() for observation updates
• getCorrectedTime(raw_time) to retrieve corrected timestamps

5. Real-World Validation Results

Test Platform: PX4 FMUv5 (STM32H7), ICM-20602 IMU, u-blox M8P GPS, FLIR BFS-U3 camera, Jetson Nano companion

Scenario 1: Indoor Static (No GPS) — 30 minutes

• Initial time offset: ~2ms
• Converged offset: 35–50µs
• Estimated drift rate: 2.3e-7 s/s

Scenario 2: Indoor Dynamic Flight (No GPS)

• Speed: 3–5 m/s
• Sync accuracy: 40–80µs
• EKF position error reduced vs baseline

Scenario 3: Outdoor with GPS PPS

• Post-sync accuracy: 10–30µs
• >1 hour runtime: stable convergence, zero drift accumulation

Metric	Traditional	HTS
Time Error	0.5–5ms	10–50µs
Time Jitter	High	Extremely Low
EKF Innovation Fluctuation	Visible	Stable
Visual Fusion Jumping	Intermittent	Smooth & Continuous
High-Speed Trajectory Drift	Significant	Well-Converged
GPS-Denied Operation	Unavailable	Fully Operational

Real Impact Example

LiDAR-camera fusion at 50 km/h:

• Traditional (3ms error): 42cm spatial misalignment — unsafe for obstacle avoidance
• HTS (50µs error): 0.7cm spatial misalignment — reliable for safety-critical navigation

6. Engineering Pitfalls & Lessons Learned

Pitfall 1: Timestamp at Data Ready, Not in Driver

Taking timestamps in the driver layer produced 100–500µs jitter. Moving to the SPI DMA completion interrupt reduced it to <10µs:

// In stm32_irq_handler
void IMU::dataReadyInterrupt() {
    uint64_t timestamp = hrt_absolute_time();
    _imu_data.timestamp = timestamp; // Stamp immediately
    // THEN start DMA read
}

Pitfall 2: Tuning Noise Parameters Q_t and Q_d

Parameter	Value	Effect if Wrong
Q_t	1e-6	Too large → oscillation; Too small → slow convergence
Q_d	1e-8	Must match hardware clock drift characteristics

Pitfall 3: ROS2 Clock Can Be Tens of Milliseconds Off

The ROS2 system clock may differ from PX4’s HRT by tens of milliseconds. Solution: one-time sync at boot, then continuous PTP/NTP correction.

7. Summary: Why HTS Matters

The core philosophy:

Instead of eliminating time error, estimate it.

Advantages

• No extra hardware synchronization required
• Works in GPS-denied environments (indoors, forests, urban canyons)
• Microsecond-level precision sufficient for high-speed flight
• Online estimation adapts to clock drift automatically
• Code planned for PR to PX4 mainline

Limitations

• Requires modifying EKF core — moderately invasive
• Parameter tuning requires experience
• Initial convergence takes a few seconds

Frequently Asked Questions

---

Any questions pls contact: [email protected]