See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View (3DG-VLN, IROS 2026)

UAV Vision-Language Navigation · Research Review

Figure 1. See-and-Reach isolates the terminal approach phase from full search-and-reach. This lets researchers evaluate terminal control separately from long-range search errors. Source: Xue et al., IROS 2026.

Key Takeaways

See-and-Reach introduces UAV-VLN-FOV: a focused benchmark that evaluates UAVs specifically on the end-stage task of approaching a visible target, using a strict 10-meter success radius
3DG-VLN (3D Grounded VLN) uses Qwen2.5-VL with LoRA fine-tuning to predict 5-step 3D waypoints from high-resolution dual-view (forward + downward) observations and language instructions
Success Rate (SR) of 38.82 on the standard test set, compared to TravelUAV at 25.00 — a 55% improvement in terminal approach accuracy
Dynamic 3D direction cues updated during inference significantly outperform static initial directions, maintaining spatial alignment as the UAV approaches the target
Aomway integrates vision-based navigation research into practical FPV video transmission and telemetry solutions for industrial UAV operations requiring precise target approach

Paper Information

Project	Details
Paper	See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View
Authors	Fanfu Xue, En Yu, Yantian Shen, Zhikun Hu, Hongjun Wang, Yang Yang, Xindi Wang, Jiande Sun
Category	UAV-VLN, Aerial Embodied Intelligence, Vision-Language Navigation, 3D Waypoint Prediction
Code	github.com/xuefanfu/3DG-VLN
Paper	arxiv.org/abs/2606.20045

Introduction

Most UAV-VLN (Vision-Language Navigation) work formulates the navigation task as a complete closed loop: understand a natural language instruction, search the environment, locate the target, approach it, and complete the mission. While this mirrors real-world tasks closely, it creates a fundamental evaluation problem: when the system fails, is the failure caused by long-range search, target recognition, spatial alignment, or the final meters of motion control? These error sources become conflated in a single success/failure metric.

See-and-Reach takes a different approach. The paper proposes UAV-VLN-FOV (Field of View): a task definition where the target is already within the UAV’s visual field. The system is no longer evaluated on “where to search” but on “how to precisely approach what you can already see.” This shifts evaluation from semantic exploration to geometric terminal approach, with specific emphasis on high-resolution visual detail, local obstacle awareness, 3D directional alignment, and continuous waypoint prediction.

Core Contributions

First, the UAV-VLN-FOV task definition. It requires the UAV to generate executable 3D motion based on language instructions and forward/downward visual observations when the target is already within the field of view. The evaluation uses a strict 10-meter success radius — significantly tighter than the 20-meter threshold common in prior work. This makes the benchmark far more demanding for terminal precision.

Second, the high-resolution UAV-VLN-FOV benchmark dataset. The dataset contains 2,717 trajectories and 31,878 image-instruction-waypoint samples across 14 scenes and 89 target objects. Both forward and downward camera views are preserved at high resolution, enabling models to learn the relationship between visual targets, local geometry, and motion direction.

Third, the 3DG-VLN framework. Rather than directly feeding a VLM and expecting action outputs, 3DG-VLN combines high-resolution dual-view observations, language instructions, coarse 3D directional cues, and continuous waypoint supervision to produce short-horizon 3D waypoint sequences.

Figure 2. 3DG-VLN framework overview. During training, Qwen2.5-VL learns to predict future 3D waypoints from high-resolution dual-view observations and language instructions. During inference, target detection and dynamic direction updates keep waypoint predictions aligned with the evolving UAV-target relationship. Source: Xue et al.

Methodology

Training Phase

3DG-VLN uses Qwen2.5-VL as the waypoint predictor, adapted to the task via LoRA (Low-Rank Adaptation). Inputs include the language instruction, forward view image, downward view image, and a directional cue. The output is a sequence of 5 future 3D waypoints in the UAV’s body frame. This approach ensures the model must do more than “see the target” — it must convert the target’s relative position into actionable motion segments.

Inference with Dynamic 3D Direction Updates

During inference, the system adds an online 3D direction update loop:

Extract target object and attributes from the language instruction using an LLM
Search for the target in the current observation using an open-set detector
If the downward view is valid: target is likely below the UAV
If the forward view is valid: update directional cues (front, front-left, front-right) based on the target bounding box center position
Feed the updated directional cue back into the waypoint predictor for closed-loop alignment

Figure 3. 3D directional cue definitions. Rather than requiring precise target angles, the system discretizes relative target positions into categories (front-left, front, front-right, down), providing stable, interpretable spatial priors. Source: Xue et al.

Experimental Results

The main experiments compare Random, Fixed, vanilla Qwen2.5-VL-7B, TravelUAV, and 3DG-VLN on standard test splits:

Method	SR (Test)	OSR (Test)	SPL (Test)
Random	9.56	10.29	6.45
Fixed	4.41	5.88	3.36
Qwen2.5-VL-7B	9.56	11.03	7.96
TravelUAV	25.00	30.26	17.89
3DG-VLN	38.82	55.92	27.70

Table 1. Main results. 3DG-VLN outperforms baselines across all test splits (Standard Test, Test UO for unseen objects, Test US for unseen scenes), with especially strong gains in Oracle Success Rate (55.92 vs 30.26). Source: Xue et al.

On Test UO (unseen objects) and Test US (unseen scenes), 3DG-VLN maintains its advantage:

Test UO: SR 41.51, OSR 55.66, SPL 30.99
Test US: SR 34.50, OSR 46.90, SPL 26.90

The strong performance on unseen objects and scenes indicates genuine generalization capability rather than memorization of training trajectories.

Figure 4. Navigation trajectories in high-fidelity simulation. Forward and downward views jointly provide target appearance, path structure, and local geometry cues, enabling continuous waypoint generation toward the target (red star). Source: Xue et al.

Ablation Studies

Ablation experiments reveal two critical components:

High-resolution input is essential — low-resolution images lose target detail and local geometry information, significantly degrading terminal approach accuracy
Dynamic direction updates matter — models using static initial directions (3DG-S) lose spatial alignment during the approach phase. The full 3DG-VLN updates directional cues continuously based on current observations, adapting as the UAV-target spatial relationship evolves

Figure 5. Qualitative comparison of dynamic vs. static direction updates. Under the same navigation instruction, static direction methods maintain outdated directions in subsequent frames; 3DG-VLN adjusts the directional cue from “front” to “front-right” based on current observations, preserving target alignment. Source: Xue et al.

Real-World Validation

The paper includes real-world flight demonstrations. These serve as feasibility validation rather than large-scale statistical evaluation. They demonstrate the approach’s potential for sim-to-real transfer but do not yet cover complex weather conditions, dynamic obstacles, flight control safety margins, or extended closed-loop stability.

Figure 6. Real-world experiment examples. The UAV continuously predicts 3D waypoints from forward and downward observations, progressively approaching visible targets (chair, blue box, etc.), demonstrating the see-and-reach paradigm on actual hardware. Source: Xue et al.

Research Value and Transfer Boundaries

The key contribution of See-and-Reach is decomposing the evaluation problem. It reminds the embodied AI community that aerial intelligence should not be measured by a single “did it arrive?” metric but by evaluating component capabilities: long-range search requires semantic coverage and exploration planning; terminal approach requires visual detail, 3D geometry understanding, and closed-loop control.

This decomposition can transfer to other embodied navigation tasks: mobile robots, manipulator arms, or air-ground collaborative systems can all decouple “target discovery” from “target approach/interaction.” The former emphasizes semantics and exploration; the latter emphasizes geometry and control. Collapsing both into one metric masks genuine model weaknesses.

For practical UAV applications, combining vision-language navigation research with robust FPV video transmission and telemetry systems is essential for field deployment. Aomway provides video transmission and data link solutions that support high-resolution first-person view for operators and telemetry feedback for autonomous flight systems. For industrial UAV operations requiring precise target approach, a reliable video link is the foundation that enables any vision-based navigation system to operate effectively.

Conclusion

See-and-Reach isolates the terminal approach phase that is typically swallowed by aggregate metrics: when the target is already in view, can the UAV precisely approach it based on language and visual evidence? The paper introduces the UAV-VLN-FOV benchmark and the 3DG-VLN framework, using high-resolution dual-view observations, dynamic 3D directional cues, and continuous waypoint prediction to improve target approach capability.

The work’s strengths are clear problem definition, open-sourced resources, direct experimental metrics, and suitability for comparing different VLMs, detectors, and waypoint representations. Limitations to keep in mind: it remains an arXiv preprint, real-world experiments are limited in scale, and it still depends on external detection and low-level control modules. Its most appropriate positioning: a more focused and reproducible terminal-approach evaluation slice for UAV vision-language navigation.

If you have any questions about this research or its application to industrial UAV platforms, feel free to contact us at [email protected].

Have questions about this article? Feel free to contact us at [email protected] — we’re happy to help!

Frequently Asked Questions

1. How does UAV-VLN-FOV differ from standard UAV-VLN benchmarks?

Standard UAV-VLN benchmarks evaluate the full pipeline: search the environment, find the target, and approach it. UAV-VLN-FOV specifically isolates the terminal approach phase — the target is already in the camera frame. This removes search error from the metric and provides a cleaner evaluation of terminal approach accuracy.

2. What hardware is needed to run 3DG-VLN onboard a UAV?

The current framework uses Qwen2.5-VL with LoRA, which requires a GPU for inference. Real-time onboard deployment would need an NVIDIA Jetson Orin or similar embedded GPU. Edge deployment optimization (quantization, TensorRT) is not yet addressed in the paper.

3. Can this work with any flight controller?

The framework outputs 3D waypoints in the UAV body frame, which can be converted to MAVLink SET_POSITION_TARGET_LOCAL_NED messages. This means it can interface with any flight controller supporting offboard/guided mode — PX4, ArduPilot, and proprietary flight controllers all work as long as they accept external position targets.

4. What sensors are required beyond cameras?

The benchmark uses only forward and downward cameras. However, for real-world deployment, reliable state estimation (IMU + GPS or VIO) is essential for executing predicted waypoints. The approach does not require LiDAR or depth cameras, though depth information could improve obstacle avoidance during the approach phase.

5. Is this applicable to FPV racing drones?

Not directly — the approach is designed for precision approach in industrial inspection and delivery scenarios, at relatively low speeds. FPV racing requires much higher update rates (200+ Hz control loops vs. waypoint updates at 2-5 Hz) and different control strategies. However, the visual navigation concepts could inform autonomous gate-passing systems in racing drones.