Key Takeaways
- The T1-S build runs one task chain end to end: SLAM + Nav2 navigation, fine docking, then a SmolVLA policy episode at each station.
- Two hosts split the load: a NUC13 onboard computer drives the chassis, localization, docking and the Robot Agent, while a ROG NUC with an RTX 5070 Ti runs SmolVLA inference.
- SmolVLA carries roughly 450 million parameters and generates continuous action chunks with flow matching.
- The policy in this demo came from 60 teleoperated SO101 demonstrations and 3,000 training iterations.
- VLN is deliberately excluded for now because stations A, B and home are already bound to fixed map coordinates.
Adding a SO101 arm to a T1-S rover turns a navigation problem into a mobile manipulation problem, and the hard part is not any single module but the handoff between them. This walkthrough from Aomway lays out a working SLAM + Navigation2 + SmolVLA stack, from hardware wiring and host roles down to the exact launch commands and the nine-step task script that moves the vehicle between two workstations and back.
What this article covers
- The current architecture, host split and task flow.
- SmolVLA fundamentals and how it compares with ACT and pi0.5.
- The difference between VLA and VLN, and why this build uses mapping/localization, Nav2 and fine docking.
- What you need ready before installing.
- Start-up, status checks, task execution and shutdown in a configured environment.
- How to judge results, common problems and feedback for the next stage.
1. What Are We Building?
This tutorial explores a combined SLAM + Navigation2 + VLA architecture.
On top of the existing T1-S smart rover we mounted an open-source SO101 arm, added a chest camera and a gripper-side camera, and tilted the lidar forward.
The result looks like this:
Once the T1-S carries a SO101 arm, how do you get the vehicle to different workstations and then invoke a learned policy to manipulate objects?
This demo links three parts into a single task flow:
- Autonomous navigation: drive to the vicinity of the target workstation using a prepared map and pose configuration.
- Fine docking: adjust the vehicle pose further so the arm starts from a usable condition.
- Arm policy execution: call SmolVLA on the external compute host and let it generate actions from camera frames and arm state.
The configured route is:
-> depart from the start point
-> station A: navigate -> dock -> SO101 SmolVLA policy -> undock
-> station B: navigate -> dock -> SO101 SmolVLA policy -> undock
-> return to start
2. Overall System Design
Hardware notes:
The vehicle modifications are shown above.
Next, the key nodes and the communication links between them.
Host roles:
The system uses a dual-host architecture.
In this architecture the ROS NUC handles model inference, while localization and navigation stay on the T1-S onboard computer. Both hosts sit on the same local network.
Software notes:
For localization and navigation we reuse the existing Fast LIO2 and Navigation2 stack. For training and inference we use SmolVLA.
3. Understanding SmolVLA
3.1 VLA inputs and outputs
VLA stands for Vision-Language-Action. SmolVLA takes multiple RGB images, robot state and a natural-language task description, and generates a future chunk of actions.
The published SmolVLA has roughly 450 million parameters. It ships pretrained weights plus training and inference recipes, and has been tested on platforms such as SO100 and SO101. It pairs a vision-language model with an action expert and generates continuous action chunks through flow matching. See the official introduction.
3.2 Why it fits this project
- Small footprint: the size makes it practical to estimate deployment cost against existing hardware, but real throughput and VRAM usage still need to be measured.
- Pretrained weights are reusable: you adapt existing capability to the target task; results depend on data, camera viewpoints and how actions are defined.
- Language-conditioned input is supported: task text can drive the policy, but the exact strings passed in this project, the model’s ability to distinguish tasks, and its robustness to rewording all need separate verification.
- Asynchronous inference is available: action execution and prediction of the next chunk can overlap to cut waiting time; whether this project realizes the expected gain depends on the actual configuration and runtime measurement.
3.3 How it compares with ACT and pi0.5
The table below compares method positioning. It is not a measured ranking of the three models on this vehicle.
| Item | ACT (original) | SmolVLA | pi0.5 |
|---|---|---|---|
| Basic positioning | Policy learned from demonstrations | Lightweight VLA | VLA focused on open-environment generalization |
| Typical inputs | Images, joint state | Images, robot state, task description | Vision, language and robot state, depending on the implementation |
| Action generation | Transformer predicts action chunks | Flow matching generates action chunks | Flow matching generates action chunks |
| Usual adaptation path | Train a policy on task demonstrations | Pretrained weights plus task adaptation | Pretrained weights plus platform and task adaptation |
| Question worth evaluating here | Can a fixed task be made reliable | How much do pretraining and language conditioning buy | Is the generalization gain worth the extra adaptation and compute |
ACT’s core design includes action chunking, and the original version does not treat natural language as a standard input. See the ACT project page.
pi0.5 studies generalization to new environments by joint training on data from many sources. The paper describes a system that includes high-level semantic subtask prediction; the current openpi repository states that pi0.5 training and inference support only a flow-matching action head, so the capability of the full paper system should not be equated with what you get after downloading the public weights. See the pi0.5 introduction and openpi.
For this tutorial, ACT generalizes poorly and is sensitive to environment change, so it is not a reference point. pi0.5 demands far more compute on both the training and the execution host, and the generalization gain exceeds the extra adaptation work and hardware this experiment justifies — a sledgehammer to crack a nut. Both are left for later validation.
At this stage we integrate SmolVLA into the task chain.
We teleoperated the SO101 leader and follower arms, collected 60 demonstration episodes, ran 3,000 training iterations on the ROG NUC, and preloaded the exported model file into evaluate_smolvla_mutil_task.py ready to be called.
3.4 VLA versus VLN
VLA is a family of modeling approaches covering vision, language and action. VLN (Vision-and-Language Navigation) is a task direction: interpret a language navigation instruction from visual observations and complete the movement. The two are not mutually exclusive categories. A VLA action space can include locomotion, and a VLN system can also use learned policies combined with maps, planning and conventional control.
3.5 Why multi-point navigation uses SLAM/localization + Nav2 instead of VLN
The reasoning below is an engineering choice for the current task conditions, not a measured conclusion about VLN versus Nav2 performance.
- The targets are already known. A, B and home are bound to positions and docking poses in the field. The task does not require the robot to find an unknown workstation from free-form language. Multi-point navigation itself does not require VLN.
- Manipulation needs a repeatable arrival condition. Nav2 brings the vehicle to a pre-docking pose, then the docking module adjusts the manipulation pose. Reaching the vicinity of a station and reaching an arm-usable pose are two separate steps, and choosing Nav2 does not by itself guarantee docking accuracy.
- Modules can be tested and debugged separately. Map and localization, path execution, docking and policy execution can each be checked on their own, so a failure in the full task can be attributed to a specific stage. Actual stability still requires repeated runs.
- It keeps the scope of this stage under control. The current priority is validating the multi-station mobile manipulation chain. Adding VLN would mean also handling language target grounding, visual environment adaptation, chassis interfaces and new evaluation conditions.
3.6 When is VLN worth adding?
VLN or semantic goal understanding deserves a separate evaluation once the requirement becomes “find a workstation from a language description,” “understand signage and navigate in an unmapped scene,” or “convert an open language goal into navigation subgoals.”
A layered combination is a reasonable future direction: a vision-language module proposes goals or subgoals, Nav2 executes geometric navigation, docking handles close-range positioning, and SmolVLA performs the manipulation at the station. Mapping “go to point A” onto an existing coordinate is only a language-to-named-target conversion and is not evidence of open-environment VLN capability. All of this is future work, not a currently implemented feature.
4. Operation
4.1 Onboard host — start the Host
python -m lerobot.robots.T1.t1_host \
--robot-id t1_robot \
--robot-model t1 \
--profile-timing true
What it does:
- Connects the right arm and the chest and wrist cameras.
- Configures position control and enables arm torque.
- Listens on action port 5555 and observation port 5556.
- Receives joint targets and returns arm state and images.
4.2 ROS NUC host — start SmolVLA and the ROS bridge
python3 `examples/T2/evaluate_smolvla_mutil_task.py --management_server ...`
This node connects to the T2 Host, loads the model onto CUDA, and stays resident waiting for tasks to run inference.
python3 examples/T2/t2_lerobot_ros_bridge.py
This node converts ROS requests into local inference service management requests and returns feedback and results.
4.3 Onboard host — start the ROS 2 system
source /opt/ros/humble/setup.bash
source /home/rover/catkin_ws/install/setup.bash
ros2 launch t1_robot_agent t1_system.launch.py
This starts the chassis driver, the lidar and camera drivers, FastLIO2, Navigation2 and the other nodes one by one, along with the t1_robot_agent node.
4.4 Onboard host — start the demo task
Once the conditions above are confirmed, run in a terminal:
ros2 service call \
/t2_robot_agent/execute_demo \
std_srvs/srv/Trigger \
'{}'
This is what actually triggers the task. The current task configuration is:
| Step | Skill | Purpose |
|---|---|---|
| 1 | navigate_to(cabinet_A) |
Navigate to the pre-docking position in front of the cabinet |
| 2 | dock(cabinet_A) |
Fine docking |
| 3 | pick(object_M) |
Run the full policy episode for the first station |
| 4 | undock(cabinet_A) |
Leave the first station |
| 5 | navigate_to(table_B) |
Navigate to the position in front of the table |
| 6 | dock(table_B) |
Fine docking |
| 7 | pick(object_N) |
Run the full policy episode for the second station |
| 8 | undock(table_B) |
Leave the second station |
| 9 | navigate_to(home) |
Return to the start point |
object_M and object_N are internal task identifiers and do not prove the object was recognized automatically. The actual task text, objects and target regions need to be documented in the release material.
Summary
In this installment we took the modified T1-S through a task flow spanning autonomous navigation, fine docking and arm manipulation,
explained the role SmolVLA plays in it,
and covered how it differs from ACT and pi0.5, plus the basic deployment and start-up steps.
In this design the onboard ROS 2 system owns the chassis, localization and navigation, docking and task scheduling. The external NUC owns SmolVLA inference. The onboard hardware service connects the SO101 and the two cameras and carries observations and actions. SmolVLA is responsible for the arm manipulation at each station, while the order of the whole task is still set by configuration.
Beyond the live robot flow, this installment covered SmolVLA’s characteristics, how it differs in positioning from ACT and pi0.5, and the distinction between VLA and VLN. Because the current targets are already configured, movement is handled with mapping/localization, Nav2 and docking; open language goal understanding is a direction for later evaluation. Model performance on this vehicle still needs a unified test.
The development focus at this stage was to connect navigation, docking and a learned policy into one task chain. Repeated testing is still needed to improve success detection, failure handling and scene adaptability. This demo is also meant to collect real requirements, to decide whether the next stage should prioritize lowering the deployment barrier, improving full-task stability, or expanding tasks and running model comparisons.
FAQ
What does SmolVLA actually control in this build?
Only the SO101 arm manipulation at each station. Vehicle movement, localization and docking stay with Fast LIO2, Nav2 and the docking module. The order of the whole task is fixed by configuration, not decided by the model.
Why run inference on a second host instead of the onboard computer?
The NUC13 onboard computer already carries the chassis drivers, localization and the Robot Agent, where timing matters. Offloading SmolVLA to a ROG NUC with an RTX 5070 Ti keeps GPU load off the control loop, at the cost of one extra network hop over 2.5G Ethernet.
How much demonstration data did the policy need?
60 teleoperated episodes on the SO101 leader and follower arms, followed by 3,000 training iterations on the ROG NUC. That is enough for a single-station pick demo, not for broad generalization across objects.
Can I swap in VLN and drop Nav2?
Not for this task. Stations A, B and home are already bound to map coordinates, so there is no language goal to ground. VLN becomes worth evaluating when the robot must find an unregistered station from a description or interpret signage in an unmapped scene.
Why is pi0.5 not used here?
Its compute requirements on both the training and execution host are much higher, and the generalization gain does not justify the added adaptation work and hardware for a two-station demo. The author’s own framing is that it would be a sledgehammer to crack a nut.
About Aomway
Aomway has spent more than a decade building wireless video, datalink and flight-control hardware for FPV and industrial UAV platforms. Explore the Aomway product line for video transmitters, antennas, fiber-optic links and onboard electronics that hold up in real flight conditions.
Have questions about this article? Feel free to contact us at [email protected] — we’re happy to help!
