The COMFORT Localization Benchmark evaluates state-estimation and localization methods on the GrandTour dataset. It is organized with the IROS 2026 workshop Data in Field Robotics: From State Estimation to Navigation.
Develop on two validation missions, then submit trajectories for six hidden test missions. Only test-phase submissions count toward the final ranking.
Opens July 13 at 00:00 UTC and transitions to the test phase on August 3 at 00:00 UTC.
Opens August 3 at 00:00 UTC and closes October 30 at 23:59 UTC. Reference trajectories remain hidden.
The final benchmark ranking uses the average Absolute Trajectory Error (ATE) from the test phase; lower is better. Award-eligible submissions must be made between August 3, 2026 at 00:00 UTC and September 21, 2026 at 23:59 Anywhere on Earth (AOE), equivalent to September 22 at 11:59 UTC. Novel methods are prioritized during award selection, and recipients must present their results at the IROS 2026 workshop.
Include every mission file in the selected phase for a complete score. Mission links open the corresponding entry in the GrandTour data overview.
heap-1.tumarc-3.tumLimit: 20 per day and 100 total.
arc-2.tumarc-7.tumcon-4.tumeig-1.tumsnow-2.tumspx-2.tumLimit: 15 per day and 60 total.
Upload one .zip archive containing TUM trajectory files at the archive root—do not add subdirectories. The archive itself may have any name. Mission-file matching is case-insensitive and accepts .tum or .txt extensions.
comfort_submission.zip ├── heap-1.tum └── arc-3.tum
# timestamp x y z qx qy qz qw 1731588412.024509 -13.128140 -4.550792 -0.846991 0 0 0 1 1731588412.109509 -13.128138 -4.551096 -0.847018 0 0 0 1 ...
Strictly increasing timestamps are recommended. Duplicate or non-increasing timestamps produce a warning but can still be scored when timestamp association succeeds.
Every row must provide T_world_prism: the pose of the robot's prism frame in the estimator's world or odometry frame. Do not submit the origin of base, an IMU, LiDAR, camera, or another sensor.
T_world_prism(t) = T_world_source(t) * T_source_prism p_world_prism(t) = p_world_source(t) + R_world_source(t) * p_source_prism
The fixed source-to-prism offset is expressed in the source frame and must rotate with the robot. See Sensor Frames for the platform frame definitions.
All metrics are reported in meters, and lower is better. The leaderboard is sorted by average ATE across every mission in the submitted phase.
| Metric | Description | Reported value |
|---|---|---|
| ATE | Absolute Translation Error after trajectory alignment | RMSE |
| RTE | Relative Translation Error over relative pose differences | RMSE |
| LE | Last Error from the final absolute trajectory sample | Final sample |