A Nav2 evaluation platform, not just a test runner.

Write navigation scenarios in YAML, run them from one CLI, then rank planners on a leaderboard, watch quality trend across commits, and replay trajectories over the map.

See the Benchmark Quickstart Evaluation Docs

python3 -m pip install -e .
nav2_scenario_runner init .
nav2_scenario_runner run scenarios/ \
  --report-dir reports/ \
  --html-report index.html

HTML report generated from a real Nav2 TurtleBot3 scenario in Gazebo, including metrics and robot trajectory — Real Nav2/Gazebo run metrics + sampled /odom trajectory

Live Benchmark Dashboards

Generated by the runner itself on every deploy — each one a single self-contained HTML file.

Measured Real Nav2 + Gazebo, run in CI

Live metrics from the Nav2 Real Benchmark workflow — a full ROS 2 Jazzy + Nav2 + Gazebo stack navigating the tb3_sandbox world on a GPU-less runner. The job auto-commits each run under examples/benchmark/real/, so this updates itself every week.

scenarios · real Nav2 + Gazebo

Explorer · Measured ▶

Scrub the robot along the real /odom trajectories over the tb3_sandbox occupancy map.

weekly

real-stack run · tracked per commit

Trend · Measured

Pass rate and real travel_time / path_length across each weekly real-stack run.

Demo Deterministic sample fixtures

Representative fixtures (navfn / smac / teb) from the example benchmark suite so multi-planner comparison renders deterministically without a GPU/sim.

Interactive explorer: toggle planners and scrub the robot along trajectories over a warehouse map

Explorer · Interactive ▶

Toggle planners, pick a scenario, and scrub/play the robot along every trajectory over the live map.

Leaderboard ranking three Nav2 planners by composite score

Evaluation · Leaderboard

Rank planners 0–100 across a scenario suite with metric-comparison bars.

Line charts of navigation metrics trending across commits

Trend · Over Time

Watch pass rate and per-scenario metrics drift across commits to catch regressions.

Animated robot trajectory replayed over a warehouse occupancy map

Replay · On The Map

Animate the robot along its /odom path over the real occupancy grid.

Reproduce locally: bash scripts/build_dashboards.sh

Built For Navigation Regression

Scenario YAML

Review goals, waits, dynamic obstacles, and assertions as code in pull requests.

Nav2 Semantics

Track goal success, travel time, path length, replans, recoveries, and collision signals.

CI Artifacts

Emit JSON, JUnit XML, Markdown, HTML reports, GitHub summaries, and trace timelines.

One Scenario, Reviewable Output

scenario.yaml

apiVersion: nav2.scenario/v1alpha1
kind: Scenario
metadata:
  name: straight_line_goal
  tags: [smoke, navigation]
steps:
  - wait_for_nav2_active: {timeout: 30}
  - set_initial_pose: {x: 0, y: 0, yaw: 0}
  - send_goal:
      name: main_goal
      pose: {x: 10, y: 0, yaw: 0}
  - expect_goal_reached:
      goal: main_goal
      timeout: 60
assertions:
  - goal_reached: {}
  - path_length: {max: 13.0}
  - travel_time: {max: 60.0}

CI summary

Scenario	Status	Time	Path	Replan
straight_line_goal	passed	9.44s	2.28m	9
dynamic_obstacle	warning	31.8s	11.4m	6

Documentation

ArchitectureDesign goals, components, roadmap Scenario DSLYAML schema, actions, assertions MetricsOutcome, quality, behavior, regression EvaluationLeaderboard scoring across configs TrendRecord runs, track quality over time ReplayAnimated trajectories over the map PR Benchmark BotSticky PR comment, badges, leaderboard CI/CDGitHub Actions, reports, artifacts Gazebo SimAdapter lifecycle and execution DoctorEnvironment checks and diagnostics AdaptersGazebo, Webots, Isaac, RMF boundaries PluginsAction, assertion, metric, reporter plugins