OpenVLA: LIBERO policy evaluation

Pinned OpenVLA policy rollout/evaluation entrypoint for one explicitly selected LIBERO simulation task suite. Statically reviewed source only; not run and no Hub preparation adapter.

openvla-experiments-robot-libero-run-libero-eval-py | Record version: source-369d9fc3d59067998ebfadafbd516872c7ff6f9f06a0a48cda8b6ca48051de77

Record JSON | Markdown | Pinned upstream source

Access

Fetch the pinned repository, verify this source checksum, and use upstream installation and invocation. This record does not install or execute code.

Hub preparation adapter: false. Follow the upstream setup and invocation contract.

Source and license

Repository: https://github.com/openvla/openvla
Revision: c8f03f48af692657d3060c19588038c7220e9af9
Path: experiments/robot/libero/run_libero_eval.py
Source SHA256: 369d9fc3d59067998ebfadafbd516872c7ff6f9f06a0a48cda8b6ca48051de77
Original authors: Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti
License: MIT
License scope: Selected first-party Python workflow source only. Preserve upstream copyright and per-file notices. Model weights, datasets, third-party code, robot drivers/firmware and services retain separate terms; no rights for those assets are inferred.
License SHA256: c4ee2ba5958af03d74b1d3dfa174e3749171d689fb4c23cc170749c05ae3eeb5
Hub commit: 1d6d9eb134e397d910265a8d6e7d8f257f9e8d7f

Artifact

{
  "kind": "workflow",
  "format": "python",
  "workflow_role": "policy-evaluation"
}

Requirements

{
  "python": "Project metadata >=3.8; this reproduction setup specifically reports Python 3.10.13.",
  "packages": [
    "Pinned OpenVLA checkout",
    "Upstream PyTorch 2.2.0, transformers 4.40.1, timm 0.9.10 and tokenizers 0.19.1 pins",
    "Workflow-specific dependencies in pinned README and source"
  ],
  "display": "Hardware: README reports LIBERO evaluations on an NVIDIA A100 and pins its software versions; no CPU verification or universal VRAM minimum is declared by this review. Weights: Requires a selected pretrained checkpoint. README's per-suite model-ID examples automatically retrieve weights if executed; no suite/model cross-product is expanded into records here. Datasets: Requires the separately installed LIBERO simulator and task/init-state assets. README marks modified training-data download optional for evaluation because it supplies pretrained checkpoints. Services/auth: W&B logging is optional, controlled by use_wandb=False by default. No mandatory API key is declared in this selected script.",
  "execution_domain": "simulation",
  "source_imports": [
    "dataclasses",
    "draccus",
    "experiments.robot.libero.libero_utils",
    "experiments.robot.openvla_utils",
    "experiments.robot.robot_utils",
    "libero.libero",
    "numpy",
    "os",
    "pathlib",
    "sys",
    "tqdm",
    "typing",
    "wandb"
  ],
  "notes": "LIBERO task-suite choices are parameters of this one source entrypoint. No Panda/Franka label is inferred from external simulator knowledge. Code licensing does not establish model, checkpoint, dataset or service permission.",
  "declared_requirements": {
    "python": "Project metadata >=3.8; this reproduction setup specifically reports Python 3.10.13.",
    "weights": "Requires a selected pretrained checkpoint. README's per-suite model-ID examples automatically retrieve weights if executed; no suite/model cross-product is expanded into records here.",
    "datasets": "Requires the separately installed LIBERO simulator and task/init-state assets. README marks modified training-data download optional for evaluation because it supplies pretrained checkpoints.",
    "hardware": "README reports LIBERO evaluations on an NVIDIA A100 and pins its software versions; no CPU verification or universal VRAM minimum is declared by this review.",
    "external_services_and_auth": "W&B logging is optional, controlled by use_wandb=False by default. No mandatory API key is declared in this selected script.",
    "evidence": [
      {
        "anchor": "def eval_libero",
        "line": 91
      },
      {
        "anchor": "use_wandb: bool = False",
        "line": 81
      },
      {
        "anchor": "env.step(action.tolist())",
        "line": 228
      }
    ],
    "license_scopes": {
      "code": "MIT for selected first-party workflow source.",
      "weights": "README explicitly says the released OpenVLA models derive from Llama-2 and are subject to the Llama Community License. No checkpoint/model card/base-model license was fetched or evaluated here; MIT is asserted only for the selected first-party Python source.",
      "datasets": "README names BridgeData V2, Open X-Embodiment and LIBERO datasets. Individual dataset terms were not reviewed, and repository MIT is not applied to them.",
      "third_party_code": "External dependencies retain their own licenses."
    },
    "interpretation": "Pinned upstream declarations only; no dependency resolution, model/data access, compatibility inference or runtime validation was performed."
  }
}

Invocation

{
  "entrypoint": "python experiments/robot/libero/run_libero_eval.py --model_family openvla --pretrained_checkpoint <approved-checkpoint> --task_suite_name <suite> --center_crop <reviewed-setting>",
  "notes": "LIBERO task-suite choices are parameters of this one source entrypoint. No Panda/Franka label is inferred from external simulator knowledge."
}

Explicit compatibility

[]

Missing environment fields remain unknown. Discovery labels do not establish robot compatibility.

Evidence

Level: source_reviewed; runtime verification: not_run.

Verified exact source, applicable license and author/setup document bytes at a full Git commit; parsed the Python AST and reviewed this guarded operational entrypoint and its explicit license notices. No upstream code, installation, robot command, dataset, mesh or checkpoint was executed or loaded.

Source review and contributor reports do not establish task completion. Preserve upstream attribution and licenses.