LIRRA  1.0.0
Loading...
Searching...
No Matches
Installation & Usage

Clone the Repository

git clone https://github.com/evannsmc/rl-stl.git
cd rl-stl
git lfs install && git lfs pull # trained policies, datasets, videos (runs/**)

Run outputs under runs/ (.npz, .pkl, .mp4, .gif) are stored with Git LFS. Without git lfs pull those files are small text pointers, and the trained policies cannot be loaded. To fetch only part of them, e.g. the videos:

git lfs pull --include "runs/patrol_videos/**"

Environment

./setup.sh # cuda12 extra if nvidia-smi sees a GPU, otherwise CPU
./setup.sh --gpu # force CUDA 12; --cpu and --wandb are also available
source .venv/bin/activate

setup.sh installs uv if it is missing, runs uv sync --locked (Python 3.12, the dev group, and rl_stl installed editable from src/), checks that rl_stl imports from src/ and which device JAX sees, registers an rl-stl Jupyter kernel, and warns if no Gurobi licence is found. It is safe to re-run. The manual equivalent is uv sync (CPU) or uv sync --extra cuda12 (NVIDIA GPU, then uv run --no-sync ...).

Pinned stack: MuJoCo/MJX 3.13.0, MuJoCo Playground 0.2.0, Brax 0.14.2, JAX 0.11.1, Flax 0.12.9, stlpy 0.3.0 and gurobipy 13.0.3.

  • Gurobi. The pip licence is size-limited: the L1-cost double-integrator problems fit, but the X2 and blimp synthesis needs an academic licence. See docs/guides/gurobi_setup.pdf.
  • Device. RL_STL_DEVICE=cpu|gpu|auto (default auto) selects where JAX, MJX and Brax run.
  • Rendering. The MuJoCo views need an offscreen OpenGL backend, e.g. MUJOCO_GL=egl.

Tests

uv run pytest -n 8 -m "not slow" # fast unit tests
uv run pytest -m slow # JIT and training smoke tests

Patrol Pipeline

Each stage writes a run directory under runs/ containing the resolved configuration, seeds, git revision, package versions, hardware, logs and a summary. <sys> is di, x2 or blimp. <spec> selects one of the eight specifications:

<spec> Specification
theta, phi θ and φ on the planar square, exact boundaries
theta_plus, phi_plus θ and φ on the 3-D cube, exact boundaries
theta_u, phi_u θ and φ on the square, uncertain boundaries
theta_plus_u, phi_plus_u θ and φ on the cube, uncertain boundaries
Stage Command Output
Expert dataset experiments/patrol/dataset.py --system <sys> --spec <spec> runs/patrol_dataset/<sys>/
Behavior cloning experiments/patrol/train_bc.py --system <sys> --spec <spec> --data <dataset.npz> runs/patrol_bc/<sys>/
DAgger, margin BC, PPO cluster/dagger_ppo_pipeline.sh <sys> <spec> <bc policy.npz> <dataset.npz> runs/patrol_bc/, runs/patrol_ppo/
PPO only experiments/patrol/train_ppo.py --system <sys> --spec <spec> --init <policy.npz> runs/patrol_ppo/<sys>/
Evaluation experiments/patrol/evaluate.py --system <sys> --spec <spec> --policy <policy.npz> runs/patrol_eval/<sys>/
Videos experiments/patrol/render_episode.py --system <sys> --spec <spec> --policy <policy.npz> --view composite --gif runs/patrol_videos/<sys>/
Results tables experiments/patrol/main_table.py docs/results/main_table.md

Prefix each Python command with uv run python. The DAgger script is a plain shell script that runs every stage on the current machine (BC and PPO on the GPU if there is one). experiments/patrol/pipeline_stage.py runs one stage using the newest matching run as its input, so stages can be chained without copying paths by hand:

uv run python experiments/patrol/pipeline_stage.py bc --system blimp --spec theta_plus

Evaluation options

evaluate.py --pairs selects the loops to run on the same start states and disturbances:

Pair Loop
always_on standard I-STL runtime assurance: synthesis at every planning step
shield LIRRA: verified proposals with delay-compensated fallback
policy_only the learned proposer flown without verification
raw_trigger proposals accepted on nominal robustness alone (no tubes or terminal sets)
no_delay_comp fallback synthesis rooted at the measured instead of the predicted state

--latency-mode measured uses real solver times instead of a fixed budget of \(d\) steps, and --nominal-predicates (uncertain specifications only) plans and checks with the nominal boundaries while scoring against the true region.

Timing and CPU Energy

evaluate.py records the per-call wall time of a solve, a check and a policy inference. To also record CPU package energy from Intel RAPL counters (idle baseline subtracted), run it through the energy wrapper:

experiments/run_with_energy.sh experiments/patrol/evaluate.py --system blimp --spec theta \
--policy runs/patrol_bc/blimp/<run>/policy.npz --seeds 10

Without readable RAPL counters, energy is reported as NaN.