LIRRA  1.0.0
Loading...
Searching...
No Matches
rl_stl.policies.segment Namespace Reference

Classes

class  LearnedProposer
 
class  LevelTables
 
class  SegmentPolicy
 

Functions

LevelTables make_level_tables (list problems, PatrolTask task, np.ndarray waypoints, np.ndarray w_levels, str reference="rerooted", float ref_input_scale=1.0)
 
 outside_margin (y, float half_width, xp=np)
 
 interval_margin (LevelTables T, y, r=0.0, xp=np)
 
 signal_margin (LevelTables T, y, xp=np)
 
 features (LevelTables T, x, hist_y, hist_mask, waypoint_index, level, xp=np, x_ref=None)
 
 squash (LevelTables T, a, level, xp=np)
 
np.ndarray unsquash (LevelTables T, np.ndarray U, int level, float eps=1e-6)
 
 rollout_jax (A, B, x0, U)
 
 patrol_certificate (LevelTables T, x0, U, hist_y, hist_mask, first_window, level)
 
list[dict] init_mlp (key, list[int] sizes)
 
 mlp (list[dict] params, x)
 

Variables

int BIG = 1e3
 

Detailed Description

Learned segment proposer: features, differentiable certificate, network, BC training.

The policy maps an observation to ``N`` synthesis-model inputs (a trajectory segment through
the ``f_syn`` rollout). One feature definition is used at runtime (numpy, from a
:class:`~rl_stl.supervisor.receding.ProposalContext`) and in training (JAX).

Observation (STL context and the disturbance *bound*, never the realised
disturbance):

* state ``x`` scaled by the state box half-widths;
* measured output history ``y_{t-h+1}..y_{t-1}`` (scaled), a validity mask (samples before
  the episode start are not enforced), and each sample's outside-``S`` margin;
* one-hot of the waypoint the mission reference is heading for (4 waypoints for the square, 6 for
  the cube specs);
* the disturbance-bound level ``w / w_max`` (for the ``_u`` specs followed by the predicate
  uncertainty bounds ``(pred_offset, pred_rotation)``, scaled the same way; they are equal at every
  level, so the two features are constant 1: they document the bound and keep the interface ready
  for randomised predicate bounds);
* clock-indexed reference only (``LevelTables.ref_offsets``): the reference minus the state,
  ``(x_nom(k + j) - x_k)`` scaled, at the offsets ``j`` (the rerooted reference starts at
  ``x_k`` and is a function of the other features, the clock reference is not).

Action: ``a in R^{N x m}``; inputs ``u_i = c_i + r_i tanh(a_i)`` inside the tightened input box
``[u_lo + kz(i), u_hi - kz(i)]`` of a measured root, so input feasibility holds by
construction.

:func:`patrol_certificate` is a JAX implementation of the runtime check
(:func:`rl_stl.synthesis.check.check_plan`) for the patrol formulas: it returns
``min(rho_lower - delta, terminal margin, state-bound margin)``, non-negative iff the
proposal is a feasible point of the synthesis problem (up to tolerances). It is
differentiable almost everywhere and is used as a hinge in the BC loss and as the PPO
reward; the numpy check remains the authority at runtime.

Uncertain-predicate specs (``*_u``, :attr:`LevelTables.interval`): the face predicates are interval
predicates ``[a]^T y - [b] >= 0`` and every place that used the unit-normal margin ``|y_i| - c`` uses
the I-STL lower endpoint instead (:func:`signal_margin`: history margin feature, Boolean history/root
samples, and the planned margin with the tube box in :func:`patrol_certificate`).

Function Documentation

◆ features()

rl_stl.policies.segment.features ( LevelTables  T,
  x,
  hist_y,
  hist_mask,
  waypoint_index,
  level,
  xp = np,
  x_ref = None 
)
Observation vector (numpy or ``jax.numpy`` via ``xp``); ``x_ref (N+1, n)``: the mission
reference of a plan rooted here (needed iff ``T.ref_offsets``).

◆ init_mlp()

list[dict] rl_stl.policies.segment.init_mlp (   key,
list[int]  sizes 
)

◆ interval_margin()

rl_stl.policies.segment.interval_margin ( LevelTables  T,
  y,
  r = 0.0,
  xp = np 
)
``max_f lower([a_f]^T [y - r, y + r] - [b_f])``: I-STL lower endpoint of ``x notin S`` (interval
face predicates; ``y``, ``r``: ``(..., p)``).

◆ make_level_tables()

LevelTables rl_stl.policies.segment.make_level_tables ( list  problems,
PatrolTask  task,
np.ndarray  waypoints,
np.ndarray  w_levels,
str   reference = "rerooted",
float   ref_input_scale = 1.0 
)
Stack the problems built for each disturbance level (same ``N``, model, bounds).

``reference="clock"`` adds the reference-offset features at ``j = 0, N//2, N``;
``ref_input_scale`` is the clock reference's input saturation (ignored when rerooted).

◆ mlp()

rl_stl.policies.segment.mlp ( list[dict]  params,
  x 
)

◆ outside_margin()

rl_stl.policies.segment.outside_margin (   y,
float  half_width,
  xp = np 
)
``max_i |y_i| - c``: signed outside-``S`` margin (>= 0 outside) over every signal axis
(square for 2-D ``y``, cube for 3-D ``y``).

◆ patrol_certificate()

rl_stl.policies.segment.patrol_certificate ( LevelTables  T,
  x0,
  U,
  hist_y,
  hist_mask,
  first_window,
  level 
)
``(cert, parts)``; ``cert >= 0`` iff the plan passes the runtime check (JAX).

◆ rollout_jax()

rl_stl.policies.segment.rollout_jax (   A,
  B,
  x0,
  U 
)

◆ signal_margin()

rl_stl.policies.segment.signal_margin ( LevelTables  T,
  y,
  xp = np 
)
Outside-``S`` margin of measured samples: :func:`outside_margin` for exact predicates, the
interval lower endpoint for the ``_u`` specs (``>= 0`` iff the sample satisfies ``x notin S``).

◆ squash()

rl_stl.policies.segment.squash ( LevelTables  T,
  a,
  level,
  xp = np 
)
Raw network output ``(N*m,)`` -> inputs ``(N, m)`` inside the tightened box.

◆ unsquash()

np.ndarray rl_stl.policies.segment.unsquash ( LevelTables  T,
np.ndarray  U,
int  level,
float   eps = 1e-6 
)

Variable Documentation

◆ BIG

int rl_stl.policies.segment.BIG = 1e3