MuJoCo Playground (MJX/JAX) hover task for the Menagerie Skydio X2.
Action
Normalized CTBR ``a = [T, p, q, r] in [-1, 1]^4`` (:mod:`.ctbr`). The action is
held for ``n_substeps = ctrl_dt / sim_dt`` physics steps; the body-rate loop and
the prioritized mixer run at every physics step on the measured body rates.
Observation (flat, float32; frames stated per block)
- ``pos_err_world`` (3): ``p_target - p_com`` in world frame [m]; ``p_target`` is
fixed (hover) or sampled per episode within ``target_range`` (waypoint variant)
- ``vel_com_world`` (3): COM linear velocity, world frame [m/s]
- ``rot_body_to_world`` (9): rotation matrix, row-major (encodes gravity direction and yaw)
- ``ang_vel_body`` (3): body-frame angular velocity (gyro) [rad/s]
- ``last_action`` (4)
- ``context`` (3): ``[time_to_deadline / deadline, in_box_now, episode_time_fraction]``
-- STL/task context slots (clock and instantaneous obligation predicate)
- ``stl_window`` (``window * 3``, only if ``stl.enabled``): the last ``window``
COM positions relative to the target, oldest first, so the windowed STL
penalty below is a function of the observation (Markov).
Reward (task terms; each term is logged under ``reward/<name>``)
``pos`` = exp(-|e_p|^2 / sigma_pos^2), ``vel`` = exp(-|v|^2 / sigma_vel^2),
``upright`` = (1 + R_zz) / 2, ``ang_vel`` = |w|^2, ``action_rate`` = |a - a_prev|^2,
``alive`` = 1; ``reward = sum(scale_i * term_i)`` (penalty scales are negative).
Optional STL term (``stl.enabled``; disabled by default = task reward only)
Specification (built with stlpy, selected by ``stl.spec``) over the signal
``y = p_com - p_target``:
- ``hover_box`` (default): ``phi = G_[0, window-1] (y in box(halfwidth))`` --
stay inside the box for the whole window;
- ``periodic_exit``: ``phi = F_[0, window-1] (y not in box(halfwidth))`` -- leave
the box at least once per window, i.e. never dwell inside it for ``window``
consecutive samples. Scored at every step, this is the sliding-window form of
``G F_[0, window-1] (not in box)``.
At control step ``k`` the realized window ``y_{k-window+1..k}`` is scored with the
compiled JAX robustness and the step receives ``-stl_scale * max(0, delta - rho_k)``.
``rho_k`` is computed (and logged) whether or not the term is enabled, so a
task-only policy can be checked against the same spec. Timing and credit
assignment are explicit:
- the term is **zero before** ``k_deadline + window - 1`` (``k_deadline =
deadline_s / ctrl_dt``): the obligation holds from the deadline on, and only
fully post-deadline windows are scored;
- ``rho_k`` is a *past-time sliding-window* value: one sample that decides the
window (an out-of-box sample for ``hover_box``, the only exit for
``periodic_exit``) enters the robustness of each of the ``window`` steps whose
window contains it (credit is spread over the window, not assigned to a single
transition);
- no trajectory-level robustness is copied to all transitions.
Termination
NaN/Inf in state or observation, tilt beyond ``max_tilt_deg``, COM below
``min_z``, or position error above ``max_pos_err``.
Floor contacts are disabled by default (``contacts=False``): the task terminates
above the floor, and disabling contacts removes collision cost from the batch.
Brax's ``BraxAutoResetWrapper`` with ``full_reset=False`` restores ``data``/``obs``
but not ``info``; the env therefore re-initializes its per-episode info whenever
``data.time == 0`` at the start of a step.