class

`SushiAI::Optim::LossScaleState`

Host-side state machine tracking dynamic loss scale adjustments.

Declared in: include/SushiAI/optim/loss_scaler.hpp

Public member functions

explicit LossScaleState(LossScaleOptions options={})

Builds a state machine over options.

Parameters

options: The four constants; all must be powers of two.

Exceptions

Error: If a constant is not a power of two, if the bounds do not bracket the initial scale, if growth is below one or backoff is not in (0, 1].

void observe(bool overflow) noexcept

Folds one step's verdict into the state.

Parameters

overflow: Whether any gradient came back non-finite.

double scale() const noexcept

The scale the next step will run at.

double inverse_scale() const noexcept

1 / scale, exactly.

Exact because the scale is a power of two, which is the property the whole design is arranged around: the unscale is a change of exponent and nothing else.

bool last_step_skipped() const noexcept

Whether the step just observed was thrown away.

std::uint64_t consecutive_good_steps() const noexcept

How many clean steps have run since the last overflow or growth.

std::uint64_t skipped_steps() const noexcept

How many steps have been thrown away in total.

std::uint64_t backoffs() const noexcept

How many times the scale has halved.

std::uint64_t growths() const noexcept

How many times the scale has doubled.

const LossScaleOptions & options() const noexcept

The options in force.