Contents

struct

SushiAI::Optim::AdamWOptions

AdamW's hyperparameters, with the reference defaults.

Declared in
include/SushiAI/optim/adamw.hpp

Public attributes

double learning_rate = 1e-3

Step size.

double beta1 = 0.9

First-moment decay.

double beta2 = 0.999

Second-moment decay.

double eps = 1e-8

Denominator floor, added after the bias correction.

double weight_decay = 0.01

Decoupled weight decay, applied multiplicatively before the step.

UpdateKernel kernel = UpdateKernel::FUSED

One fused kernel per parameter, or the twelve separate ops.

FUSED is the default and the shipping path; SEPARATE exists to be compared against. See optimizer.hpp.