hive.utils.torch_utils module
- hive.utils.torch_utils.numpify(t)[source]
Convert object to a numpy array.
- Parameters
t (np.ndarray | torch.Tensor | obj) – Converts object to
np.ndarray
.
- class hive.utils.torch_utils.RMSpropTF(params, lr=0.01, alpha=0.9, eps=1e-10, weight_decay=0, momentum=0.0, centered=False, decoupled_decay=False, lr_in_momentum=True)[source]
Bases:
Optimizer
Direct cut-paste from rwhightman/pytorch-image-models. https://github.com/rwightman/pytorch-image-models/blob/f7d210d759beb00a3d0834a3ce2d93f6e17f3d38/timm/optim/rmsprop_tf.py Licensed under Apache 2.0, https://github.com/rwightman/pytorch-image-models/blob/master/LICENSE
Implements RMSprop algorithm (TensorFlow style epsilon)
NOTE: This is a direct cut-and-paste of PyTorch RMSprop with eps applied before sqrt and a few other modifications to closer match Tensorflow for matching hyper-params. Noteworthy changes include:
Epsilon applied inside square-root
square_avg initialized to ones
LR scaling of update accumulated in momentum buffer
Proposed by G. Hinton in his course. The centered version first appears in Generating Sequences With Recurrent Neural Networks.
- Parameters
params (iterable) – iterable of parameters to optimize or dicts defining parameter groups
lr (float, optional) – learning rate (default: 1e-2)
momentum (float, optional) – momentum factor (default: 0)
alpha (float, optional) – smoothing (decay) constant (default: 0.9)
eps (float, optional) – term added to the denominator to improve numerical stability (default: 1e-10)
centered (bool, optional) – if
True
, compute the centered RMSProp, the gradient is normalized by an estimation of its varianceweight_decay (float, optional) – weight decay (L2 penalty) (default: 0)
decoupled_decay (bool, optional) – decoupled weight decay as per https://arxiv.org/abs/1711.05101
lr_in_momentum (bool, optional) – learning rate scaling is included in the momentum buffer update as per defaults in Tensorflow