โ‰ˆ the.bay.news

Weight Decay 0.05: Catastrophe in Adam, Optimum in AdamW

DEV Community
Weight Decay 0.05: Catastrophe in Adam, Optimum in AdamW
The dose-response series reaches the knob I've been inheriting since September: weight_decay=5e-4, copied from SGD folklore into Adam

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.