How to train model on large batches when GPU can’t hold more than a few samples

Tech It Yourself September 22, 2019
How can you train model on large batches when GPU can’t hold more than a few samples?
There are some solutions:
  - Gradient Accumulation
  - Gradient Checkpointing
  - Distributed training: training on several machines

Post a Comment

0 Comments

Emoji
(y)
:)
:(
hihi
:-)
:D
=D
:-d
;(
;-(
@-)
:P
:o
:>)
(o)
:p
(p)
:-s
(m)
8-)
:-t
:-b
b-(
:-#
=p~
x-)
(k)