HyCache: Hybrid Caching for Accelerating Input Preprocessing Pipelines in DNN training
Talk, Sheraton Boston, MA, USA,
End-to-end deep neural networks’ (DNNs) training performance depends not only on the time spent in training the model weights but also on the time spent in loading and preprocessing the training data. Recent advances in GPU hardware have made training substantially faster. As a result, the bottleneck has shifted to the CPU-based input pipeline. This pipeline must fetch and transform each sample through multiple stages before it can be consumed by the GPU.