What are Convolution-BatchNorm (ConvBN) blocks?
Convolution-BatchNorm (ConvBN) blocks are essential for various computer vision tasks and other domains. They operate in three modes: Train, Eval, and Deploy.
The Trade-off between Stability and Efficiency in ConvBN Blocks
The Deploy mode is efficient but suffers from training instability, while the Eval mode is widely used in transfer learning but lacks efficiency. To solve this dilemma, a new Tune mode has been proposed to bridge the gap between Eval mode and Deploy mode.
The Benefits of the Proposed Tune Mode
The Tune mode is as stable as the Eval mode for transfer learning, and its computational efficiency closely matches that of the Deploy mode. Extensive experiments in object detection, classification, and adversarial example generation have shown that the proposed Tune mode retains performance while significantly reducing GPU memory footprint and training time, thereby contributing efficient ConvBN blocks for transfer learning and beyond. The method has been integrated into both PyTorch and MMCV/MMEngine, making it easily accessible for practitioners.