This repository provides the official implementation of IDAP++, a novel neural network compression approach that unifies both filter-level (width) and architecture-level (depth) pruning through information flow divergence analysis. The proposed method establishes a unified approach applicable to diverse neural architectures, including convolutional networks and transformer-based models.
We propose the first pruning methodology that systematically optimizes neural networks along both width (filter-level) and depth (layer-level) dimensions through a unified flow-divergence criterion. The framework combines:
- Divergence-Aware Filter Pruning (IDAP)
- Flow-Guided Layer Truncation
MAIN FILES WITH IMPLEMENTATION OF OUR METHOD: main.py, src/divergence_aware_pruning.py. Some variants of the pruned models can be found in the models folder.
- Python 3.10+
- PyTorch 2.0+
- CUDA-compatible GPU
- Other dependencies listed in
requirements.txt - Minimal setup to reproduce results: RTX 3060, Batch Size = 1
- Clone the repository:
git clone https://github.com/user852154/divergence_aware_pruning.git
cd divergence_aware_pruning- Create and activate a virtual environment:
python -m venv venv
source venv/bin/activate- Install dependencies:
pip install -r requirements.txt
- Pruning Results for Different Architectures Using IDAP++: Base vs. Pruned Models (Acc@1, GFlops, Δ%)
The table below presents the outcomes of our experiments, offering a comparative analysis of pruning across various model architectures and datasets. It reports top-1 accuracy (Acc@1) for both the original and pruned models, along with their computational cost measured in GFlops. The Δ% columns indicate the relative changes in accuracy and computational complexity resulting from pruning.
| Pruning Results for Different Architectures Using IDAP++: Base vs. Pruned Models (Acc@1, GFlops, Inference Time) | ||||||||||
| Architecture | Dataset | Acc@1 Base | Acc@1 Pruned | Δ% | GFlops Base | GFlops Pruned | Δ% | Inference Time Base | Inference Time Pruned | Speedup x |
| ResNet-50 | ImageNet | 76.13 | 74.62 | -1.99 | 4.1 | 1.5 | -63 | 8.9 | 4.5 | 2.0 |
| ResNet-50 | CIFAR-100 | 86.61 | 84.18 | -2.80 | 4.1 | 1.2 | -71 | 8.6 | 4.0 | 2.2 |
| ResNet-50 | CIFAR-10 | 98.20 | 95.98 | -2.26 | 4.1 | 1.1 | -72 | 8.8 | 4.5 | 2.0 |
| ResNet-50 | Stanford Cars | 92.52 | 90.14 | -2.57 | 4.1 | 1.2 | -70 | 8.8 | 4.4 | 2.0 |
| ResNet-50 | Flowers-102 | 97.91 | 96.75 | -1.19 | 4.1 | 1.5 | -64 | 8.7 | 4.3 | 2.0 |
| ResNet-50 | iNaturalist | 76.14 | 74.49 | -2.17 | 4.1 | 1.4 | -65 | 8.8 | 4.5 | 2.0 |
| ResNet-50 | Food101 | 90.45 | 88.58 | -2.07 | 4.1 | 1.3 | -67 | 8.6 | 3.9 | 2.2 |
| ResNet-50 | Oxford-IIIT Pet | 93.12 | 92.19 | -1.00 | 4.1 | 1.4 | -65 | 8.7 | 4.2 | 2.1 |
| ResNet-50 | Fashion MNIST | 93.18 | 91.79 | -1.49 | 4.1 | 0.8 | -80 | 8.8 | 4.0 | 2.2 |
| ResNet-50 | FER2013 | 71.80 | 69.52 | -3.18 | 4.1 | 1.3 | -67 | 8.7 | 4.2 | 2.1 |
| EfficientNet-B4 | ImageNet | 83.38 | 81.85 | -1.84 | 4.5 | 1.5 | -65 | 23.3 | 10.2 | 2.3 |
| EfficientNet-B4 | CIFAR-100 | 90.12 | 88.07 | -2.27 | 4.5 | 1.5 | -65 | 23.2 | 9.5 | 2.4 |
| EfficientNet-B4 | CIFAR-10 | 96.91 | 95.52 | -1.44 | 4.5 | 1.3 | -70 | 23.2 | 9.5 | 2.4 |
| EfficientNet-B4 | Stanford Cars | 91.34 | 89.06 | -2.50 | 4.5 | 1.4 | -68 | 24.0 | 12.6 | 1.9 |
| EfficientNet-B4 | Flowers-102 | 96.91 | 95.50 | -1.46 | 4.5 | 1.5 | -63 | 23.3 | 10.3 | 2.3 |
| EfficientNet-B4 | iNaturalist | 70.58 | 68.72 | -2.64 | 4.5 | 1.3 | -68 | 23.2 | 9.4 | 2.5 |
| EfficientNet-B4 | Food101 | 91.23 | 88.91 | -2.54 | 4.5 | 1.5 | -65 | 23.0 | 8.7 | 2.6 |
| EfficientNet-B4 | Oxford-IIIT Pet | 87.85 | 85.71 | -2.43 | 4.5 | 1.6 | -61 | 23.2 | 9.4 | 2.5 |
| EfficientNet-B4 | Fashion MNIST | 94.98 | 93.27 | -1.80 | 4.5 | 1.4 | -66 | 23.2 | 9.5 | 2.4 |
| EfficientNet-B4 | FER2013 | 74.17 | 72.23 | -2.61 | 4.5 | 1.4 | -68 | 23.2 | 9.2 | 2.5 |
| ViT-Base/16 | ImageNet | 81.07 | 79.49 | -1.95 | 17.5 | 6.3 | -64 | 33.4 | 18.9 | 1.8 |
| ViT-Base/16 | CIFAR-100 | 94.25 | 92.19 | -2.19 | 17.5 | 5.8 | -67 | 33.1 | 18.5 | 1.8 |
| ViT-Base/16 | CIFAR-10 | 98.61 | 96.99 | -1.64 | 17.5 | 4.3 | -75 | 33.1 | 18.8 | 1.8 |
| ViT-Base/16 | Stanford Cars | 93.74 | 91.05 | -2.87 | 17.5 | 5.1 | -71 | 33.3 | 18.7 | 1.8 |
| ViT-Base/16 | Flowers-102 | 95.53 | 94.56 | -1.01 | 17.5 | 5.5 | -68 | 33.4 | 19.0 | 1.8 |
| ViT-Base/16 | iNaturalist | 68.65 | 67.16 | -2.17 | 17.5 | 6.8 | -61 | 33.4 | 19.0 | 1.8 |
| ViT-Base/16 | Food101 | 87.41 | 85.00 | -2.76 | 17.5 | 6.5 | -63 | 33.4 | 18.7 | 1.8 |
| ViT-Base/16 | Oxford-IIIT Pet | 89.57 | 87.32 | -2.51 | 17.5 | 4.9 | -72 | 33.4 | 18.1 | 1.8 |
| ViT-Base/16 | Fashion MNIST | 92.83 | 90.81 | -2.18 | 17.5 | 6.5 | -63 | 33.2 | 17.0 | 2.0 |
| ViT-Base/16 | FER2013 | 70.21 | 67.95 | -3.23 | 17.5 | 6.0 | -66 | 33.2 | 18.7 | 1.8 |
| MobileNetV3-L | ImageNet | 74.04 | 72.05 | -2.68 | 0.2 | 0.1 | -67 | 9.1 | 3.9 | 2.3 |
| MobileNetV3-L | CIFAR-100 | 77.70 | 76.04 | -2.13 | 0.2 | 0.1 | -63 | 8.8 | 3.8 | 2.3 |
| MobileNetV3-L | CIFAR-10 | 89.81 | 88.56 | -1.40 | 0.2 | 0.1 | -68 | 8.9 | 3.9 | 2.3 |
| MobileNetV3-L | Stanford Cars | 83.87 | 82.37 | -1.79 | 0.2 | 0.1 | -66 | 9.3 | 4.1 | 2.3 |
| MobileNetV3-L | Flowers-102 | 90.02 | 88.68 | -1.48 | 0.2 | 0.0 | -64 | 9.2 | 4.3 | 2.1 |
| MobileNetV3-L | iNaturalist | 68.32 | 67.16 | -1.70 | 0.2 | 0.1 | -66 | 9.1 | 4.0 | 2.3 |
| MobileNetV3-L | Food101 | 87.42 | 85.59 | -2.09 | 0.2 | 0.1 | -72 | 9.2 | 4.4 | 2.1 |
| MobileNetV3-L | Oxford-IIIT Pet | 85.54 | 83.33 | -2.59 | 0.2 | 0.1 | -68 | 9.1 | 4.8 | 1.9 |
| MobileNetV3-L | Fashion MNIST | 92.74 | 90.60 | -2.31 | 0.2 | 0.1 | -73 | 9.0 | 3.9 | 2.3 |
| MobileNetV3-L | FER2013 | 69.87 | 67.79 | -2.98 | 0.2 | 0.1 | -63 | 9.3 | 4.2 | 2.2 |
| DenseNet-121 | ImageNet | 74.65 | 73.84 | -1.08 | 2.8 | 0.9 | -68 | 22.1 | 9.8 | 2.3 |
| DenseNet-121 | CIFAR-100 | 72.07 | 70.11 | -2.72 | 2.8 | 0.9 | -69 | 22.0 | 9.9 | 2.2 |
| DenseNet-121 | CIFAR-10 | 94.21 | 92.84 | -1.46 | 2.8 | 0.7 | -74 | 22.2 | 9.4 | 2.4 |
| DenseNet-121 | Stanford Cars | 83.14 | 81.06 | -2.50 | 2.8 | 0.9 | -70 | 22.2 | 8.8 | 2.5 |
| DenseNet-121 | Flowers-102 | 91.03 | 88.75 | -2.51 | 2.8 | 0.8 | -70 | 22.2 | 9.5 | 2.3 |
| DenseNet-121 | iNaturalist | 69.74 | 67.94 | -2.57 | 2.8 | 0.8 | -71 | 22.0 | 9.7 | 2.3 |
| DenseNet-121 | Food101 | 87.34 | 84.87 | -2.82 | 2.8 | 0.8 | -72 | 22.0 | 9.6 | 2.3 |
| DenseNet-121 | Oxford-IIIT Pet | 85.23 | 83.59 | -1.92 | 2.8 | 0.7 | -76 | 21.7 | 11.4 | 1.9 |
| DenseNet-121 | Fashion MNIST | 93.01 | 90.88 | -2.29 | 2.8 | 0.9 | -66 | 21.4 | 10.2 | 2.1 |
| DenseNet-121 | FER2013 | 65.13 | 63.13 | -3.07 | 2.8 | 0.8 | -71 | 21.7 | 9.6 | 2.3 |
| ConvNeXt-Small | ImageNet | 83.61 | 81.21 | -2.87 | 8.6 | 2.6 | -70 | 17.5 | 8.3 | 2.1 |
| ConvNeXt-Small | CIFAR-100 | 85.58 | 83.36 | -2.59 | 8.6 | 2.2 | -74 | 17.0 | 8.9 | 1.9 |
| ConvNeXt-Small | CIFAR-10 | 94.21 | 92.00 | -2.35 | 8.6 | 2.3 | -74 | 16.8 | 7.9 | 2.1 |
| ConvNeXt-Small | Stanford Cars | 82.19 | 80.77 | -1.72 | 8.6 | 2.8 | -68 | 17.2 | 6.9 | 2.5 |
| ConvNeXt-Small | Flowers-102 | 90.09 | 88.44 | -1.84 | 8.6 | 3.5 | -59 | 16.9 | 8.1 | 2.1 |
| ConvNeXt-Small | iNaturalist | 68.90 | 67.53 | -1.98 | 8.6 | 3.3 | -61 | 17.5 | 7.5 | 2.3 |
| ConvNeXt-Small | Food101 | 86.05 | 84.33 | -2.00 | 8.6 | 3.1 | -64 | 16.9 | 7.0 | 2.4 |
| ConvNeXt-Small | Oxford-IIIT Pet | 84.08 | 82.18 | -2.26 | 8.6 | 2.9 | -67 | 17.2 | 8.1 | 2.1 |
| ConvNeXt-Small | Fashion MNIST | 93.01 | 90.85 | -2.32 | 8.7 | 2.6 | -69 | 17.1 | 8.9 | 1.9 |
| ConvNeXt-Small | FER2013 | 76.10 | 74.05 | -2.70 | 8.6 | 2.7 | -68 | 17.4 | 7.4 | 2.3 |
| VGG19-BN | ImageNet | 74.22 | 72.64 | -2.13 | 19.7 | 6.8 | -65 | 13.9 | 5.4 | 2.6 |
| VGG19-BN | CIFAR-100 | 73.89 | 71.38 | -3.40 | 19.6 | 5.9 | -70 | 11.1 | 4.2 | 2.7 |
| VGG19-BN | CIFAR-10 | 93.45 | 91.89 | -1.67 | 19.6 | 4.8 | -76 | 11.1 | 3.9 | 2.8 |
| VGG19-BN | Stanford Cars | 88.12 | 86.54 | -1.80 | 19.6 | 6.2 | -68 | 13.9 | 5.6 | 2.5 |
| VGG19-BN | Flowers-102 | 92.34 | 90.99 | -1.46 | 19.6 | 5.5 | -72 | 13.9 | 5.6 | 2.5 |
| VGG19-BN | iNaturalist | 67.21 | 65.77 | -2.15 | 19.7 | 6.1 | -69 | 14.3 | 5.4 | 2.7 |
| VGG19-BN | Food101 | 85.67 | 83.39 | -2.66 | 19.6 | 5.8 | -70 | 14.0 | 5.2 | 2.7 |
| VGG19-BN | Oxford-IIIT Pet | 86.45 | 83.93 | -2.91 | 19.6 | 5.6 | -71 | 13.9 | 5.0 | 2.8 |
| VGG19-BN | Fashion MNIST | 91.78 | 89.48 | -2.51 | 19.6 | 5.5 | -72 | 11.0 | 4.2 | 2.6 |
| VGG19-BN | FER2013 | 68.34 | 66.68 | -2.43 | 19.6 | 6.8 | -65 | 11.0 | 4.3 | 2.5 |
| ShuffleNetV2 x2.0 | ImageNet | 76.23 | 74.40 | -2.40 | 0.6 | 0.2 | -63 | 9.1 | 4.5 | 2.0 |
| ShuffleNetV2 x2.0 | CIFAR-100 | 75.32 | 73.14 | -2.89 | 0.6 | 0.2 | -63 | 9.1 | 4.8 | 1.9 |
| ShuffleNetV2 x2.0 | CIFAR-10 | 90.45 | 88.66 | -1.98 | 0.6 | 0.1 | -83 | 9.0 | 4.7 | 1.9 |
| ShuffleNetV2 x2.0 | Stanford Cars | 82.56 | 80.45 | -2.56 | 0.6 | 0.2 | -61 | 9.1 | 4.1 | 2.2 |
| ShuffleNetV2 x2.0 | Flowers-102 | 89.12 | 87.78 | -1.50 | 0.6 | 0.2 | -63 | 9.0 | 4.2 | 2.1 |
| ShuffleNetV2 x2.0 | iNaturalist | 66.78 | 65.35 | -2.15 | 0.6 | 0.2 | -67 | 9.1 | 4.3 | 2.1 |
| ShuffleNetV2 x2.0 | Food101 | 84.23 | 82.30 | -2.29 | 0.6 | 0.2 | -64 | 9.1 | 4.8 | 1.9 |
| ShuffleNetV2 x2.0 | Oxford-IIIT Pet | 83.67 | 81.79 | -2.25 | 0.6 | 0.2 | -66 | 9.3 | 4.3 | 2.1 |
| ShuffleNetV2 x2.0 | Fashion MNIST | 90.89 | 89.08 | -2.00 | 0.6 | 0.1 | -83 | 9.0 | 4.8 | 1.9 |
| ShuffleNetV2 x2.0 | FER2013 | 67.45 | 65.55 | -2.82 | 0.6 | 0.2 | -64 | 9.4 | 4.8 | 2.0 |
We also present our results as plots:
- Comparative Accuracy of IDAP++ (Ours) and Prior Compression Techniques
The table below presents a comparison between our method and other pruning techniques. The results show that our approach achieves comparable weight reduction while preserving higher accuracy than alternative methods.
| Table: Comparative Accuracy of IDAP++ (Ours) and Prior Compression Techniques | ||||
| Architecture | Dataset | Method | Inference Time (ms) | Metric |
|---|---|---|---|---|
| Image Classification Task. Metric: Top-1 Acc (%) | ||||
| ResNet-50 | CIFAR-10 | Baseline | 8.8 | 98.2 |
| ResNet-50 | CIFAR-10 | SWD | 6.6 | 94.8 |
| ResNet-50 | CIFAR-10 | SFP | 5.9 | 93.2 |
| ResNet-50 | CIFAR-10 | OBC | 4.6 | 95.8 |
| ResNet-50 | CIFAR-10 | HAWQ-V3 (8-bit) | 4.5 | 95.2 |
| ResNet-50 | CIFAR-10 | IDAP++ (Ours) | 4.5 | 96.0 |
| EfficientNet-B4 | CIFAR-100 | Baseline | 23.2 | 90.1 |
| EfficientNet-B4 | CIFAR-100 | SWD | 12.8 | 86.7 |
| EfficientNet-B4 | CIFAR-100 | SFP | 12.6 | 86.6 |
| EfficientNet-B4 | CIFAR-100 | OBC | 10.0 | 87.9 |
| EfficientNet-B4 | CIFAR-100 | HAWQ-V3 (8-bit) | 11.6 | 87.9 |
| EfficientNet-B4 | CIFAR-100 | IDAP++ (Ours) | 9.5 | 88.1 |
| ViT-Base/16 | ImageNet | Baseline | 33.4 | 81.1 |
| ViT-Base/16 | ImageNet | SWD | 23.7 | 78.5 |
| ViT-Base/16 | ImageNet | SFP | 22.9 | 78.2 |
| ViT-Base/16 | ImageNet | OBC | 18.7 | 79.2 |
| ViT-Base/16 | ImageNet | HAWQ-V3 (8-bit) | 16.7 | 78.5 |
| ViT-Base/16 | ImageNet | IDAP++ (Ours) | 18.9 | 79.5 |
| MobileNetV3-L | Fashion MNIST | Baseline | 9.0 | 92.7 |
| MobileNetV3-L | Fashion MNIST | SWD | 5.2 | 89.8 |
| MobileNetV3-L | Fashion MNIST | SFP | 5.6 | 89.1 |
| MobileNetV3-L | Fashion MNIST | OBC | 4.1 | 90.0 |
| MobileNetV3-L | Fashion MNIST | HAWQ-V3 (8-bit) | 4.2 | 88.9 |
| MobileNetV3-L | Fashion MNIST | IDAP++ (Ours) | 3.9 | 90.6 |
| DenseNet-121 | Food101 | Baseline | 22.0 | 87.3 |
| DenseNet-121 | Food101 | SWD | 12.8 | 83.2 |
| DenseNet-121 | Food101 | SFP | 12.1 | 83.2 |
| DenseNet-121 | Food101 | OBC | 10.2 | 84.5 |
| DenseNet-121 | Food101 | HAWQ-V3 (8-bit) | 10.8 | 83.6 |
| DenseNet-121 | Food101 | IDAP++ (Ours) | 9.6 | 84.9 |
| ConvNeXt-Small | Flowers-102 | Baseline | 16.9 | 90.1 |
| ConvNeXt-Small | Flowers-102 | SWD | 10.5 | 87.0 |
| ConvNeXt-Small | Flowers-102 | SFP | 10.2 | 86.7 |
| ConvNeXt-Small | Flowers-102 | OBC | 8.4 | 87.9 |
| ConvNeXt-Small | Flowers-102 | HAWQ-V3 (8-bit) | 8.0 | 87.8 |
| ConvNeXt-Small | Flowers-102 | IDAP++ (Ours) | 8.1 | 88.4 |
| VGG19-BN | Stanford Cars | Baseline | 13.9 | 88.1 |
| VGG19-BN | Stanford Cars | SWD | 9.1 | 85.9 |
| VGG19-BN | Stanford Cars | SFP | 8.8 | 85.6 |
| VGG19-BN | Stanford Cars | OBC | 6.2 | 85.3 |
| VGG19-BN | Stanford Cars | HAWQ-V3 (8-bit) | 5.8 | 85.9 |
| VGG19-BN | Stanford Cars | IDAP++ (Ours) | 5.6 | 86.5 |
| ShuffleNetV2 x2.0 | iNaturalist | Baseline | 9.1 | 66.8 |
| ShuffleNetV2 x2.0 | iNaturalist | SWD | 6.7 | 64.4 |
| ShuffleNetV2 x2.0 | iNaturalist | SFP | 6.3 | 64.2 |
| ShuffleNetV2 x2.0 | iNaturalist | OBC | 5.1 | 65.2 |
| ShuffleNetV2 x2.0 | iNaturalist | HAWQ-V3 (8-bit) | 5.2 | 64.7 |
| ShuffleNetV2 x2.0 | iNaturalist | IDAP++ (Ours) | 4.3 | 65.4 |
| Image Generation Task. Metric: FID | ||||
| DCGAN | CIFAR-10 | Baseline | 102.3 | 24.1 |
| DCGAN | CIFAR-10 | SWD | 73.5 | 30.2 |
| DCGAN | CIFAR-10 | SFP | 67.1 | 35.6 |
| DCGAN | CIFAR-10 | OBC | 61.7 | 28.1 |
| DCGAN | CIFAR-10 | HAWQ-V3 (8-bit) | 56.3 | 25.2 |
| DCGAN | CIFAR-10 | IDAP++ (Ours) | 50.8 | 25.9 |
| VQGAN | COCO-Stuff | Baseline | 2003.7 | 18.5 |
| VQGAN | COCO-Stuff | SWD | 1412.6 | 25.1 |
| VQGAN | COCO-Stuff | SFP | 1287.4 | 27.7 |
| VQGAN | COCO-Stuff | OBC | 1189.5 | 22.2 |
| VQGAN | COCO-Stuff | HAWQ-V3 (8-bit) | 1115.3 | 23 |
| VQGAN | COCO-Stuff | IDAP++ (Ours) | 1004.2 | 20.1 |
| VQ-VAE | FFHQ | Baseline | 503.2 | 6.4 |
| VQ-VAE | FFHQ | SWD | 354.8 | 10.9 |
| VQ-VAE | FFHQ | SFP | 325.6 | 12.1 |
| VQ-VAE | FFHQ | OBC | 306.4 | 10.3 |
| VQ-VAE | FFHQ | HAWQ-V3 (8-bit) | 279.6 | 9.5 |
| VQ-VAE | FFHQ | IDAP++ (Ours) | 252.9 | 6.9 |
| Stable Diffusion v1.5 | MS-COCO | Baseline | 5032.8 | 12.3 |
| Stable Diffusion v1.5 | MS-COCO | SWD | 3518.6 | 18.2 |
| Stable Diffusion v1.5 | MS-COCO | SFP | 3274.9 | 20.7 |
| Stable Diffusion v1.5 | MS-COCO | OBC | 2988.1 | 14.5 |
| Stable Diffusion v1.5 | MS-COCO | HAWQ-V3 (8-bit) | 2836.7 | 16.3 |
| Stable Diffusion v1.5 | MS-COCO | IDAP++ (Ours) | 2486.3 | 13.5 |
- Model Compression Dynamics of ResNet-50 on CIFAR-10 Using the Two-Stage IDAP++ Framework
The tables below demonstrate the pruning dynamics of different models using our IDAP++ algorithm. The results show the gradual reduction in model parameters and computational complexity while maintaining high accuracy throughout most of the pruning process.
| ResNet-50, CIFAR-10 | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 4.1 | 98.20 | 99.86 |
| 2 | Filter Prune | 3.9 | 97.66 | 99.85 |
| 3 | Filter Prune | 3.7 | 97.23 | 99.84 |
| 4 | Filter Prune | 3.5 | 96.99 | 99.73 |
| 5 | Filter Prune | 3.3 | 97.11 | 99.89 |
| 6 | Filter Prune | 3.1 | 97.74 | 99.89 |
| 7 | Filter Prune | 2.9 | 97.62 | 99.84 |
| 8 | Filter Prune | 2.7 | 97.93 | 99.87 |
| 9 | Filter Prune | 2.6 | 98.09 | 99.76 |
| 10 | Filter Prune | 2.5 | 98.05 | 99.75 |
| 11 | Filter Prune | 2.4 | 97.87 | 99.77 |
| 12 | Filter Prune | 2.3 | 97.85 | 99.81 |
| 13 | Filter Prune | 2.2 | 97.84 | 99.77 |
| 14 | Filter Prune | 2.1 | 97.77 | 99.79 |
| 15 | Filter Prune | 2.0 | 97.70 | 99.76 |
| 16 | Filter Prune | 1.9 | 97.85 | 99.80 |
| 17 | Filter Prune | 1.9 | 97.56 | 99.81 |
| 18 | Filter Prune | 1.9 | 97.50 | 99.79 |
| 19 | Filter Prune | 1.8 | 97.42 | 99.80 |
| 20 | Filter Prune | 1.8 | 97.35 | 99.78 |
| 21 | Filter Prune | 1.7 | 97.28 | 99.75 |
| 22 | Filter Prune | 1.7 | 97.50 | 99.77 |
| 23 | Filter Prune | 1.5 | 97.52 | 99.78 |
| 24 | Filter Prune | 1.5 | 97.08 | 99.77 |
| 25 | Filter Prune | 1.4 | 97.50 | 99.80 |
| 26 | Filter Prune | 1.3 | 97.40 | 99.81 |
| 27 | Filter Prune | 1.3 | 96.91 | 99.79 |
| 28 | Filter Prune | 1.3 | 97.25 | 99.78 |
| 29 | Filter Prune | 1.2 | 97.52 | 99.80 |
| 30 | Filter Prune | 1.2 | 97.63 | 99.81 |
| 31 | Layer Trunc | 1.2 | 97.22 | 99.39 |
| 32 | Layer Trunc | 1.2 | 96.78 | 98.94 |
| 33 | Layer Trunc | 1.2 | 96.42 | 98.57 |
| 34 | Layer Trunc | 1.1 | 95.57 | 98.03 |
| 35 | Final Fine-Tune | 1.1 | 95.98 | 98.12 |
| EfficientNet-B4, ImageNet | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 4.2 | 83.38 | 96.70 |
| 2 | Filter Prune | 4.0 | 83.05 | 96.62 |
| 3 | Filter Prune | 3.8 | 82.94 | 96.63 |
| 4 | Filter Prune | 3.6 | 82.76 | 96.58 |
| 5 | Filter Prune | 3.4 | 82.69 | 96.57 |
| 6 | Filter Prune | 3.3 | 82.61 | 96.53 |
| 7 | Filter Prune | 3.1 | 82.53 | 96.51 |
| 8 | Filter Prune | 2.9 | 82.47 | 96.50 |
| 9 | Filter Prune | 2.8 | 82.41 | 96.49 |
| 10 | Filter Prune | 2.7 | 82.35 | 96.47 |
| 11 | Filter Prune | 2.6 | 82.27 | 96.45 |
| 12 | Filter Prune | 2.5 | 82.19 | 96.42 |
| 13 | Filter Prune | 2.4 | 82.11 | 96.41 |
| 14 | Filter Prune | 2.3 | 82.04 | 96.39 |
| 15 | Filter Prune | 2.2 | 81.96 | 96.38 |
| 16 | Filter Prune | 2.1 | 81.87 | 96.37 |
| 17 | Filter Prune | 2.0 | 81.77 | 96.34 |
| 18 | Filter Prune | 2.0 | 81.68 | 96.33 |
| 19 | Filter Prune | 1.9 | 81.61 | 96.31 |
| 20 | Filter Prune | 1.8 | 81.55 | 96.30 |
| 21 | Filter Prune | 1.8 | 81.48 | 96.29 |
| 22 | Filter Prune | 1.7 | 81.40 | 96.28 |
| 23 | Filter Prune | 1.7 | 81.34 | 96.26 |
| 24 | Filter Prune | 1.6 | 81.29 | 96.25 |
| 25 | Filter Prune | 1.6 | 81.22 | 96.24 |
| 26 | Filter Prune | 1.5 | 81.16 | 96.22 |
| 27 | Filter Prune | 1.5 | 81.11 | 96.20 |
| 28 | Filter Prune | 1.5 | 81.05 | 96.18 |
| 29 | Layer Trunc | 1.4 | 80.83 | 96.07 |
| 30 | Layer Trunc | 1.4 | 80.45 | 95.89 |
| 31 | Layer Trunc | 1.4 | 80.11 | 95.63 |
| 32 | Layer Trunc | 1.4 | 79.78 | 95.46 |
| 33 | Layer Trunc | 1.4 | 79.54 | 95.32 |
| 34 | Final Fine-Tune | 1.4 | 81.85 | 96.60 |
| ViT-Base/16, CIFAR-100 | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 17.5 | 94.25 | 99.12 |
| 2 | Filter Prune | 16.9 | 94.10 | 99.08 |
| 3 | Filter Prune | 16.5 | 94.02 | 99.07 |
| 4 | Filter Prune | 16.0 | 93.94 | 99.05 |
| 5 | Filter Prune | 15.6 | 93.86 | 99.03 |
| 6 | Filter Prune | 15.2 | 93.79 | 99.02 |
| 7 | Filter Prune | 14.8 | 93.70 | 99.00 |
| 8 | Filter Prune | 14.4 | 93.62 | 98.98 |
| 9 | Filter Prune | 14.0 | 93.53 | 98.96 |
| 10 | Filter Prune | 13.6 | 93.44 | 98.94 |
| 11 | Filter Prune | 13.3 | 93.35 | 98.92 |
| 12 | Filter Prune | 12.9 | 93.26 | 98.90 |
| 13 | Filter Prune | 12.6 | 93.18 | 98.88 |
| 14 | Filter Prune | 12.3 | 93.10 | 98.86 |
| 15 | Filter Prune | 12.0 | 93.02 | 98.84 |
| 16 | Filter Prune | 11.7 | 92.94 | 98.82 |
| 17 | Filter Prune | 11.4 | 92.86 | 98.80 |
| 18 | Filter Prune | 11.1 | 92.78 | 98.78 |
| 19 | Filter Prune | 10.8 | 92.70 | 98.76 |
| 20 | Filter Prune | 10.6 | 92.63 | 98.74 |
| 21 | Filter Prune | 10.3 | 92.56 | 98.72 |
| 22 | Filter Prune | 10.1 | 92.48 | 98.70 |
| 23 | Filter Prune | 9.8 | 92.41 | 98.68 |
| 24 | Filter Prune | 9.6 | 92.33 | 98.66 |
| 25 | Filter Prune | 9.4 | 92.26 | 98.64 |
| 26 | Filter Prune | 9.1 | 92.19 | 98.62 |
| 27 | Filter Prune | 8.9 | 92.12 | 98.60 |
| 28 | Filter Prune | 8.7 | 92.05 | 98.58 |
| 29 | Filter Prune | 8.5 | 91.98 | 98.56 |
| 30 | Filter Prune | 8.3 | 91.90 | 98.54 |
| 31 | Filter Prune | 8.2 | 91.82 | 98.50 |
| 32 | Filter Prune | 8.0 | 91.75 | 98.47 |
| 33 | Layer Trunc | 7.1 | 91.30 | 98.10 |
| 34 | Layer Trunc | 6.3 | 90.85 | 97.82 |
| 35 | Layer Trunc | 6.0 | 90.47 | 97.60 |
| 36 | Layer Trunc | 5.9 | 90.10 | 97.40 |
| 37 | Layer Trunc | 5.8 | 89.85 | 97.30 |
| 38 | Layer Trunc | 5.7 | 89.60 | 97.15 |
| 39 | Layer Trunc | 5.6 | 89.40 | 97.05 |
| 40 | Final Fine-Tune | 5.8 | 92.19 | 98.65 |
| MobileNetV3-L, Flowers-102 | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 0.2 | 90.02 | 98.45 |
| 2 | Filter Prune | 0.2 | 89.90 | 98.40 |
| 3 | Filter Prune | 0.2 | 89.85 | 98.38 |
| 4 | Filter Prune | 0.2 | 89.78 | 98.35 |
| 5 | Filter Prune | 0.2 | 89.70 | 98.33 |
| 6 | Filter Prune | 0.2 | 89.62 | 98.30 |
| 7 | Filter Prune | 0.2 | 89.55 | 98.28 |
| 8 | Filter Prune | 0.2 | 89.48 | 98.25 |
| 9 | Filter Prune | 0.2 | 89.42 | 98.23 |
| 10 | Filter Prune | 0.2 | 89.35 | 98.20 |
| 11 | Filter Prune | 0.2 | 89.28 | 98.18 |
| 12 | Filter Prune | 0.1 | 89.20 | 98.15 |
| 13 | Filter Prune | 0.1 | 89.12 | 98.12 |
| 14 | Filter Prune | 0.1 | 89.05 | 98.10 |
| 15 | Filter Prune | 0.1 | 88.98 | 98.08 |
| 16 | Filter Prune | 0.1 | 88.92 | 98.06 |
| 17 | Filter Prune | 0.1 | 88.85 | 98.03 |
| 18 | Filter Prune | 0.1 | 88.78 | 98.00 |
| 19 | Filter Prune | 0.1 | 88.72 | 97.98 |
| 20 | Filter Prune | 0.1 | 88.66 | 97.95 |
| 21 | Filter Prune | 0.1 | 88.59 | 97.93 |
| 22 | Filter Prune | 0.1 | 88.53 | 97.90 |
| 23 | Filter Prune | 0.1 | 88.47 | 97.88 |
| 24 | Filter Prune | 0.1 | 88.42 | 97.86 |
| 25 | Filter Prune | 0.1 | 88.36 | 97.83 |
| 26 | Filter Prune | 0.1 | 88.30 | 97.80 |
| 27 | Filter Prune | 0.1 | 88.24 | 97.78 |
| 28 | Filter Prune | 0.1 | 88.18 | 97.75 |
| 29 | Layer Trunc | 0.1 | 87.90 | 97.50 |
| 30 | Layer Trunc | 0.1 | 87.65 | 97.30 |
| 31 | Layer Trunc | 0.1 | 87.43 | 97.15 |
| 32 | Layer Trunc | 0.1 | 87.20 | 97.00 |
| 33 | Layer Trunc | 0.1 | 87.05 | 96.90 |
| 34 | Layer Trunc | 0.1 | 86.92 | 96.82 |
| 35 | Layer Trunc | 0.1 | 86.80 | 96.75 |
| 36 | Final Fine-Tune | 0.1 | 88.68 | 97.98 |
| DenseNet-121, iNaturalist | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 2.8 | 69.74 | 93.80 |
| 2 | Filter Prune | 2.7 | 69.62 | 93.75 |
| 3 | Filter Prune | 2.6 | 69.56 | 93.72 |
| 4 | Filter Prune | 2.5 | 69.49 | 93.70 |
| 5 | Filter Prune | 2.5 | 69.42 | 93.66 |
| 6 | Filter Prune | 2.4 | 69.35 | 93.60 |
| 7 | Filter Prune | 2.3 | 69.28 | 93.58 |
| 8 | Filter Prune | 2.3 | 69.20 | 93.54 |
| 9 | Filter Prune | 2.2 | 69.12 | 93.50 |
| 10 | Filter Prune | 2.1 | 69.05 | 93.48 |
| 11 | Filter Prune | 2.1 | 68.98 | 93.45 |
| 12 | Filter Prune | 2.0 | 68.92 | 93.40 |
| 13 | Filter Prune | 1.9 | 68.86 | 93.36 |
| 14 | Filter Prune | 1.9 | 68.80 | 93.34 |
| 15 | Filter Prune | 1.8 | 68.73 | 93.30 |
| 16 | Filter Prune | 1.8 | 68.66 | 93.27 |
| 17 | Filter Prune | 1.7 | 68.60 | 93.24 |
| 18 | Filter Prune | 1.7 | 68.54 | 93.20 |
| 19 | Filter Prune | 1.6 | 68.48 | 93.18 |
| 20 | Filter Prune | 1.6 | 68.42 | 93.15 |
| 21 | Filter Prune | 1.5 | 68.36 | 93.12 |
| 22 | Filter Prune | 1.5 | 68.30 | 93.10 |
| 23 | Filter Prune | 1.4 | 68.24 | 93.08 |
| 24 | Filter Prune | 1.4 | 68.18 | 93.05 |
| 25 | Filter Prune | 1.3 | 68.12 | 93.02 |
| 26 | Filter Prune | 1.3 | 68.06 | 92.99 |
| 27 | Filter Prune | 1.3 | 68.00 | 92.96 |
| 28 | Filter Prune | 1.3 | 67.95 | 92.93 |
| 29 | Filter Prune | 1.2 | 67.90 | 92.90 |
| 30 | Filter Prune | 1.2 | 67.85 | 92.88 |
| 31 | Filter Prune | 1.2 | 67.80 | 92.85 |
| 32 | Filter Prune | 1.1 | 67.75 | 92.82 |
| 33 | Filter Prune | 1.1 | 67.71 | 92.79 |
| 34 | Filter Prune | 1.1 | 67.67 | 92.76 |
| 35 | Filter Prune | 1.1 | 67.63 | 92.73 |
| 36 | Filter Prune | 1.0 | 67.59 | 92.70 |
| 37 | Filter Prune | 1.0 | 67.55 | 92.67 |
| 38 | Filter Prune | 1.0 | 67.51 | 92.65 |
| 39 | Filter Prune | 1.0 | 67.48 | 92.62 |
| 40 | Filter Prune | 1.0 | 67.45 | 92.60 |
| 41 | Filter Prune | 0.9 | 67.42 | 92.58 |
| 42 | Filter Prune | 0.9 | 67.39 | 92.55 |
| ConvNeXt-Small, Fashion MNIST | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 8.7 | 93.01 | 99.50 |
| 2 | Filter Prune | 8.4 | 92.90 | 99.47 |
| 3 | Filter Prune | 8.0 | 92.78 | 99.43 |
| 4 | Filter Prune | 7.7 | 92.67 | 99.40 |
| 5 | Filter Prune | 7.4 | 92.56 | 99.38 |
| 6 | Filter Prune | 7.1 | 92.44 | 99.35 |
| 7 | Filter Prune | 6.8 | 92.33 | 99.30 |
| 8 | Filter Prune | 6.4 | 92.21 | 99.27 |
| 9 | Filter Prune | 6.1 | 92.10 | 99.24 |
| 10 | Filter Prune | 5.8 | 91.99 | 99.20 |
| 11 | Filter Prune | 5.5 | 91.87 | 99.16 |
| 12 | Filter Prune | 5.2 | 91.76 | 99.12 |
| 13 | Filter Prune | 4.8 | 91.65 | 99.07 |
| 14 | Filter Prune | 4.5 | 91.53 | 99.03 |
| 15 | Filter Prune | 4.2 | 91.42 | 98.99 |
| 16 | Filter Prune | 3.9 | 91.30 | 98.94 |
| 17 | Filter Prune | 3.6 | 91.19 | 98.90 |
| 18 | Layer Trunc | 3.2 | 91.08 | 98.78 |
| 19 | Layer Trunc | 2.9 | 90.96 | 98.60 |
| 20 | Final Fine-Tune | 2.6 | 90.85 | 98.50 |
| VGG19-BN, ImageNet | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 19.6 | 75.20 | 92.50 |
| 2 | Filter Prune | 19.4 | 75.17 | 92.47 |
| 3 | Filter Prune | 19.2 | 75.13 | 92.44 |
| 4 | Filter Prune | 19.0 | 75.08 | 92.41 |
| 5 | Filter Prune | 18.8 | 75.04 | 92.37 |
| 6 | Filter Prune | 18.5 | 75.00 | 92.34 |
| 7 | Filter Prune | 18.3 | 74.95 | 92.31 |
| 8 | Filter Prune | 18.0 | 74.91 | 92.27 |
| 9 | Filter Prune | 17.8 | 74.86 | 92.24 |
| 10 | Filter Prune | 17.5 | 74.82 | 92.20 |
| 11 | Filter Prune | 17.2 | 74.77 | 92.16 |
| 12 | Filter Prune | 16.9 | 74.72 | 92.12 |
| 13 | Filter Prune | 16.6 | 74.67 | 92.08 |
| 14 | Filter Prune | 16.3 | 74.61 | 92.03 |
| 15 | Filter Prune | 15.9 | 74.56 | 91.99 |
| 16 | Filter Prune | 15.6 | 74.51 | 91.95 |
| 17 | Filter Prune | 15.3 | 74.45 | 91.90 |
| 18 | Filter Prune | 14.9 | 74.40 | 91.85 |
| 19 | Filter Prune | 14.6 | 74.34 | 91.81 |
| 20 | Filter Prune | 14.2 | 74.28 | 91.76 |
| 21 | Filter Prune | 13.9 | 74.22 | 91.71 |
| 22 | Filter Prune | 13.5 | 74.16 | 91.66 |
| 23 | Filter Prune | 13.2 | 74.10 | 91.61 |
| 24 | Filter Prune | 12.8 | 74.03 | 91.56 |
| 25 | Filter Prune | 12.4 | 73.97 | 91.51 |
| 26 | Filter Prune | 12.0 | 73.90 | 91.46 |
| 27 | Filter Prune | 11.7 | 73.83 | 91.40 |
| 28 | Filter Prune | 11.3 | 73.76 | 91.35 |
| 29 | Filter Prune | 10.9 | 73.69 | 91.29 |
| 30 | Filter Prune | 10.5 | 73.61 | 91.23 |
| 31 | Filter Prune | 10.1 | 73.54 | 91.17 |
| 32 | Filter Prune | 9.7 | 73.46 | 91.11 |
| 33 | Filter Prune | 9.3 | 73.38 | 91.05 |
| 34 | Filter Prune | 8.9 | 73.30 | 90.99 |
| 35 | Filter Prune | 8.5 | 73.21 | 90.93 |
| 36 | Filter Prune | 8.1 | 73.13 | 90.87 |
| 37 | Filter Prune | 7.7 | 73.04 | 90.80 |
| 38 | Filter Prune | 7.3 | 72.95 | 90.74 |
| 39 | Filter Prune | 6.9 | 72.86 | 90.67 |
| 40 | Filter Prune | 6.5 | 72.77 | 90.60 |
| 41 | Filter Prune | 6.0 | 72.67 | 90.53 |
| 42 | Filter Prune | 5.6 | 72.58 | 90.46 |
| 43 | Filter Prune | 5.2 | 72.48 | 90.38 |
| 44 | Filter Prune | 4.7 | 72.38 | 90.31 |
| 45 | Filter Prune | 4.3 | 72.33 | 90.25 |
| 46 | Filter Prune | 3.8 | 72.30 | 90.23 |
| 47 | Layer Trunc | 3.5 | 72.29 | 90.22 |
| 48 | Layer Trunc | 3.4 | 72.28 | 90.21 |
| 49 | Layer Trunc | 3.3 | 72.27 | 90.20 |
| 50 | Layer Trunc | 3.1 | 72.25 | 90.18 |
| 51 | Layer Trunc | 4.4 | 72.27 | 90.21 |
| 52 | Layer Trunc | 4.1 | 72.20 | 90.16 |
| 53 | Layer Trunc | 3.7 | 72.14 | 90.11 |
| 54 | Layer Trunc | 3.4 | 72.07 | 90.05 |
| 55 | Final Fine-Tune | 3.0 | 72.00 | 90.00 |
| ShuffleNetV2 x2.0, Stanford Cars | ||||
| Pruning Step | Stage | GFlops | Top-1 Acc. (%) | Top-5 Acc. (%) |
|---|---|---|---|---|
| 1 | Baseline | 0.6 | 82.56 | 97.50 |
| 2 | Filter Prune | 0.6 | 82.45 | 97.48 |
| 3 | Filter Prune | 0.6 | 82.33 | 97.45 |
| 4 | Filter Prune | 0.5 | 82.25 | 97.43 |
| 5 | Filter Prune | 0.5 | 82.13 | 97.40 |
| 6 | Filter Prune | 0.5 | 82.02 | 97.38 |
| 7 | Filter Prune | 0.5 | 81.92 | 97.36 |
| 8 | Filter Prune | 0.5 | 81.82 | 97.33 |
| 9 | Filter Prune | 0.4 | 81.70 | 97.30 |
| 10 | Filter Prune | 0.4 | 81.60 | 97.28 |
| 11 | Filter Prune | 0.4 | 81.49 | 97.25 |
| 12 | Filter Prune | 0.4 | 81.37 | 97.22 |
| 13 | Filter Prune | 0.4 | 81.27 | 97.19 |
| 14 | Filter Prune | 0.3 | 81.16 | 97.16 |
| 15 | Filter Prune | 0.3 | 81.04 | 97.13 |
| 16 | Filter Prune | 0.3 | 80.93 | 97.10 |
| 17 | Filter Prune | 0.3 | 80.84 | 97.07 |
| 18 | Filter Prune | 0.3 | 80.76 | 97.05 |
| 19 | Filter Prune | 0.3 | 80.68 | 97.02 |
| 20 | Filter Prune | 0.3 | 80.59 | 96.99 |
| 21 | Filter Prune | 0.2 | 80.51 | 96.96 |
| 22 | Filter Prune | 0.2 | 80.45 | 96.94 |
| 23 | Filter Prune | 0.2 | 80.40 | 96.92 |
| 24 | Filter Prune | 0.2 | 80.36 | 96.90 |
| 25 | Filter Prune | 0.2 | 80.33 | 96.88 |
| 26 | Filter Prune | 0.2 | 80.30 | 96.85 |
| 27 | Filter Prune | 0.2 | 80.28 | 96.83 |
| 28 | Filter Prune | 0.2 | 80.25 | 96.80 |
| 29 | Filter Prune | 0.2 | 80.22 | 96.78 |
| 30 | Filter Prune | 0.2 | 80.18 | 96.75 |
| 31 | Filter Prune | 0.1 | 80.15 | 96.73 |
| 32 | Filter Prune | 0.1 | 80.12 | 96.70 |
| 33 | Filter Prune | 0.1 | 80.08 | 96.67 |
| 34 | Layer Trunc | 0.1 | 80.05 | 96.50 |
| 35 | Layer Trunc | 0.1 | 80.03 | 96.20 |
| 36 | Layer Trunc | 0.1 | 80.00 | 95.95 |
| 37 | Layer Trunc | 0.1 | 80.00 | 95.60 |
| 38 | Final Fine-Tune | 0.1 | 80.45 | 95.95 |
| Performance of IDAP++ combined with quantization and distillation methods across diverse architectures. Baseline values correspond to unmodified FP32 models. | |||||
| Model, Dataset | Method | Compression Type | GFlops | Top-1 Acc (%) | Inference Time (ms) |
|---|---|---|---|---|---|
| ResNet-50, CIFAR-10 | Baseline | None (FP32) | 4.11 | 98.20 | 8.9 |
| ResNet-50, CIFAR-10 | HAWQ-V3 | Quantization (8-bit) | 4.11 | 95.22 | 4.5 |
| ResNet-50, CIFAR-10 | HAQ | Quantization (4-bit) | 4.11 | 94.87 | 4.2 |
| ResNet-50, CIFAR-10 | IDAP++ | Pruning | 1.10 | 95.98 | 4.5 |
| ResNet-50, CIFAR-10 | IDAP++ & HAWQ-V3 | Pruning + Quant (8-bit) | 1.10 | 95.13 | 2.3 |
| ResNet-50, CIFAR-10 | IDAP++ & HAQ | Pruning + Quant (4-bit) | 1.10 | 95.01 | 2.0 |
| ResNet-50, CIFAR-10 | IDAP++ & CRD | Pruning + Distillation | 1.10 | 96.24 | 4.5 |
| ResNet-50, CIFAR-10 | IDAP++ & VanillaKD | Pruning + Distillation | 1.10 | 96.18 | 4.5 |
| ViT-Base/16, ImageNet | Baseline | None (FP32) | 17.50 | 81.07 | 33.4 |
| ViT-Base/16, ImageNet | HAWQ-V3 | Quantization (8-bit) | 17.50 | 78.54 | 16.7 |
| ViT-Base/16, ImageNet | HAQ | Quantization (4-bit) | 17.50 | 78.05 | 15.9 |
| ViT-Base/16, ImageNet | IDAP++ | Pruning | 6.30 | 79.49 | 18.9 |
| ViT-Base/16, ImageNet | IDAP++ & HAWQ-V3 | Pruning + Quant (8-bit) | 6.30 | 78.32 | 9.5 |
| ViT-Base/16, ImageNet | IDAP++ & HAQ | Pruning + Quant (4-bit) | 6.30 | 77.97 | 9.3 |
| ViT-Base/16, ImageNet | IDAP++ & CRD | Pruning + Distillation | 6.30 | 80.60 | 18.9 |
| ViT-Base/16, ImageNet | IDAP++ & VanillaKD | Pruning + Distillation | 6.30 | 80.48 | 18.9 |
| EfficientNet-B4, CIFAR-100 | Baseline | None (FP32) | 4.49 | 90.12 | 23.2 |
| EfficientNet-B4, CIFAR-100 | HAWQ-V3 | Quantization (8-bit) | 4.49 | 87.92 | 11.6 |
| EfficientNet-B4, CIFAR-100 | HAQ | Quantization (4-bit) | 4.49 | 87.69 | 10.8 |
| EfficientNet-B4, CIFAR-100 | IDAP++ | Pruning | 1.50 | 88.07 | 9.5 |
| EfficientNet-B4, CIFAR-100 | IDAP++ & HAWQ-V3 | Pruning + Quant (8-bit) | 1.50 | 87.88 | 5.0 |
| EfficientNet-B4, CIFAR-100 | IDAP++ & HAQ | Pruning + Quant (4-bit) | 1.50 | 86.51 | 4.8 |
| EfficientNet-B4, CIFAR-100 | IDAP++ & CRD | Pruning + Distillation | 1.50 | 89.68 | 9.5 |
| EfficientNet-B4, CIFAR-100 | IDAP++ & VanillaKD | Pruning + Distillation | 1.50 | 89.32 | 9.5 |
The figures below illustrate the training dynamics of ResNet-50 on the CIFAR-10 dataset, showing how various metrics evolve during the pruning process. The plots demonstrate the changes in computational complexity (GFLOPs), parameter count, and Top-1 accuracy across pruning steps, providing a comprehensive view of the model's behavior during optimization.
To reproduce the results reported in our paper:
- Follow the installation instructions above
- Download the preprocessed datasets using the provided scripts
- Run the training and evaluation scripts
- Use plot_training_metrics.py script to generate training dynamics plots and metrics visualization
We would like to express our gratitude to the following sources for providing pre-trained models that were used in this research:
- The authors of "ResNet strikes back: An improved training procedure in timm" (Wightman et al., 2021) for their foundational work on ResNet architectures;
- The authors of "Which backbone to use: A resource-efficient domain specific comparison for computer vision" (Jeevan & Sethi, 2024) for their contributions to efficient model architectures;
- The authors of "DepGraph: Towards any structural pruning" (Fang et al., 2023) for their codebase for the structural pruning;
- The PyTorch Vision team for their comprehensive model zoo (https://docs.pytorch.org/vision/0.19/models).
This project is licensed under the MIT License - see the LICENSE file for details.