Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Last update: Nov 03, 2022

Related tags

Overview

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

This repository is the official implementation of paper [Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples].

Requirements

Python 3.6
PyTorch 1.8.1
Refer requirements.txt for other requirements

To install requirements:

pip install -r requirements.txt

Training

For Imagenet training, change the path of the validation set in .hocon file. To train the model described in the paper, run one of this command:

./run_cifar10_4bit.sh
./run_cifar100_4bit.sh
./run_imgnet_resnet18_4bit.sh
./run_imgnet_resnet50_4bit.sh
./run_imgnet_mobilenet_v2_4bit.sh

Refer other_train_scripts folder for 5bit settings.

Evaluation

To evaluate trained model, run the command below after training:

./eval_cifar10_4bit.sh
./eval_cifar100_4bit.sh
./eval_imgnet_resnet18_4bit.sh
./eval_imgnet_resnet50_4bit.sh
./eval_imgnet_mobilenet_v2_4bit.sh

Visualizing Feature Space

Feature space visualization of real or synthetic images described in Figure 3.

python experiments.py --pca_source
python experiments.py --gdfq_generator_path GENERATOR_WEIGHT_PATH --pca_gdfq --image_gdfq
python experiments.py --qimera_generator_path GENERATOR_WEIGHT_PATH --pca_qimera --pca_mix --pca_path --image_gdfq --image_mix

Results

Our model achieves the following performance on :

Dataset	Model	Teacher Net Accuracy	4bit Quantized Model Accuracy	5bit Quantized Model Accuracy
Cifar-10	ResNet-20	93.89%	91.26%	93.46%
Cifar-100	ResNet-20	70.33%	65.10%	69.02%
ImageNet	ResNet-18	71.47%	63.84%	69.29%
ImageNet	ResNet-50	77.73%	66.25%	75.32%
ImageNet	MobileNetV2	73.03%	61.62%	70.45%

Generated Synthetic Images for Cifar10 :

License

This project is licensed under the terms of the GNU General Public License v3.0

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Related tags

Overview

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Requirements

Training

Evaluation

Visualizing Feature Space

Results

License

Owner

Kanghyun Choi

Code for CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.

Code for "Modeling Indirect Illumination for Inverse Rendering", CVPR 2022

Code for the ECIR'22 paper "Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators"

TextureGAN in Pytorch

Adaout is a practical and flexible regularization method with high generalization and interpretability

A semantic segmentation toolbox based on PyTorch

A robust pointcloud registration pipeline based on correlation.

Learning Saliency Propagation for Semi-supervised Instance Segmentation

Implementation of "RaScaNet: Learning Tiny Models by Raster-Scanning Image" from CVPR 2021.

A boosting-based Multiple Instance Learning (MIL) package that includes MIL-Boost and MCIL-Boost

9th place solution

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers

Code to reproduce the results for Compositional Attention

A basic duplicate image detection service using perceptual image hash functions and nearest neighbor search, implemented using faiss, fastapi, and imagehash

Informal Persian Universal Dependency Treebank

A Parameter-free Deep Embedded Clustering Method for Single-cell RNA-seq Data

An end-to-end framework for mixed-integer optimization with data-driven learned constraints.

code for our ECCV-2020 paper: Self-supervised Video Representation Learning by Pace Prediction

Python inverse kinematics for your robot model based on Pinocchio.