Models

Saving and loading pytorch models can be an error prone process especially when you train on multiple GPUs. TorchFusion Utils provides you easy to use functions to save and load models in a way that frees you from running into any issues.

Saving Models

from torchfusion_utils.models import *
from torchvision.models import resnet18
model = resnet18(num_classes=100)
#save weights only
save_model(model,"model.pth")
#save weights and architecture
save_model(mode,"fullmodel.pth",save_architecture=True)

The save_model function takes care of properly converting your model before saving, for example, if your model is wrapped in nn.DataParallel, it extracts your model and converts to CPU before it saves. This ensures you can easily load your model anywhere without errors.

If save_architecture is set to True , the entire model is serialized to disk. This ensures you never have to redefine your model in order to load it again.

Loading models

from torchfusion_utils.models import *
from torchvision.models import resnet18
model = resnet18(num_classes=100)
#load pretrained weights
load_model(model,"saved_model.pth")

Model Summary

Often, it is useful to understand the structure, number of parameters, number of layers and the computational cost of your deep learning models. TorchFusion Utils provides a simple function to analyse your model.

from torchfusion_utils.models import *
from torchvision.models import resnet18
import torch
model = resnet18(num_classes=1000)
dummy_input = torch.FloatTensor(1,3,224,224)
#obtain model summary
summary = model_summary(model,dummy_input)
print(summary)

Resnet18 Summary

Model Summary
Name Input Size Output Size Parameters Multiply Adds (Flops)
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_1 [1, 3, 224, 224] [1, 64, 112, 112] 9408 118013952
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_1 [1, 64, 112, 112] [1, 64, 112, 112] 128 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_1 [1, 64, 112, 112] [1, 64, 112, 112] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
MaxPool2d_1 [1, 64, 112, 112] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_2 [1, 64, 56, 56] [1, 64, 56, 56] 36864 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_2 [1, 64, 56, 56] [1, 64, 56, 56] 128 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_2 [1, 64, 56, 56] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_3 [1, 64, 56, 56] [1, 64, 56, 56] 36864 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_3 [1, 64, 56, 56] [1, 64, 56, 56] 128 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_3 [1, 64, 56, 56] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_1 [1, 64, 56, 56] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_4 [1, 64, 56, 56] [1, 64, 56, 56] 36864 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_4 [1, 64, 56, 56] [1, 64, 56, 56] 128 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_4 [1, 64, 56, 56] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_5 [1, 64, 56, 56] [1, 64, 56, 56] 36864 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_5 [1, 64, 56, 56] [1, 64, 56, 56] 128 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_5 [1, 64, 56, 56] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_2 [1, 64, 56, 56] [1, 64, 56, 56] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_6 [1, 64, 56, 56] [1, 128, 28, 28] 73728 57802752
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_6 [1, 128, 28, 28] [1, 128, 28, 28] 256 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_6 [1, 128, 28, 28] [1, 128, 28, 28] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_7 [1, 128, 28, 28] [1, 128, 28, 28] 147456 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_7 [1, 128, 28, 28] [1, 128, 28, 28] 256 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_8 [1, 64, 56, 56] [1, 128, 28, 28] 8192 6422528
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_8 [1, 128, 28, 28] [1, 128, 28, 28] 256 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_7 [1, 128, 28, 28] [1, 128, 28, 28] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_3 [1, 64, 56, 56] [1, 128, 28, 28] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_9 [1, 128, 28, 28] [1, 128, 28, 28] 147456 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_9 [1, 128, 28, 28] [1, 128, 28, 28] 256 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_8 [1, 128, 28, 28] [1, 128, 28, 28] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_10 [1, 128, 28, 28] [1, 128, 28, 28] 147456 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_10 [1, 128, 28, 28] [1, 128, 28, 28] 256 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_9 [1, 128, 28, 28] [1, 128, 28, 28] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_4 [1, 128, 28, 28] [1, 128, 28, 28] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_11 [1, 128, 28, 28] [1, 256, 14, 14] 294912 57802752
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_11 [1, 256, 14, 14] [1, 256, 14, 14] 512 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_10 [1, 256, 14, 14] [1, 256, 14, 14] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_12 [1, 256, 14, 14] [1, 256, 14, 14] 589824 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_12 [1, 256, 14, 14] [1, 256, 14, 14] 512 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_13 [1, 128, 28, 28] [1, 256, 14, 14] 32768 6422528
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_13 [1, 256, 14, 14] [1, 256, 14, 14] 512 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_11 [1, 256, 14, 14] [1, 256, 14, 14] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_5 [1, 128, 28, 28] [1, 256, 14, 14] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_14 [1, 256, 14, 14] [1, 256, 14, 14] 589824 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_14 [1, 256, 14, 14] [1, 256, 14, 14] 512 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_12 [1, 256, 14, 14] [1, 256, 14, 14] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_15 [1, 256, 14, 14] [1, 256, 14, 14] 589824 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_15 [1, 256, 14, 14] [1, 256, 14, 14] 512 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_13 [1, 256, 14, 14] [1, 256, 14, 14] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_6 [1, 256, 14, 14] [1, 256, 14, 14] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_16 [1, 256, 14, 14] [1, 512, 7, 7] 1179648 57802752
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_16 [1, 512, 7, 7] [1, 512, 7, 7] 1024 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_14 [1, 512, 7, 7] [1, 512, 7, 7] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_17 [1, 512, 7, 7] [1, 512, 7, 7] 2359296 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_17 [1, 512, 7, 7] [1, 512, 7, 7] 1024 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_18 [1, 256, 14, 14] [1, 512, 7, 7] 131072 6422528
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_18 [1, 512, 7, 7] [1, 512, 7, 7] 1024 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_15 [1, 512, 7, 7] [1, 512, 7, 7] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_7 [1, 256, 14, 14] [1, 512, 7, 7] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_19 [1, 512, 7, 7] [1, 512, 7, 7] 2359296 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_19 [1, 512, 7, 7] [1, 512, 7, 7] 1024 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_16 [1, 512, 7, 7] [1, 512, 7, 7] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Conv2d_20 [1, 512, 7, 7] [1, 512, 7, 7] 2359296 115605504
----------------------------------------------------------------------------------------------------------------------------------
BatchNorm2d_20 [1, 512, 7, 7] [1, 512, 7, 7] 1024 Not Available
----------------------------------------------------------------------------------------------------------------------------------
ReLU_17 [1, 512, 7, 7] [1, 512, 7, 7] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
BasicBlock_8 [1, 512, 7, 7] [1, 512, 7, 7] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
AdaptiveAvgPool2d_1 [1, 512, 7, 7] [1, 512, 1, 1] 0 Not Available
----------------------------------------------------------------------------------------------------------------------------------
Linear_1 [1, 512] [1, 1000] 513000 512000
----------------------------------------------------------------------------------------------------------------------------------
Total Parameters: 11689512
----------------------------------------------------------------------------------------------------------------------------------
Total Multiply Adds (For Convolution and Linear Layers only): 1814073344
----------------------------------------------------------------------------------------------------------------------------------
Number of Layers
Linear : 1 layers ReLU : 17 layers AdaptiveAvgPool2d : 1 layers Conv2d : 20 layers BasicBlock : 8 layers BatchNorm2d : 20 layers MaxPool2d : 1 layers