一种基于Lasso回归与SVD融合的深度学习模型压缩方法

针对深度学习模型所需的海量参数及强大的计算资源而导致其不能很便捷地应用于嵌入式设备或移动端的问题,在Lasso（Least Absolute Shrinkage and Selection Operator）回归通道挑选法的基础上,提出了Lasso+奇异值分解（Singular Value Decomposition,SVD）的融合压缩法。使用VGG-16为初始模型,分别在不同的小型数据集上进行迁移学习,使用迁移学习后的模型在不同的加速率下进行测试。实验结果表明,相对于传统的模型压缩算法,Lasso+SVD的融合压缩法实现了在加速和参数压缩两方面的优势,进而以目标检测为应用方向,在保证准确率的同时不仅降低了模型存储需求,而且也较大提升了模型的实时性。

A deep learning model compression method based on Lasso regression and SVD fusion

The massive parameters and powerful computing resources required by the deep learning model make it not be easily applied to embedded devices or mobile terminals.Based on the least absolute shrinkage and selection operator(Lasso) regression channel selection method,a fusion compression method of Lasso+SVD(singular value decomposition) is proposed.Using VGG-16 as the initial model,the migration learning is performed on different small data sets,and the model after migration learning is tested at different acceleration rates.The experimental results show that compared with the traditional model compression algorithm,the fusion compression method of Lasso+SVD realizes the advantages of both acceleration and parameter compression,and then uses the target detection as the application direction to ensure the accuracy,while not only reducing the model storage requirements but also greatly improving the real-time performance of the model.