0534541 20240103224732.520201117235959.9 85093082913 10.1007/978-3-030-58526-6_31 18 s. P cav_un_epca*0534540978-3-030-58525-90302-9743LNCS522-539ChamSpringer Nature Switzerland AG 20202020VedaldiAndreaBischofHorstBroxThomasFrahmJan-Michael Convolutional neural network acceleration Low-rank tensor decomposition Degeneracy correction cav_un_auth*0382249 Phan A. H. RU cav_un_auth*0399439 Sobolev K. RU cav_un_auth*0399440 Sozykin K. RU cav_un_auth*0399441 Ermilov D. RU cav_un_auth*0399442 Gusak J. RU cav_un_auth*0101212 Tichavský Petr UTIA-B Stochastická informatika Department of Stochastic Informatics SI SI Department of Stochastic Informatics 10 Ústav teorie informace a automatizace AV ČR, v. v. i. cav_un_auth*0399443 Glukhov V. CN cav_un_auth*0399444 Oseledets I. RU cav_un_auth*0382250 Cichocki A. RU http://library.utia.cas.cz/separaty/2020/SI/tichavsky-0534541.pdf Most state-of-the-art deep neural networks are overparameterized and exhibit a high computational cost. A straightforward approach to this problem is to replace convolutional kernels with its low-rank tensor approximations, whereas the Canonical Polyadic tensor Decomposition is one of the most suited models. However, fitting the convolutional tensors by numerical optimization algorithms often encounters diverging components, i.e.,extremely large rank-one tensors but canceling each other. Such degeneracy often causes the non-interpretable result and numerical instability for the neural network ne-tuning. This paper is the first study on degeneracy in the tensor decomposition of convolutional kernels. We present a novel method, which can stabilize the low-rank approximation of convolutional kernels and ensure efficient compression while preserving the high quality performance of the neural networks. We evaluate our approach on popular CNN architectures for image classification and show that our method results in much lower accuracy degradation and provides consistent performance. cav_un_auth*0399445 European Conference on Computer Vision 2020 /16./ 20200823 20200828 Glasgow GB JD 20000 20200 20201 2021 9 PR RVO:67985556 http://hdl.handle.net/11104/0313191 S 2020 85093082913 SCOPUS PUBMED WOS cav_un_epca*0534540 ECCV 2020 978-3-030-58525-9 0302-9743 1611-3349 522 539 Cham Springer Nature Switzerland AG 2020 2020 Lecture Notes in Computer Science 12374 LNCS 340 Vedaldi Andrea 340 Bischof Horst 340 Brox Thomas 340 Frahm Jan-Michael