Deep Neural Networks: Google releases MorphNet as open source

Deep Neural Networks: Google releases MorphNet as open source
Deep Neural Networks: Google releases MorphNet as open source

Deep neural networks (DNNs) have become indispensable as tools for the processing of image classification, text recognition or language transcription tasks. In practice, however, the models used must always be adapted to the specific challenge in order to deliver the best possible results. Google introduced a new approach under the name of MorphNet last year, which allows the structure of an existing DNN to be developed and optimized automatically, step by step. Now, Google releases MorphNet’s TensorFlow implementation as open source for the community.

MorphNet supports model optimization among others in terms of scalability and portability. In the course of an iterative process of shrinking and expansion phases, not only can the training costs be reduced, but the models can also be used again for other tasks and re-trained. One of the core functions of MorphNet, however, is the targeted regularization, in which relevant parameters such as FLOPS by inference or the model size are optimized for the intended task. The following graphic illustrates the procedure by way of example using the ResNet 101 model, which was trained with the JFT data set.

MorphNet Targeted Regularization: The rectangle width is proportional to the number of channels in the layer. The purple bar at the bottom is the input layer. Left: Basic network used as input to MorphNet. Middle: output with FLOP regulator. Right: Output with resizing. (Image: Google)

In topology morphing, MorphNet may remove whole branches from a network. This occurs, for example, when optimizing the neurons per layer, the algorithm should reduce the number of neurons to zero. In the case of ResNet models, MorphNet then removes residual connections; in Inception models, complete parallel strands can be affected.

Depending on the number of optimization cycles that have been taken, MorphNet models can either be improved in terms of FLOP-related costs or trimmed to higher accuracy. Google itself uses MorphNet primarily for image classification in order to be able to downsize DNNs while maintaining the same quality or operate at lower FLOP costs. More details about MorphNet can be found in the blog post , the technical background is described in an article ( PDF download ). If you want to try MorphNet, download the TensorFlow implementation from the GitHub project site.

SHARE
Previous articleNintendo has apparently stopped Super Mario Bros for C64
Next articleRemaking the bed as soon as you get up can damage your health because
Amit Kumar
Amit Kumar is editor-in-chief and founder of Revyuh Media. He has been ensuring journalistic quality and shaping the future of Revyuh.com - in terms of content, text, personnel and strategy. He also develops herself further, likes to learn new things and, as a trained mediator, considers communication and freedom to be essential in editorial cooperation. After studying and training at the Indian Institute of Journalism & Mass Communication He accompanied an ambitious Internet portal into the Afterlife and was editor of the Scroll Lib Foundation. After that He did public relations for the MNC's in India. Email: amit.kumar (at) revyuh (dot) com ICE : 00 91 (0) 99580 61723