Democratizing the optimization of AI’s arcane neural networks

We’re only a few weeks into the new year, but already we’re seeing signs that automated machine learning modeling, sometimes known as autoML, is rising to a new plateau of sophistication.

Specifically, it appears that a promising autoML approach known as “neutral architecture search” will soon become part of data scientists’ core toolkits. This refers to tools and methodologies for automating creation of optimized architectures for convolutional, recurrent, and other neural network architectures at the heart of AI’s machine learning models.

Neural architecture search tools optimize the structure, weights, and hyperparameters of a machine learning model’s algorithmic “neurons” in order to make them more accurate, speedy, and efficient in performing data-driven inferences. This technology has only recently begun to emerge from labs devoted to basic research in AI tools and techniques. The research literature shows that neural architecture search tools have already outperformed manually designed neural nets in many AI R&D projects.

Commercialization coming to neural architecture search

Within the burgeoning autoML space, neural architecture search is showing signs of early commercialization.

At CES 2020 in Las Vegas earlier this month, I met with Montreal-based AI startup Deeplite. Its Lightweight Intelligence tool can automatically optimize a neural network for high-performance inferencing on a range of edge-device hardware platforms. It does this without requiring manual inputs or guidance from scarce, expensive data scientists.

To see how Deeplite’s tool accomplishes this, check out this discussion of the firm’s partnership with Taiwanese company Andes Technology. The RL (reinforcement learning) engine in Deeplite’s hardware-aware neural-architecture search engine automatically found, trained and deployed large neural network models to Andes’ RISC-V hardware. It compressed MobileNet models that were trained on a Visual Wake Words dataset from 13MB down to less than 188KB, a drop of almost 99 percent, with only a 1 percent drop in neural-net inferencing accuracy.

Another key milestone in the maturation of neural architecture search was Amazon’s recent launch of an open source autoML toolkit with this capability built in. Released the same week as CES, Amazon’s new AutoGluon tool enables AI developers of all skill levels to automate the optimization of new or existing models for high-performance inferencing on diverse target hardware platforms.

AutoGluon automates data preparation, model development, hyperparameter tuning, and training within the devops flow of an ML model. It can optimize existing PyTorch and MXNet ML models. It can also interface with existing AI devops pipelines via APIs to automatically tweak an existing ML model and thereby improve its performance of inferencing tasks.

Amazon currently has AutoGluon running on Linux platforms but has announced plans for MacOS and Windows support. Available from this project website or GitHub, AutoGluon can automatically generate a high-performance ML model from as few as three lines of Python code. It taps into available compute resources and uses reinforcement learning algorithms to search for the best fitting neural network architecture for its target environment.

AutoGluon uses RL to speed automated neural architecture searches using computing resources efficiently. Indeed, RL—as implemented both in AutoGluon and in Deeplite’s solution—is proving to be the most fruitful approach for recent advances in this area, using agent-centric actions and rewards to search the space of optimal neural architectures based on estimates of the performance of trained architectures on unseen data.

RL is an up-and-coming alternative to evolutionary algorithms, which have been central to neural architecture search since the 1990s in AI R&D environments. Evolutionary algorithms are still widely used in lab environments such as OpenAI, Uber Labs, Sentient labs (noe Evolv), DeepMind and Google Brain.

Still early for neural architecture search in mainstream AI devops

As autoML data science platforms become prevalent in the enterprise world, neural architecture search tools such as these will be a standard component. However, this capability is still scarcely evident in most AI devops environments.

By the end of this year, I predict that more than half of commercial and open source AI devops workbenches will add neural architecture search as an integrated feature. As autoML gains adoption and improves, it will boost data scientists’ productivity by guiding their decisions regarding whether to build their models on established machine learning algorithms, such as linear regression and random forest algorithms, or on any of the newer, more advanced neural-network algorithms.

As the decade proceeds, neural architecture search will reduce the need for data scientists to understand the neural-net guts of their ML models. This emerging approach will democratize AI by freeing developers to evolve their skillset away from tweaking arcane algorithms and toward developing powerfully predictive intelligent apps.