News classification
Contact us
- Add: No. 9, North Fourth Ring Road, Haidian District, Beijing. It mainly includes face recognition, living detection, ID card recognition, bank card recognition, business card recognition, license plate recognition, OCR recognition, and intelligent recognition technology.
- Tel: 13146317170 廖经理
- Fax:
- Email: 398017534@qq.com
Neural evolution is the future of deep learning
Neural evolution is the future of deep learning
Over the past few years, we have a good team working on artificial intelligence research and experiments. The team focuses on developing new evolutionary computation methods (EC), including designing artificial neural network architecture, building business applications, and using natural evolutionary methods to deal with challenging computing problems. The momentum of this category is very strong. We believe that evolutionary computing is likely to be the next serious problem in artificial intelligence technology.
EC, like Deep Learning (DL), was introduced decades ago, and EC can also be promoted from available computing and large data. But it deals with a very different demand: we all know that DL emphasizes learning about what we know, and EC focuses on creating new learning. In this sense, it is the next step of DL: DL can identify objects and speech in familiar categories, and EC enables us to discover new objects and behaviors to maximize the objects and behaviors of specific purposes. Therefore, EC makes many new applications possible: designing more effective behaviors for robots and virtual agents, inventing more effective and cheaper health interventions, promoting agricultural mechanization and biological processes.
Not long ago, we released 5 papers and reports, which made significant pause in the field. The report focused on three aspects: (1) the DL architecture has reached the latest technology level in three standardized machine learning benchmarks. (2) the performance and reliability of the development technology used in the application of progressive practice. (3) the treatment of evolutionary problems is proved on very difficult computational problems.
This article will focus on the first category in which the DL architecture is optimized with EC.
Sentient is a reminder of the breaking discussion of neural evolution
The overall situation of deep learning depends on the scope and complexity of the network. With the evolution of the nerve, the structure of the DL system (i.e., network topology, modules and super parameters) can be stopped out of the human ability. We will introduce three examples in this article: Omni Draw, Celeb Match, and Music Maker (speech modeling). In these three examples, Sentient uses neural evolution to triumphantly surpass the most advanced DL benchmarks.
Music manufacturing (speech modeling)
In the language modeling system is used to predict the category of exercise "speech library" in a word, such as "the Wall Street journal" in a few years a large number of text convergence, forecast results in the network, the input can also be cycle input, so the network can generate a good sequence of words. Interestingly, the same technology also applies to the music sequence, and the following is a demonstration. The user enters some initial notes, and the system improvises a perfect melody on the basis of the starting point. After neuron evolution, Sentient optimized the design of gated periodicity (long term short-term memory or LSTM) node, that is, the memory structure of network, making the model more accurate when predicting the next note.
In the category of speech modeling (in a speech corpus called Penn Tree Bank, we predict the next word), the benchmark is defined by the perplexity point, which is used to measure the probability model how to predict the real sample. Of course, the lower the better, because we want the model to be "puzzled" as little as possible when predicting the next word. In this situation, the perceptron lost the standard LSTM structure with 10.8 of the bewilderment clicks. It is worth noting that, in the past 25 years, although human beings have designed some LSTM variants, the performance of LSTM has not improved. In fact, our neuroevolutionary experiments indicate that LSTM can significantly improve performance through increasing complexity, namely, memory cells and more nonlinear and parallel ways.
Why is this breaking very important? Speech is a powerful and complex intelligent structure of human beings. Speech modeling, the next word in the prediction text, is a benchmark for how machine learning approaches how to learn speech construction. Therefore, it is an agent for constructing natural language disposal system, including speech and speech interfaces, Machine Translation, and even medical data such as DNA sequence and heart rate diagnosis. We can do better in the speech modeling benchmarks, and we can use the same technology to set up a better speech disposal system.
Omni Draw
Omniglot is a handwritten character recognition benchmark that recognizes 50 different alphabetic characters, including authentic speech like Cyril, written Japanese, Japanese and Hebrew, and artificial speech such as Tengwar.
The example above shows multitask learning. The model can learn all the words at the same time and apply the relationship between characters in different languages. For example, when users input images, the system outputs different speech meanings based on matching output. "This will be X in Latin, Y in Japanese and Z in Tengwar". This is different from a single task learning environment. In a single environment, models only stop exercising one language, and cannot set up the same cohesion on speech dataset.
Although Omniglot is an example of a dataset, the data of each speech is relatively small. For example, it may only be a few Greek letters, but many of them are Japanese. It can use the knowledge of the relationship between words to search for a processing plan. Why is this important? For many practical applications, the acquisition of flag data is very expensive or risky (such as medical applications, agriculture and robot rescue), so it can be applied to automatically design the relationship with similar or related data sets.
EC, like Deep Learning (DL), was introduced decades ago, and EC can also be promoted from available computing and large data. But it deals with a very different demand: we all know that DL emphasizes learning about what we know, and EC focuses on creating new learning. In this sense, it is the next step of DL: DL can identify objects and speech in familiar categories, and EC enables us to discover new objects and behaviors to maximize the objects and behaviors of specific purposes. Therefore, EC makes many new applications possible: designing more effective behaviors for robots and virtual agents, inventing more effective and cheaper health interventions, promoting agricultural mechanization and biological processes.
Not long ago, we released 5 papers and reports, which made significant pause in the field. The report focused on three aspects: (1) the DL architecture has reached the latest technology level in three standardized machine learning benchmarks. (2) the performance and reliability of the development technology used in the application of progressive practice. (3) the treatment of evolutionary problems is proved on very difficult computational problems.
This article will focus on the first category in which the DL architecture is optimized with EC.
Sentient is a reminder of the breaking discussion of neural evolution
The overall situation of deep learning depends on the scope and complexity of the network. With the evolution of the nerve, the structure of the DL system (i.e., network topology, modules and super parameters) can be stopped out of the human ability. We will introduce three examples in this article: Omni Draw, Celeb Match, and Music Maker (speech modeling). In these three examples, Sentient uses neural evolution to triumphantly surpass the most advanced DL benchmarks.
Music manufacturing (speech modeling)
In the language modeling system is used to predict the category of exercise "speech library" in a word, such as "the Wall Street journal" in a few years a large number of text convergence, forecast results in the network, the input can also be cycle input, so the network can generate a good sequence of words. Interestingly, the same technology also applies to the music sequence, and the following is a demonstration. The user enters some initial notes, and the system improvises a perfect melody on the basis of the starting point. After neuron evolution, Sentient optimized the design of gated periodicity (long term short-term memory or LSTM) node, that is, the memory structure of network, making the model more accurate when predicting the next note.
In the category of speech modeling (in a speech corpus called Penn Tree Bank, we predict the next word), the benchmark is defined by the perplexity point, which is used to measure the probability model how to predict the real sample. Of course, the lower the better, because we want the model to be "puzzled" as little as possible when predicting the next word. In this situation, the perceptron lost the standard LSTM structure with 10.8 of the bewilderment clicks. It is worth noting that, in the past 25 years, although human beings have designed some LSTM variants, the performance of LSTM has not improved. In fact, our neuroevolutionary experiments indicate that LSTM can significantly improve performance through increasing complexity, namely, memory cells and more nonlinear and parallel ways.
Why is this breaking very important? Speech is a powerful and complex intelligent structure of human beings. Speech modeling, the next word in the prediction text, is a benchmark for how machine learning approaches how to learn speech construction. Therefore, it is an agent for constructing natural language disposal system, including speech and speech interfaces, Machine Translation, and even medical data such as DNA sequence and heart rate diagnosis. We can do better in the speech modeling benchmarks, and we can use the same technology to set up a better speech disposal system.
Omni Draw
Omniglot is a handwritten character recognition benchmark that recognizes 50 different alphabetic characters, including authentic speech like Cyril, written Japanese, Japanese and Hebrew, and artificial speech such as Tengwar.
The example above shows multitask learning. The model can learn all the words at the same time and apply the relationship between characters in different languages. For example, when users input images, the system outputs different speech meanings based on matching output. "This will be X in Latin, Y in Japanese and Z in Tengwar". This is different from a single task learning environment. In a single environment, models only stop exercising one language, and cannot set up the same cohesion on speech dataset.
Although Omniglot is an example of a dataset, the data of each speech is relatively small. For example, it may only be a few Greek letters, but many of them are Japanese. It can use the knowledge of the relationship between words to search for a processing plan. Why is this important? For many practical applications, the acquisition of flag data is very expensive or risky (such as medical applications, agriculture and robot rescue), so it can be applied to automatically design the relationship with similar or related data sets.