Wu Dao 2.0: Why China is Leading the Artificial Intelligence Race?

Wu Dao 2.0 has surpassed OpenAI’s GPT-3 in so many ways.

China could grow to monopolise the language modelling world. Artificial intelligence models have become a strong informal indicator of national and continental progress.

Introducing Wu Dao 2.0

Wu Dao 2.0 means enlightenment. It is dubbed as China’s first homegrown super-scale intelligent model system, and was led by BAAI Research Academic Vice President and Tsinghua University Professor Tang Jie. 

The GPT-3 was deemed so strong and powerful that Microsoft licensed exclusive use of the language model and its underlying code.

Just a year later, however, another language model took over both GPT-3 and Turing-NLG in terms of its innovation and ingenuity.

Wu Dao 2.0 is a monster of 1.75 trillion parameters. It is now the largest neural network ever created and probably the most powerful. Its potential and limits are yet to be fully disclosed, but the expectations are high and rightly so.

Unveiled by the Beijing Academy of Artificial Intelligence (BAAI), China’s latest artificial intelligence model can process natural language text, write essays and poems, and recognise images. 

Wu Dao 2.0 is multimodal. It can learn from text and images and tackle tasks that include both types of data. This is something GPT-3 can’t do.

What's Wrong With Today's Artificial Intelligence?

Today’s artificial intelligence (AI) systems inductively “learn” from selected training data, as from experience, observations and trial and error, as if to acquire knowledge on their own, and this is known as machine learning. 

Standard machine and deep learning algorithms extract data patterns as statistical correlations from raw data, structured or unstructured. This is true for simple algorithms, like logistic regression, or sophisticated algorithms, like neural networks, which can learn deeper patterns from input data. 

But there is a big BUT, which is a fatal flaw in the whole enterprise. Today's AI is largely machine learning techniques, deep learning algorithms and deep neural networks can’t identify causality, its elements and structures, processes and mechanisms, rules and relationships, data and models, all what makes our world.

This leads to all sorts of decision and prediction errors, data and algorithmic biases, the lack of quality data, and implementation failings, or the absence of real machine intelligence and learning.

Integrating the symbolic AI with the statistical AI of Machine Learning, Causal Machine Intelligence and Learning makes the next generation of powerful intelligent machines, running the master [causal] learning algorithms. It allows intelligent machines to think about the world, factual and counterfactual, computing its alternatives and scenarios, and effectively interacting with any complex environments, physical or virtual.

Wu Dao 2.0 Benchmark Achievements

Wu Dao 2.0 reached/surpassed state-of-the-art (SOTA) levels on 9 benchmark tasks widely recognized by the AI community, as reported by BAAI (benchmark: achievement):

• ImageNet (zero-shot): SOTA, surpassing OpenAI CLIP.
• LAMA (factual and commonsense knowledge): Surpassed AutoPrompt.
• LAMBADA (cloze tasks): Surpassed Microsoft Turing NLG.
• SuperGLUE (few-shot): SOTA, surpassing OpenAI GPT-3.
• UC Merced Land Use (zero-shot): SOTA, surpassing OpenAI CLIP.
• MS COCO (text generation diagram): Surpassed OpenAI DALL·E.
• MS COCO (English graphic retrieval): Surpassed OpenAI CLIP and Google ALIGN.
• MS COCO (multilingual graphic retrieval): Surpassed UC² (best multilingual and multimodal pre-trained model).
• Multi 30K (multilingual graphic retrieval): Surpassed UC².

Wu Dao 2.0: A Monster of 1.75 Trillion Parameters

Some of the Beijing Academy of Artificial Intelligence (BAAI) most important members expressed their thoughts on Wu Dao 2.0’s role on the road towards AGI (artificial general intelligence):

“The way to artificial general intelligence is big models and big computers. […] What we are building is a power plant for the future of AI. With mega data, mega computing power, and mega models, we can transform data to fuel the AI applications of the future.”

Dr. Zhang Hongjiang, chairman of BAAI

“These sophisticated models, trained on gigantic data sets, only require a small amount of new data when used for a specific feature because they can transfer knowledge already learned into new tasks, just like human beings. […] Large-scale pre-trained models are one of today’s best shortcuts to artificial general intelligence.”

Blake Yan, AI researcher

Wu Dao 2.0 aims to enable machines to think like humans and achieve cognitive abilities beyond the Turing test.”

Tang Jie, lead researcher behind Wu Dao 2.0

They bet on GPT-like multimodal and multitasking models to reach artificial general intelligence (AGI).

China Developed The First AI Powered Virtual Student

China is Now Ahead in the Artificial Intelligence Race

What makes the world nervous is that once China pulls ahead, other nations may never catch up. Language models are often a symbol of how advanced a certain country is in its artificial intelligence capabilities. Once a clearly superior model emerges, such as Wu Dao 2.0, Chinese companies will adopt it in order to compete in the global market. Just as Google runs the playing field for search engines, Wu Dao 2.0 could grow to monopolise the language modelling world.

Seven European AI industry associations, including ones from Germany, Austria, Sweden, Croatia, Slovenia, the Netherlands, France and Bulgaria, approached the EU Commission to draw attention to the situation and propose measures to develop large-scale AI models in Europe.

The latest developments around Wu Dao 2.0 raise fears that Europe is facing the situation of losing its digital sovereignty in the field of artificial intelligence.

From a tech perspective, this is a fascinating achievement made by China. For other countries and regions including the United States, EU and Russia, this a warning signal not to completely fall behind.