Wu Dao 2.0 has surpassed OpenAI’s GPT-3 in so many ways.
Wu Dao 2.0 means enlightenment. It is dubbed as China’s first homegrown super-scale intelligent model system, and was led by BAAI Research Academic Vice President and Tsinghua University Professor Tang Jie.
The GPT-3 was deemed so strong and powerful that Microsoft licensed exclusive use of the language model and its underlying code.
Just a year later, however, another language model took over both GPT-3 and Turing-NLG in terms of its innovation and ingenuity.
Wu Dao 2.0 is a monster of 1.75 trillion parameters. It is now the largest neural network ever created and probably the most powerful. Its potential and limits are yet to be fully disclosed, but the expectations are high and rightly so.
Unveiled by the Beijing Academy of Artificial Intelligence (BAAI), China’s latest artificial intelligence model can process natural language text, write essays and poems, and recognise images.
Wu Dao 2.0 is multimodal. It can learn from text and images and tackle tasks that include both types of data. This is something GPT-3 can’t do.
Today’s artificial intelligence (AI) systems inductively “learn” from selected training data, as from experience, observations and trial and error, as if to acquire knowledge on their own, and this is known as machine learning.
Standard machine and deep learning algorithms extract data patterns as statistical correlations from raw data, structured or unstructured. This is true for simple algorithms, like logistic regression, or sophisticated algorithms, like neural networks, which can learn deeper patterns from input data.
But there is a big BUT, which is a fatal flaw in the whole enterprise. Today's AI is largely machine learning techniques, deep learning algorithms and deep neural networks can’t identify causality, its elements and structures, processes and mechanisms, rules and relationships, data and models, all what makes our world.
This leads to all sorts of decision and prediction errors, data and algorithmic biases, the lack of quality data, and implementation failings, or the absence of real machine intelligence and learning.
Integrating the symbolic AI with the statistical AI of Machine Learning, Causal Machine Intelligence and Learning makes the next generation of powerful intelligent machines, running the master [causal] learning algorithms. It allows intelligent machines to think about the world, factual and counterfactual, computing its alternatives and scenarios, and effectively interacting with any complex environments, physical or virtual.
Wu Dao 2.0 reached/surpassed state-of-the-art (SOTA) levels on 9 benchmark tasks widely recognized by the AI community, as reported by BAAI (benchmark: achievement):
Some of the Beijing Academy of Artificial Intelligence (BAAI) most important members expressed their thoughts on Wu Dao 2.0’s role on the road towards AGI (artificial general intelligence):
“The way to artificial general intelligence is big models and big computers. […] What we are building is a power plant for the future of AI. With mega data, mega computing power, and mega models, we can transform data to fuel the AI applications of the future.”
Dr. Zhang Hongjiang, chairman of BAAI
“These sophisticated models, trained on gigantic data sets, only require a small amount of new data when used for a specific feature because they can transfer knowledge already learned into new tasks, just like human beings. […] Large-scale pre-trained models are one of today’s best shortcuts to artificial general intelligence.”
Blake Yan, AI researcher
Wu Dao 2.0 aims to enable machines to think like humans and achieve cognitive abilities beyond the Turing test.”
Tang Jie, lead researcher behind Wu Dao 2.0
They bet on GPT-like multimodal and multitasking models to reach artificial general intelligence (AGI).
Hua Zhibing, China's first virtual student enabled by the country's largest pre-trained model, made its debut two months ago.
With the move, Hua officially became a student in the Department of Computer Science and Technology at Tsinghua University in Beijing.
Hua, a "woman", is able to compose poetry and music and has some ability in reasoning and emotional interaction.
The AI powered student can continuously learn and is now capable of composing poems and drawing pictures, and will be able to code in the future.
The virtual student was co-developed by the Beijing Academy of Artificial Intelligence, Zhipu AI and a company, Xiaoice. It is powered by the second generation of Wu Dao 2.0
Hua will be able to play a role in more scenarios, such as news coverage and popularization of knowledge.
What makes the world nervous is that once China pulls ahead, other nations may never catch up. Language models are often a symbol of how advanced a certain country is in its artificial intelligence capabilities. Once a clearly superior model emerges, such as Wu Dao 2.0, Chinese companies will adopt it in order to compete in the global market. Just as Google runs the playing field for search engines, Wu Dao 2.0 could grow to monopolise the language modelling world.
Seven European AI industry associations, including ones from Germany, Austria, Sweden, Croatia, Slovenia, the Netherlands, France and Bulgaria, approached the EU Commission to draw attention to the situation and propose measures to develop large-scale AI models in Europe.
The latest developments around Wu Dao 2.0 raise fears that Europe is facing the situation of losing its digital sovereignty in the field of artificial intelligence.
From a tech perspective, this is a fascinating achievement made by China. For other countries and regions including the United States, EU and Russia, this a warning signal not to completely fall behind.