Ding Lei, author of Generative Artificial Intelligence： The model should be both large and deep, and computing power may not bring breakthroughs.
Interface journalist |
Interface news editor |
At the end of 2022, the appearance of ChatGPT3.5 quickly triggered a wide discussion on concepts such as big language model and generative artificial intelligence. The general knowledge and logical ability displayed by AI big model seems to bring human beings closer to general artificial intelligence (AGI).
In the future-oriented vision, asThe high-order form of artificial intelligence, general artificial intelligence has the same wisdom as human beings or surpasses human wisdom, which can bring profound influence to various vertical fields.
However, focusing on the present, how to better land a large model with multi-modal ability is still being explored. On the one hand, such as Baidu, Iflytek, Shang Tang, Baichuan Intelligent and other technology companies have run into the market and released their own large model products; On the other hand, it seems that this new productivity tool still has no phenomenal landing case.
A person in charge of the application layer product of Big Model Ecology once told Interface News that accessing Big Model has changed the construction form of the original interaction or software infrastructure, as well as the previous way of API and data storage, which is a subject that must be faced by companies accessing Big Model. In addition to the transformation of digital base, the application layer construction of large model has to face the adaptation of various bottom chips and other problems.
According to the judgment of Ding Lei, the author of Generative Artificial Intelligence, the industry deep model fine-tuned on the basis of 100 billion big models is the final solution. Ding Lei is a Ph.D. major in artificial intelligence at Ohio State University, a postdoctoral fellow at Columbia University, and holds an advanced project management certificate from Stanford University. However, his identity is not a simple scholar. Previously, he served as the head of PayPal artificial intelligence platform, chief data scientist of Baidu Finance and other senior positions, and worked in IBM Watson Institute and Beckman Institute of University of Illinois in the United States.
"Because the divergence of the large model can’t be solved, it is necessary to form an industry-specific deep model through appropriate fine-tuning." In his view, the industry deep model is not necessarily a small model, and the size of the parameters depends on the needs of the application scenario. When the communication in the scene is relatively closed and does not need much logic and general knowledge, the small model is enough to meet the demand.
In his new book "Generative Artificial Intelligence", Ding Lei also lists the business model cases and scenarios of generative artificial intelligence in manufacturing, supply chain management, marketing and customer service. Interface News talked with Ding Lei about how to treat the safety of artificial intelligence, how to build a deep model of the industry, and which scenarios are suitable for landing.
Interface News: You mentioned in your book that in the process of moving towards AGI, transfer learning and the ability of the field itself are two important research directions. Will artificial intelligence have cognitive limitations in the process of learning in this way?
Ding Lei:Similar to the human brain is the carrier of human thinking, artificial intelligence should also think through models. Model is the digital carrier of knowledge and logic, and it has been tried to express knowledge and logic in other ways before, but model is the most successful digital carrier. As for the limitations related to whether stereotype will be formed, it can be explained from the perspective of model. As the four elements of artificial intelligence, data, model, computing power and business model determine how AI goes beyond the above limitations, and model is the core element.
Humans also learn laws through existing experience, and generative AI can also generate new pictures. In the usual sense of creativity, I think AI is available, but deeper science and art may not be competent.
Interface news:It has been mentioned in cybernetics that any effective control system must be as complex as the system it controls. How should we treat the safety of artificial intelligence when we go through AGI and move towards superhuman intelligence?
Ding Lei:Explained from the model point of view, in the face of a decision or generated content, it is considered in the form of probability. The probability that the self-driving car needs to stop or respond in other scenes is artificially set.If the probability is low, it thinks that things will not happen, and there must be a compromise threshold, which will involve ethical issues in decision-making.
The learning of AI is all in the statistical sense, that is to say, its decision is reasonable for the whole data sample, but it may make a fuzzy decision when it comes to a single data. So weIt is necessary to stipulate the rationality of the compromise outside the man-machine interface. Without such a consensus, the credibility or unreliability of the probability cannot be defined.
Interface news:The supercomputing power from AIGC to AGI mentioned in the book, is there room for optimization and breakthrough in the algorithm besides the chip upgrade iteration?
Ding Lei:The improvement and promotion of the model structure will certainly help. Computational power is only one aspect. If the structure of the model is not suitable for the data, or the training method is wrong, no amount of computational power is useful. Just like in industrial production, if there is something wrong with the equipment itself, it is useless to send more electricity into it..
So I also put forward the concept of deep model, which is not only large, but also deep and can understand the business problems in the scene. Proper network structure, training methods and data can bring deep models. "I think the traditional concept of "as long as you have computing power, you can bring breakthroughs" is somewhat biased..
Interface news:For example, ChatGPT and iFLYTEK Spark all played the slogan of building ecology and opening the interface of large-scale model development. What is the significance of this ecological attempt in promoting model iteration and commercialization?
Ding Lei:For this model,Open interface is a relatively primary thing at the IT level, and the main purpose is to learn and iterate the model. The model itself gradually adapts to the business scene and gradually improves the effect, which is called model iteration.For example,ChatGPTThere are many plug-ins and applications, which are equivalent to eyes and ears, makingChatGPTCan directly pull data from the application to help it learn iteration.
I think the final ecology will definitely develop in this direction. If I have a model ecology and call it to someone, it doesn’t necessarily mean that he has the ability to learn and evolve. If this scene is not well constructed, the model can’t learn and understand by itself.
Interface news:Can the general large model trained with large corpus and the special small model trained with proprietary corpus be parallel? What do you think of the two ideas?
Ding Lei:I think the industry deep model fine-tuned on the basis of the 100 billion model is the final solution. Because the divergence of the large model cannot be solved,It is necessary to form an industry-specific deep model through appropriate fine-tuning.How to restrict the large model while using its logical ability and general knowledge, so that it can answer the existing answers in the knowledge base when facing questions, is some topics in the frontier research at present.
The industry deep model is not necessarily a small model. The key depends on the needs of application scenarios. Some scenarios may be relatively closed and do not need too much logic and general knowledge. Small models are enough, but more scenarios need large and deep models.
Interface News: How to judge whether the scene is suitable for the use of deep models?
Ding Lei:There are four main conditions. One is that people can’t do it. For example, advertising recommendation may have tens of millions of users every day, which is impossible for people to do.The second is that people can’t do well. Before using big data for financial risk control, there are many dimensions that can be evaluated in the data. If people only judge credit problems through some simple dimensions, it may not be accurate enough; The third is that people are inefficient, compared with some basic copywriting and photo creation work; Finally, people are unstable, such asIn the scene of industrial quality inspection, people can do it, but people may not have such high stability.
Interface News: YouIn the book AI Thinking: The Alchemy of Creating Value from Data, it was emphasized that there should be data-driven thinking. Will this ability increase the literacy gap? How do you view the relationship between AI and digital divide?
Ding Lei:Xiang cHatGPT asks questions, speaks the language of AI, and the final underlying logic isIt is AI thinking, and it is a kind of thinking ability based on data training model.With the increasing use of AI, some people will accept AI faster and better, and the efficiency will be improved. If you can’t accept the improvement of AI, you may fall behind the trend of the times. Then I think the so-called digital divide will be formed.
But AI is just a tool, and it will always be someone else who replaces people.According to the report of PricewaterhouseCoopers, artificial intelligence and related technologies will replace about 26% of the existing jobs in China in the next 20 years, and will also produce38% of the new jobs will eventually add 12% of the jobs, which will only bring structural impact to the society. Artificial intelligence is not a competitor, but a working partner.We must learn to train and use artificial intelligence, so that artificial intelligence can be used by us and work with artificial intelligence.