Title:
METHOD AND APPARATUS FOR NATURAL LANGUAGE PROCESSING AND MODEL TRAINING, DEVICE AND STORAGE MEDIUM
Document Type and Number:
WIPO Patent Application WO/2024/074100
Kind Code:
A1
Abstract:
The disclosure relates to a method and apparatus for natural language processing and model training, a device and a storage medium. In the present disclosure, a machine learning model is pre-trained by means of each triple, so that the pre-trained machine learning model can seamlessly and naturally process various natural language understanding tasks in a machine reading comprehension paradigm. In addition, the data format used for model training in the pre-training stage is consistent with the data format used for model training in a fine-tuning stage, such that a pre-trained target is the same as a fine-tuned target, and thus, seamless transition can be achieved from the pre-training stage to the fine-tuning stage. After the model is pre-trained by using a large amount of low-cost data, the pre-trained machine learning model can be calibrated by means of a small amount of target task data, such that the learned general knowledge in the pre-training stage is smoothly migrated to the fine-tuned model, and the accuracy of the fine-tuned model is ensured.
More Like This:
Inventors:
XU WEIWEN (CN)
LI XIN (CN)
ZHANG WENXUAN (SG)
BING LIDONG (SG)
SI LUO (CN)
LI XIN (CN)
ZHANG WENXUAN (SG)
BING LIDONG (SG)
SI LUO (CN)
Application Number:
PCT/CN2023/121267
Publication Date:
April 11, 2024
Filing Date:
September 25, 2023
Export Citation:
Assignee:
ALIBABA DAMO HANGZHOU TECH CO LTD (CN)
International Classes:
G06F40/211; G06F16/33; G06F40/295; G06N20/00
Domestic Patent References:
| WO2020174826A1 | 2020-09-03 |
Foreign References:
| CN112507706A | 2021-03-16 | |||
| CN114565104A | 2022-05-31 | |||
| CN115879440A | 2023-03-31 | |||
| CN111581350A | 2020-08-25 |
Attorney, Agent or Firm:
BEIJING TONGJUN LAW FIRM (CN)
Download PDF: