Title:
NEURAL NETWORK TRAINING METHOD AND APPARATUS
Document Type and Number:
WIPO Patent Application WO/2024/082705
Kind Code:
A1
Abstract:
A neural network training method and apparatus, which relate to the field of neural networks, and can reduce the power consumption of training, shorten the training time and improve the performance of a neural network. The solution involves: converting a data matrix and a weight matrix, which are in a first data format, into a data conversion matrix and a weight conversion matrix, which are in a second data format (S201), wherein the total bit width of the second data format is smaller than the total bit width of the first data format; according to the data conversion matrix and the weight conversion matrix, performing layer-by-layer forward calculation on at least one matrix multiplication calculation layer, so as to obtain error data (S202); according to the error data, performing layer-by-layer backpropagation on the at least one matrix multiplication calculation layer, so as to determine an error gradient matrix corresponding to each matrix multiplication calculation layer (S203); and according to an error gradient matrix corresponding to a previous matrix multiplication calculation layer among the at least one matrix multiplication calculation layer, updating parameters corresponding to the current calculation layer (S204), wherein data corresponding to the at least one matrix multiplication calculation layer during forward calculation and backpropagation is data in the second data format.
Inventors:
ZHANG ZHONGXING (CN)
CHEN MINQI (CN)
LUO YUANYONG (CN)
WU WEI HSIANG (CN)
HUANG ZEYI (CN)
CHEN MINQI (CN)
LUO YUANYONG (CN)
WU WEI HSIANG (CN)
HUANG ZEYI (CN)
Application Number:
PCT/CN2023/104242
Publication Date:
April 25, 2024
Filing Date:
June 29, 2023
Export Citation:
Assignee:
HUAWEI TECH CO LTD (CN)
International Classes:
G06N3/04; G06N3/084
Foreign References:
| CN111950689A | 2020-11-17 | |||
| CN111290732A | 2020-06-16 | |||
| CN112836806A | 2021-05-25 | |||
| CN112613604A | 2021-04-06 | |||
| US20200193274A1 | 2020-06-18 | |||
| US20210056423A1 | 2021-02-25 |
Attorney, Agent or Firm:
BEIJING ZBSD PATENT & TRADEMARK AGENT LTD. (CN)
Download PDF:
Previous Patent: SUPPORT MODULE AND ELECTRONIC DEVICE ASSEMBLY
Next Patent: DATA PROCESSING METHOD, DEVICE, AND SYSTEM, AND READABLE STORAGE MEDIUM
Next Patent: DATA PROCESSING METHOD, DEVICE, AND SYSTEM, AND READABLE STORAGE MEDIUM