Title:
DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE.
Document Type and Number:
WIPO Patent Application WO/2024/083180
Kind Code:
A1
Abstract:
A computer implemented method includes performing a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN) using a first set of hidden weights stored in a first matrix comprising a Resistive Processing Unit (RPU) crossbar array. A second matrix comprising a second set of hidden weights is stored in a digital medium. A third matrix comprising a set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a sign-change (chopper). The third matrix is stored in the digital medium. A third set of weights is updated for the DNN from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a RPU crossbar array.
More Like This:
Inventors:
RASCH MALTE JOHANNES (US)
Application Number:
PCT/CN2023/125373
Publication Date:
April 25, 2024
Filing Date:
October 19, 2023
Export Citation:
Assignee:
IBM (US)
IBM CHINA CO LTD (CN)
IBM CHINA CO LTD (CN)
International Classes:
G06N3/084
Domestic Patent References:
| WO2021056112A1 | 2021-04-01 |
Foreign References:
| US20220327375A1 | 2022-10-13 | |||
| US20220172072A1 | 2022-06-02 | |||
| US20210110269A1 | 2021-04-15 | |||
| US20220083843A1 | 2022-03-17 | |||
| US20190164538A1 | 2019-05-30 | |||
| CN110942141A | 2020-03-31 |
Attorney, Agent or Firm:
ZHONGZI LAW OFFICE (CN)
Download PDF: