Title:
FACE-DRIVEN MODEL TRAINING METHOD, AND VIDEO GENERATION METHOD AND APPARATUS
Document Type and Number:
WIPO Patent Application WO/2024/078303
Kind Code:
A1
Abstract:
Embodiments of the present application provide a face-driven model training method, and a video generation method and apparatus. The face-driven model training method comprises: during a training process of a face-driven model, performing text content recognition on sample voice data to obtain a first eigenvector, performing non-phoneme feature recognition to obtain a second eigenvector, and using the first and second eigenvectors as input data of a facial expression prediction submodel to perform facial expression prediction so as to obtain facial expression prediction data; and then, on the basis of the facial expression prediction data and real facial expression data, updating a parameter of a model to be trained. That is, not only is the first eigenvector which is unrelated to an individual feature obtained via extracted from a phoneme dimension, but the second eigenvector which is related to the individual feature is also obtained via extraction from a non-phoneme dimension, and at the same time, a parameter of the facial expression prediction submodel is updated on the basis of the first eigenvector and the second eigenvector.
Inventors:
YANG CHUNYONG (CN)
JIANG NING (CN)
LIU MIN (CN)
ZENG LINCHENGXI (CN)
JIANG NING (CN)
LIU MIN (CN)
ZENG LINCHENGXI (CN)
Application Number:
PCT/CN2023/120778
Publication Date:
April 18, 2024
Filing Date:
September 22, 2023
Export Citation:
Assignee:
MASHANG CONSUMER FINANCE CO LTD (CN)
International Classes:
G06V40/16
Foreign References:
| CN112634413A | 2021-04-09 | |||
| CN113380271A | 2021-09-10 | |||
| CN114581980A | 2022-06-03 | |||
| CN115116109A | 2022-09-27 | |||
| CN114419702A | 2022-04-29 |
Attorney, Agent or Firm:
COHORIZON INTELLECTUAL PROPERTY INC. (CN)
Download PDF: