TRAINING NEURAL NETWORKS USING LOSS FUNCTIONS REFLECTING RELATIONSHIPS BETWEEN NEIGHBOURING TOKENS Russian patent published in 2020 - IPC G06N3/02 

FIELD: data processing.

SUBSTANCE: invention relates to methods and computer-readable data medium for training neural networks. Method comprises determining a first tag associated with a current token processed by a neural network, a second tag associated with a previous token, processed by the neural network before processing the current token, a third tag associated with the next token to be processed by the neural network after processing the current token; calculating, for a training data sample, a loss function value reflecting a first, second and third loss values, respectively presented by the first difference between the first tag and the first label associated with the current word of the training data sample, a second difference between the second tag and the second label associated with the previous word of the training data sample, the third difference between the third tag and the third label associated with the next token of the training data sample; and tuning one or more parameters of the neural network depending on the value of the loss function.

EFFECT: technical result consists in improvement of mark-up quality of input sequences performed by a neural network.

20 cl, 5 dwg