FIELD: computing.
SUBSTANCE: present invention relates to methods and systems for determining the outcome of a task in a crowdsourcing environment. The server at the training stage sends a training task associated with a predetermined training response to a lot of users of the crowdsourcing environment; gets a set of training results corresponding to the training task; receives a history of user actions, containing previously collected user actions and characteristics of user actions in a crowdsourced environment; assigns to each of the plurality of training results a label value representing a first label value assigned to one or more training results corresponding to a predetermined training answer, or a second label value assigned to one or more training results not corresponding to a predetermined training response; generates a set of training data triplets, each of which is associated with a training result from a set of training results and contains a training result, a history of user actions associated with the training result and a label value assigned to the training result; trains a machine learning algorithm (MLA), including determining for each triplet of training data a set of features containing the property of a triplet of training data, and generating, based on the set of features, a predictive function capable, based on the history of user actions, to determine the error parameter of the result obtained from the user for the problem, representing is the probability that the result is incorrect.
EFFECT: technical result consists in the possibility of determining the result for a task in a crowdsourcing environment.
20 cl, 6 dwg
Title | Year | Author | Number |
---|---|---|---|
TRAINING METHOD OF RATING MODULE USING THE TRAINING SELECTION WITH THE INTERFERENCE LABELS | 2016 |
|
RU2632143C1 |
METHOD AND SERVER FOR REPEATED TRAINING OF MACHINE LEARNING ALGORITHM | 2019 |
|
RU2743932C2 |
METHOD AND SYSTEM FOR DETERMINING RESULT OF TASK EXECUTION IN CROWDSOURCED ENVIRONMENT | 2019 |
|
RU2744032C2 |
METHOD FOR PERFORMING TASKS | 2018 |
|
RU2743898C1 |
METHOD AND SYSTEM FOR CLASSIFYING WORD AS OBSCENE WORD | 2020 |
|
RU2803576C2 |
METHOD AND SYSTEM FOR GENERATING AN OBJECT CARD | 2018 |
|
RU2739554C1 |
SYSTEM AND METHOD OF FORMING TRAINING SET FOR MACHINE LEARNING ALGORITHM | 2017 |
|
RU2711125C2 |
METHOD AND SYSTEM FOR RANKING DIGITAL OBJECTS BASED ON TARGET CHARACTERISTIC RELATED TO THEM | 2019 |
|
RU2757174C2 |
METHOD AND SERVER FOR TRAINING MACHINE LEARNING ALGORITHM IN TRANSLATION | 2020 |
|
RU2770569C2 |
METHOD AND SERVER FOR TRAINING MACHINE LEARNING ALGORITHM IN OBJECT RANKING | 2020 |
|
RU2782502C1 |
Authors
Dates
2021-03-02—Published
2019-05-27—Filed