Research

With the advent of crowdsourcing services in recent years, it has become easy to commission jobs (tasks) to a large number of people via the Internet, and these services are being used in various fields of information science (image recognition, natural language processing, information retrieval, databases, etc.). In crowdsourcing, it is important to check the work quality because not all workers necessarily have the required skills and diligence to work on a task. We have proposed a method to ensure the work quality by asking workers to report their confidence (degree of conviction) in their work results. The technical feature of this method is that it does not trust the confidence rating reported by workers as it is, but performs statistical quality control assuming the existence of over- and under-confident workers.