Hokkaido University Research Profiles


quality control: 1

Numbers of lines 20 50 No Page Break Theme Icons
  • Life Sciences
  • Information and Communication
  • Nanotechnology / Materials
  • Manufacturing Technology
  • Human and Social Sciences
  • Energy
  • Environment
  • Tourism / Community development
  • Arctic Research
  • Social Infrastructure
  • Open Facilities
  • Quality Control in Crowdsourcing

    Highly accurate decision-making using people’s confidence rating

    We are conducting research to guarantee the quality of work results in crowdsourcing, which allows us to commission jobs to many people via the Internet. The use of workers’ confidence rating on the work results will be effective to ensure high-quality work results.


    With the advent of crowdsourcing services in recent years, it has become easy to commission jobs (tasks) to a large number of people via the Internet, and these services are being used in various fields of information science (image recognition, natural language processing, information retrieval, databases, etc.). In crowdsourcing, it is important to check the work quality because not all workers necessarily have the required skills and diligence to work on a task. We have proposed a method to ensure the work quality by asking workers to report their confidence (degree of conviction) in their work results. The technical feature of this method is that it does not trust the confidence rating reported by workers as it is, but performs statistical quality control assuming the existence of over- and under-confident workers.