Truth Inference from Crowdsourcing with Optimal Return of Interest:A Strategic Macro Assignment and Micro Optimization Paradigm

Abstract

In the big data era the flourishing development of Internet services brings a lot of user generated data in which most new information cannot be systematically retrieved by current knowledge bases For example a dramatic number of new hashtags appear in the social media every day resulting in much unknown but valuable knowledge that requires reliable category/attribute labeling strategies The crowdsourcing platform provides an effective tool to leverage opinions from the Internet crowd In this thesis we propose incorporating varied task importance called Return of Interest (RoI) into resource allocation in crowdsourcing The awareness of RoI is important in the business sense but it introduces new challenges For instance a commonly applied strategy called single-objective optimization cannot be naively applied to achieve desirable results In this thesis we propose a two-phase framework called Macro-Assignment and Micro-Optimization (MAMO) to simultaneously consider the issue of budget allocation and the chance of iteratively obtaining RoI To the better practicability we apply the real practice in crowdsourcing platforms where workers are divided into diverse pools corresponding to their quality and cost With the fixed budget we prove that worker allocation to diverse pools for the best expectation of RoI in return is a NP-hard challenge We propose a Dynamic-Programming strategy to resolve the issue effectively As shown in our experimental results we demonstrate that the DP-based strategy can significantly outperform the baseline greedy approaches also indicating its feasibility to be deployed as the standard component for budget allocation in crowdsourcing

Date of Award	2017 Aug 11
Original language	English
Supervisor	Kun-Ta Chuang (Supervisor)