TY - GEN
T1 - Return-of-Interest Conscious Truth Inference for Crowdsourcing Queries
AU - Leung, Lok Him
AU - Yang, Po An
AU - Chuang, Kun Ta
N1 - Funding Information:
This work was supported in part by Ministry of Science and Technology, R.O.C., under Contract 109-2221-E-006 -187-MY3, 110-2221-E-006-001.
Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - In the big data era, the flourishing development of Internet services brings a lot of user generated data, in which most new information cannot be systematically retrieved by current knowledge bases. For example, a dramatic number of new hashtags appear in the social media every day, resulting in much unknown but valuable knowledge that requires reliable category/attribute labeling strategies. The crowdsourcing platform provides an effective tool to leverage opinions from the Internet crowd. In this paper, we propose incorporating varied task importance, called Return of Interest (RoI), into resource allocation in crowdsourcing. The awareness of RoI is important in the business sense, but it introduces new challenges. In this paper, we propose a two-phase framework, called Macro-Assignment and Micro-Optimization (MAMO), to simultaneously consider the issue of budget allocation and the chance of iteratively obtaining RoI. With the fixed budget, we prove that worker allocation to diverse pools for the best expectation of RoI in return is a NPhard challenge. We propose a Dynamic-Programming strategy to resolve the issue effectively. As shown in our experimental results, we demonstrate that the DP-based strategy can significantly outperform the baseline greedy approaches, also indicating its feasibility to be deployed as the standard component for budget allocation in crowdsourcing.
AB - In the big data era, the flourishing development of Internet services brings a lot of user generated data, in which most new information cannot be systematically retrieved by current knowledge bases. For example, a dramatic number of new hashtags appear in the social media every day, resulting in much unknown but valuable knowledge that requires reliable category/attribute labeling strategies. The crowdsourcing platform provides an effective tool to leverage opinions from the Internet crowd. In this paper, we propose incorporating varied task importance, called Return of Interest (RoI), into resource allocation in crowdsourcing. The awareness of RoI is important in the business sense, but it introduces new challenges. In this paper, we propose a two-phase framework, called Macro-Assignment and Micro-Optimization (MAMO), to simultaneously consider the issue of budget allocation and the chance of iteratively obtaining RoI. With the fixed budget, we prove that worker allocation to diverse pools for the best expectation of RoI in return is a NPhard challenge. We propose a Dynamic-Programming strategy to resolve the issue effectively. As shown in our experimental results, we demonstrate that the DP-based strategy can significantly outperform the baseline greedy approaches, also indicating its feasibility to be deployed as the standard component for budget allocation in crowdsourcing.
UR - http://www.scopus.com/inward/record.url?scp=85131916474&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85131916474&partnerID=8YFLogxK
U2 - 10.1109/TAAI54685.2021.00020
DO - 10.1109/TAAI54685.2021.00020
M3 - Conference contribution
AN - SCOPUS:85131916474
T3 - Proceedings - 2021 International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021
SP - 60
EP - 65
BT - Proceedings - 2021 International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 26th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021
Y2 - 18 November 2021 through 20 November 2021
ER -