TY - GEN
T1 - Adaptive Knowledge Sharing in Multi-Task Learning
T2 - Workshop on Research and Applications of Foundation Models for Data Mining and Affective Computing, RAFDA 2024 and International Workshop on Temporal Analytics, IWTA 2024 held in conjunction with the 27th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2023
AU - Chang, Yu Hsiang
AU - Ting, Lo Pang Yun
AU - Yin, Wei Cheng
AU - Su, Ko Wei
AU - Chuang, Kun Ta
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.
PY - 2024
Y1 - 2024
N2 - In time-series machine learning, the challenge of obtaining labeled data has spurred interest in using unlabeled data for model training. Current research primarily focuses on deep multi-task learning, emphasizing the hard parameter-sharing approach. Importantly, when correlations between tasks are weak, indiscriminate parameter sharing can lead to learning interference. Consequently, we introduce a novel framework called DPS, which separates training into dependency-learning and parameter-sharing phases. This structure allows the model to manage knowledge sharing between tasks dynamically. Additionally, we introduce a loss function to align neuron functionalities across tasks, addressing learning interference. Through experiments on real-world datasets, we demonstrate the superiority of DPS over baselines. Moreover, our results shed light on the impacts of the two designed training phases, validating that DPS consistently ensures a degree of learning stability.
AB - In time-series machine learning, the challenge of obtaining labeled data has spurred interest in using unlabeled data for model training. Current research primarily focuses on deep multi-task learning, emphasizing the hard parameter-sharing approach. Importantly, when correlations between tasks are weak, indiscriminate parameter sharing can lead to learning interference. Consequently, we introduce a novel framework called DPS, which separates training into dependency-learning and parameter-sharing phases. This structure allows the model to manage knowledge sharing between tasks dynamically. Additionally, we introduce a loss function to align neuron functionalities across tasks, addressing learning interference. Through experiments on real-world datasets, we demonstrate the superiority of DPS over baselines. Moreover, our results shed light on the impacts of the two designed training phases, validating that DPS consistently ensures a degree of learning stability.
UR - http://www.scopus.com/inward/record.url?scp=85192367640&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85192367640&partnerID=8YFLogxK
U2 - 10.1007/978-981-97-2650-9_12
DO - 10.1007/978-981-97-2650-9_12
M3 - Conference contribution
AN - SCOPUS:85192367640
SN - 9789819726493
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 148
EP - 160
BT - Trends and Applications in Knowledge Discovery and Data Mining - PAKDD 2024 Workshops, RAFDA and IWTA, Proceedings
A2 - Wang, Zhaoxia
A2 - Tan, Chang Wei
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 7 May 2024 through 10 May 2024
ER -