TY - JOUR

T1 - A Robust Distributed Hierarchical Online Learning Approach for Dynamic MEC Networks

AU - Wu, Yi Chen

AU - Lin, Che

AU - Quek, Tony Q.S.

N1 - Publisher Copyright:
© 1983-2012 IEEE.

PY - 2022/2/1

Y1 - 2022/2/1

N2 - We consider a resource allocation and offloading decision-making problem in a mobile edge computing (MEC) network. Since the locations of user equipments (UEs) vary over time in practice, we consider a dynamic network, where the UEs could leave or join the network coverage at any location. Since the joint offloading decision that minimizes the network cost also varies with the topology, the expected best offloading decision for the previous topology would not match the new topology. Consequently, the system suffers from recurring cost peaks due to the topology change. Thus, we propose a robust distributed hierarchical online learning approach to enhance the algorithm's robustness and reduce the cost peaks. Specifically, the UEs learn the utility of each offloading decision via deep Q networks (DQNs) from their interaction with the MEC network. Meanwhile, the computational access points (CAPs) train their deep neural networks (DNNs) online with the real-time data collected from the UEs to predict their corresponding Q-value vectors. Therefore, the UEs and CAPs form a hierarchical collaborative-learning structure. When the topology changes, each UE downloads its Q-value vector as the Q-bias vector and learns its difference from the actual Q-value vector via its DQN. With different agents learning distributedly, both the peak and sum costs are reduced as the joint offloading decision could start from a near-local-optimal point. In simulations, our robust approach successfully reduces the peak cost and sum cost by up to 50% and 30%, respectively. This demonstrates the need for a robust learning algorithm design in a practical dynamic MEC network.

AB - We consider a resource allocation and offloading decision-making problem in a mobile edge computing (MEC) network. Since the locations of user equipments (UEs) vary over time in practice, we consider a dynamic network, where the UEs could leave or join the network coverage at any location. Since the joint offloading decision that minimizes the network cost also varies with the topology, the expected best offloading decision for the previous topology would not match the new topology. Consequently, the system suffers from recurring cost peaks due to the topology change. Thus, we propose a robust distributed hierarchical online learning approach to enhance the algorithm's robustness and reduce the cost peaks. Specifically, the UEs learn the utility of each offloading decision via deep Q networks (DQNs) from their interaction with the MEC network. Meanwhile, the computational access points (CAPs) train their deep neural networks (DNNs) online with the real-time data collected from the UEs to predict their corresponding Q-value vectors. Therefore, the UEs and CAPs form a hierarchical collaborative-learning structure. When the topology changes, each UE downloads its Q-value vector as the Q-bias vector and learns its difference from the actual Q-value vector via its DQN. With different agents learning distributedly, both the peak and sum costs are reduced as the joint offloading decision could start from a near-local-optimal point. In simulations, our robust approach successfully reduces the peak cost and sum cost by up to 50% and 30%, respectively. This demonstrates the need for a robust learning algorithm design in a practical dynamic MEC network.

UR - http://www.scopus.com/inward/record.url?scp=85119621928&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85119621928&partnerID=8YFLogxK

U2 - 10.1109/JSAC.2021.3118342

DO - 10.1109/JSAC.2021.3118342

M3 - Article

AN - SCOPUS:85119621928

VL - 40

SP - 641

EP - 656

JO - IEEE Journal on Selected Areas in Communications

JF - IEEE Journal on Selected Areas in Communications

SN - 0733-8716

IS - 2

ER -