Optimal Transmission Scheduling Over Multihop Networks: Structural Results and Reinforcement Learning

• 成果类型: Article
• 署名作者: Yang, Lixin; Xu, Yong; Lv, Weijun; Li, Jun-Yi; Shi, Ling
• 署名单位: Guangdong University of Technology; Hong Kong University of Science & Technology
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3327622
• 发表日期: 2024
• 页码: 1826-1833
• 关键词: Intelligent sensors; wireless sensor networks; Spread spectrum communication; Wireless communication; Scheduling; State estimation; Reinforcement Learning; estimation; Kalman filtering; transmission scheduling; sensor networks

摘要

This article studies the optimal transmission scheduling for remote state estimation over multihop networks. A smart sensor observes a dynamic system, and sends its local state estimate to a remote estimator (RE). To save energy, multihop networks are deployed to relay data packets from the smart sensor to the RE. The smart sensor needs to decide the hop number communicating with the RE by adjusting its transmission power. To minimize the estimation error and the energy consumption, the transmission scheduling is formulated as a modified Markov decision process (MDP) by incorporating historical actions into the state. A sufficient condition is constructed to guarantee that the MDP has an optimal deterministic and stationary policy. The optimal policy's structure is further obtained to reduce the computation complexity. A deep reinforcement learning algorithm, i.e., dueling double Q-network, is introduced to obtain a near-optimal policy. Finally, a simulation example is provided to illustrate the developed results.