Structure-Aware Reinforcement Learning for Optimal Transmission Scheduling Over Packet Length-Dependent Lossy Networks

• 成果类型: Article
• 署名作者: Yang, Lixin; Lv, Weijun; Xu, Yong; Tao, Jie; Quevedo, Daniel E.
• 署名单位: Queensland University of Technology (QUT); Guangdong University of Technology
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2024.3488823
• 发表日期: 2025
• 页码: 2576-2583
• 关键词: sensors; Scheduling; Wireless communication; Reinforcement Learning; Fading channels; Propagation losses; Periodic structures; wireless sensor networks; State estimation; Kalman filters; Packet length-dependent lossy networks; remote state estimation; structure-aware reinforcement learning (RL); transmission scheduling

摘要

The optimal transmission scheduling over packet length-dependent lossy networks is investigated. A set of sensors observe some linear time-invariant systems, and obtain their local state estimates using Kalman filters independently. The local state estimates are encoded in a data packet, which is sent to a remote estimator over packet length-dependent lossy networks, i.e., the packet arrival probability is exponentially decreasing with the packet length. The tradeoff arises between the packet arrival probability and the data size. To improve the remote estimation performance, one needs to design a transmission scheduling policy to determine how many as well as which sensors transmit data at each time instant, which is formulated here as a Markov decision process (MDP) model. There exists an optimal stationary policy for the MDP model, which is verified to possess a threshold structure. Based on this, a structure-aware reinforcement learning algorithm is proposed to approximate the MDP's optimal policy. Some simulation examples are given to verify the optimal policy's structure, and illustrate the performance of the proposed structure-aware reinforcement learning algorithm.