A bi-level mobility-aware deep reinforcement learning approach for fault-tolerant task offloading in vehicular edge-cloud computing

چکیده مقاله

Vehicular Edge Cloud Computing (VECC) has emerged as a promising paradigm to support delay-sensitive and computation-intensive applications in Intelligent Transportation Systems (ITS). However, dynamic traffic patterns, fluctuating network conditions, and uncertain resource availability often result in high task latency and service failures. To address these challenges, this paper proposes a bi-level Deep Q-Network (DQN)-based mobility-aware framework for fault-tolerant task offloading in VECC environments. Unlike existing approaches that offload tasks solely to the receiving Roadside Unit (RSU), the proposed framework introduces a level-1 DQN agent that performs high-level scheduling by selecting the most suitable RSU for task execution based on its workload, network latency, and failure rate. In parallel, level-2 DQN agents at each RSU handle low-level decisions, including task allocation and failure-recovery strategy selection, choosing among First Result, Recovery Block, or Retry mechanisms. To eliminate centralized dependency, the level-1 DQN is replicated across RSUs at the edge layer, ensuring high accessibility and resilience for distributed scheduling. Extensive simulations conducted using an integrated SimPy/SUMO environment demonstrate that, under heavy and imbalanced traffic, the proposed bi-level DQN improves the total reward by 7.7% to 37.8% and reduces the task failure rate by 29% to 63% relative to bi-level PPO, Greedy, and No-Forwarding baselines, based on averages over the final 40 training episodes.