Hierarchical Multi-Agent Reinforcement Learning Framework with Convergence Analysis for Adaptive Control in Dynamic Environments