Autonomous vehicles operating in uncertain urban environments are required to reason over complex multi-agent interactions while adhering to stringent safety requirements. Hierarchical frameworks often use large models for high-level (virtual-layer) planning and deep reinforcement learning for low-level (physical-layer) control. However, semantic and temporal misalignment between layers leads to command errors and delayed response. We propose HiLo-Align, a hybrid hierarchical framework that unifies both layers via a shared semantic space and time scale. By explicitly modeling cross-layer alignment, HiLo-Align improves control coordination and semantic consistency. Experimental results on both simulation and real-world datasets indicate enhanced collision avoidance, generalization, and robustness in high-risk urban environments.