Accurate identification of time-variant modal parameters in base-isolated structures is crucial for understanding their dynamic response under seismic excitations. This study introduces a physics-informed long short-term memory (PI-LSTM) approach with gradient-based optimization for time-varying modal identification, referred to as PI-LSTM-Modξ(var), able to track instantaneous modal properties, including natural frequencies, damping ratios, and mode shapes. By leveraging machine learning capabilities, the proposed method optimizes modal parameter estimation within short time windows of recorded input-output vibration data, ensuring continuity and robustness in capturing nonlinear behaviors such as stiffness degradation and damping variability. The methodology is validated through a comprehensive case study of the BNCS building, a full-scale base-isolated reinforced concrete structure tested on a shake table, subjected to a suite of seismic inputs. The results demonstrate great performance in tracking time-variant properties and reduced normalized root mean square error (NRMSE) in replicating experimental acceleration and displacement responses, even during strong-motion phases. The PI-LSTM-Modξ(var) approach also enables the derivation of mode-specific empirical response spectra (ERS), providing critical insights into energy dissipation and modal contributions.