A Unified Approach to Holomorphic Anomaly Equations and Quantum Spectral Curves