Recent theory has predicted large temperature differences between the in-plane [longitudinal (LA) and transverse (TA)] and out-of-plane [flexural (ZA)] acoustic phonon baths in locally heated suspended graphene. To verify these predictions, and their implications for understanding the nonequilibrium thermodynamics of two-dimensional (2D) materials, experimental techniques are needed. Here, we present a method to determine the acoustic phonon bath temperatures from the frequency-dependent mechanical response of suspended graphene to a power-modulated laser. The mechanical motion reveals two counteracting contributions to the thermal expansion force, that are attributed to fast positive thermal expansion by the in-plane phonons and slower negative thermal expansion by the out-of-plane phonons. The magnitude of the two forces reveals that the in-plane and flexural acoustic phonons are at very different temperatures in the steady state, with typically observed values of the ratio ΔTLA+TA/ΔTZA between 0.2 and 3.7. These deviations from the generally used local thermal equilibrium assumption (ΔTLA+TA=ΔTZA) can affect the experimental analysis of the thermal properties of 2D materials.