The shear response of soft solids can be modeled with linear elasticity, provided the forcing is slow and weak. Both of these approximations must break down when the material loses rigidity, such as in foams and emulsions at their (un)jamming point-suggesting that the window of linear elastic response near jamming is exceedingly narrow. Yet precisely when and how this breakdown occurs remains unclear. To answer these questions, we perform computer simulations of stress relaxation and shear start-up tests in athermal soft sphere packings, the canonical model for jamming. By systematically varying the strain amplitude, strain rate, distance to jamming, and system size, we identify characteristic strain and time scales that quantify how and when the window of linear elasticity closes, and relate these scales to changes in the microscopic contact network.