The transition mechanism and unsteady behavior behind a backward-facing step (BFS) in the supersonic regime at Ma=1.7 and Reδ0=13718 is investigated using large-eddy simulation (LES). The visualization of the flow field shows that the transition process behind the step is initiated by a Kelvin-Helmholtz (K-H) instability of the separated shear layer, followed by secondary modal instabilities of the K-H vortices, leading to Λ-shaped vortices, hair-pin vortices and finally to a fully turbulent state. The separation system features a broadband low-frequency dynamics in the range of fδ0/u∞=0.003∼0.20 as concluded from the spectral and statistical analysis. Dynamic mode decomposition suggests that the medium frequency motions centered around fδ0/u∞=0.06 are related to the interactions between reattaching and the shedding of large coherent shear vortices, while the lower (fδ0/u∞≈0.01) and higher (fδ0/u∞≈0.1) frequency unsteadiness are associated with the periodical expansion and shrinking of the separation system and the convection of upstream K-H vortices, respectively. All these three unsteady mechanisms are coupled to the laminar-to-turbulent transition process in the different stages.