The unique and unanticipated properties of multiple principal component alloys have reinvigorated the field of alloy design and drawn strong interest across scientific disciplines. The vast compositional parameter space makes these alloys a unique area of exploration by means of computational design. However, as of now a method to compute efficiently, yet with high accuracy the thermodynamic properties of such alloys has been missing. One of the underlying reasons is the lack of accurate and efficient approaches to compute vibrational free energies—including anharmonicity—for these chemically complex multicomponent alloys. In this work, a density-functional-theory based approach to overcome this issue is developed based on a combination of thermodynamic integration and a machine-learning potential. We demonstrate the performance of the approach by computing the anharmonic free energy of the prototypical five-component VNbMoTaW refractory high entropy alloy.