Recently, thin films of yttrium oxy-hydride (YOxHy) were reported to show an unusual color-neutral photochromic effect promising for application in smart windows. Our present work demonstrates that also oxy-hydrides based on Gd, Dy, and Er have photochromic properties and crystal structures similar to YOxHy. Compared to YOxHy, the optical bandgaps of the lanthanide based oxy-hydrides are smaller while photochromic contrast and kinetics show large variation among different cations. Based on these findings, we propose that cation alloying is a viable pathway to tailor the photochromic properties of oxy-hydride materials. Furthermore, we predict that the oxy-hydrides of the other lanthanides are also potentially photochromic.