The colour of the ground layers of a painting has an influence on its visual appearance. In addition to the commonly used white ground layers, other colour ground layers have been used, for example, the grey ground layer used in Peter Paul Rubens’s painting Portrait of Clara Serena Rubens helps the colour transition of the skin tones. Understanding the effects caused by the colours of the ground layers is of significance for both technical art history and conservation. Optical non-destructive testing (NDT) techniques are useful tools for the investigation of paintings, for example, optical coherence tomography (OCT) can be used to study the surface and subsurface layers non-destructively. In this work, the interaction of light with paint and ground layers is modelled to supplement OCT measurements of paintings with ground layers. A previously described near-infrared light range OCT system provides high spatial and depth resolution measurements. A four-flux model has been developed for analysing the light interaction in the paint and ground layers. This model considers forwards-propagating collimated light, backwards-propagating collimated light, forwards-propagating diffuse light and backwards-propagating diffuse light. The model uses the optical material properties, including refractive index (RI), absorption and layer thickness, as input. This paper describes the construction of the model and an evaluation of its performance by comparison with OCT data.