Perceived object motion variance across optical contexts

Visual motion computation is challenging under real-world conditions due to continuous contextual changes such as varying lighting conditions and a large range of optical material properties. Due to these changes the retinal optical flow can drastically vary while the physical motion of an object remains constant. Especially materials with high reflective and refractive interactions can cause complex motion patterns. Here we investigate object motion constancy across various optical contexts and if the human visual system compensates for other causal sources in motion.
We performed two experiments. In the first experiment observers had to estimate which of two stimuli was rotating faster around the vertical axis. The stimuli were displayed for 500 ms in a 2-IFC staircase design. For the Match stimulus the illumination, material properties and shape were constant. The stimulus was rendered at a high temporal resolution allowing for small rotational speed changes for the staircase design. The Test stimuli varied in ten optical properties (e.g., matte, glossy, anisotropic, translucent), three illumination maps (sunny, cloudy, indoor), and three shapes (knot, cubic, blobby), the rotational speed remained constant. There were three different conditions in the second experiment: 1. unmasked Match and Test stimulus (same as experiment one); 2. masked Test stimulus (circular gaussian mask, masking outer shape contours); 3. masked Test stimulus and masked Match stimulus where the Match stimulus was replaced by horizontally moving 2D pink noise. In this experiment a subset of the optical conditions was used.
Expanding on our previously presented work [1], we applied three image-based motion capturing models (Figure 1) to gain deeper insights on motion cues that are predictive of human judgements. The models are Lucas-Kanade (optical flow), RAFT (optical flow DNN), FFV1MT (motion energy). First, we found that there are clear illusory differences of perceived rotational speed with even bigger effects when the circular mask was applied. The transparent material with the refractive index of water is systematically perceived to be rotating faster than other materials across all conditions. We performed an RSA (representational similarity analysis) to compare a range of different metrics across conditions and flow models. We find that the gradient of the optical flow is a particularly good predictor of human performance. The gradient emphasizes local speed changes in the optical flow, for example with moving highlights. Another observation is that Lucas-Kanade is most predictive of human performance under most conditions while RAFT is most stable across materials and closest to the ground truth. Our results further suggest that the human visual system does partially compensate for motion flow effects across optical contexts in object motion.
[1] Van Assen, J. J. R., Kawabe, T., & Nishida, S. Y. (2020). Object motion and flow variance across optical contexts. Journal of Vision, 20(11), 458-458.
This work has been supported by a Marie-Skłodowska-Curie Actions Individual Fellowship (H2020-MSCA-IF-2019-FLOW) and by JSPS Kakenhi JP20H05957.