The appearance of objects depends on their material, shape, and on the illumination conditions. Conversely, object appearance provides us with cues about the illumination and the material. This so-called inverse problem is basically underdetermined and therefore we expect that material and illumination perception are confounded. To gain insight into the relevant mechanisms, we rendered a set of artificial spheres for vastly different canonical light fields and reflectance functions. We used four physics-based bidirectional reflectance distribution functions (BRDFs) representing glossy, pitted, velvety, and matte material. The six illumination conditions were collimated illumination from four directions, hemispherical diffuse illumination, and fully diffuse (Ganzfeld) illumination. In three sub-experiments we presented pairs of stimuli and asked human observers to judge whether the material was the same, whether the illumination was the same, and for a subset in which either the illumination or the material was the same to judge which of the two was constant. We found that observers made many errors in all sub-experiments. In experiment 2 the illumination direction was chosen at random. Using an interactive interface, we asked human observers to match the illumination direction of a sphere of one of the four materials with that of a Lambertian sphere. We found systematical material-dependent deviations from veridical performance. Theoretical analysis of the radiance patterns suggests that judgments were based mainly on the position of the shadow edge. In conclusion, we found no evidence for 'material constancy' for perception of smooth rendered spheres despite vast quantitative and qualitative differences in illumination and in BRDF between the stimuli. Although human observers demonstrated some 'illumination constancy', they made systematic errors depending on the material reflectance, suggesting that they used mainly the location of the shadow edge. Our results suggest that material perception and light-field perception are basically confounded.