Aquatic vegetation in the coastal zone attenuates wave energy and reduces the risk of coastal hazards, e.g., flooding. Besides the attenuation of sea-swell waves, vegetation may also affect infragravity-band (IG) waves and wave setup. To date, knowledge on the effect of vegetation on IG waves and wave setup is lacking, while they are potentially important parameters for coastal risk assessment. In this study, the storm impact model XBeach is extended with formulations for attenuation of sea-swell and IG waves, and wave setup effects in two modes: the sea-swell wave phase-resolving (nonhydrostatic) and the phase-averaged (surfbeat) mode. In surfbeat mode, a wave shape model is implemented to capture the effect of nonlinear wave-vegetation interaction processes on wave setup. Both modeling modes are verified using data from two flume experiments with mimic vegetation and show good skill in computing the sea-swell and IG wave transformation, and wave setup. In surfbeat mode, the wave setup prediction greatly improves when using the wave shape model, while in nonhydrostatic mode (nonlinear) intrawave effects are directly accounted for. Subsequently, the model is used for a range of coastal geomorphological configurations by varying bed slope and vegetation extent. The results indicate that the effect of wave-vegetation interaction on wave setup may be relevant for a range of typical coastal geomorphological configurations (e.g., relatively steep to gentle slope coasts fronted by vegetation).