Whitecap coverage, a direct indicator of surface wave breaking, plays a critical role in air–sea flux exchange and sea-salt aerosol production. Using synchronized shipborne video observations collected during three cruises of the R/V TAN KAH KEE in the South China Sea, we constructed a comprehensive dataset that pairs whitecap images with concurrent meteorological variables and sea surface temperature and salinity. On this basis, we developed WF-UNetViT, an AI-based vision model for whitecap segmentation, to reduce the susceptibility of conventional whitecap estimates to wave textures, sea-surface reflections, and varying illumination under complex sea states. Evaluated against DeepLabv3+, U-Net, and UNetViT, WF-UNetViT achieves superior performance across multiple metrics, demonstrating higher segmentation accuracy and greater robustness. Using the extracted whitecap coverage, we further analyzed its temporal variability and environmental controls. The coverage exhibits pronounced intermittency, fluctuating by several orders of magnitude, and is positively correlated with the 10 m wind speed. However, wind speed alone cannot fully account for this variability; environmental factors such as the air–sea humidity difference and sea surface temperature also exert notable modulating effects. Based on these relationships, we propose a new multivariate parameterization scheme for whitecap coverage, which more effectively characterizes whitecap coverage under given wind conditions by explicitly incorporating additional environmental influences.