This study investigated the effects of filling material roughness on the H-polarized scattering signatures of a two-dimensional (2D) rectangular groove embedded on an infinite ground plane. Under the assumption of a weakly rough surface on the groove, simplifying suppositions can be used to estimate Green’s functions inside and outside the groove. Enforcing continuity in the tangential magnetic field on the rough surface enables the construction and resolution of a Fredholm’s integral equation with logarithmic singularity. The results were verified via two full-wave electromagnetic field simulations carried out using the finite element method and the method of moments. The findings showed that even a weakly rough surface, such as a polished exterior with weak residual roughness, can considerably affect and change 2D scattering patterns.