Complex transverse acoustic impedance of the superfluid $^3$He-B was measured at the frequencies of 10 to 80 MHz at 17.0 bar by a cw bridge method. The observed temperature dependence was well explained by the quasiclassical theory with random S-matrix model for a diffusive surface. The temperature dependence was influenced by pair breaking and by quasiparticle density of states at the surface, which was drastically modified from the bulk one by the presence of surface Andreev bound states.