The nonlinear susceptibility in the terahertz region is expected to have a non-negligible imaginary part originating from the momentum-dependent scattering time of free carriers, but it has been scarcely reported. By utilizing an intense 4 THz beam from a terahertz free electron laser, we investigated the azimuth angle dependence of the third harmonic generation (THG) from semiconductors. The observed angular anisotropy of THG revealed the contribution of the imaginary part of the nonlinear susceptibility originating from the momentum-scattering time relation in addition to its real part originating from the band nonparabolicity. The results provide a deeper understanding of nonlinear optics in the terahertz region.