We present a case of a pseudoaneurysm within the lumbar musculature. This occurred following a computed tomography (CT)-guided vertebral biopsy in a 79-year-old male patient and was successfully treated with percutaneous ultrasound-guided thrombin injection. The patient initially presented with severe back pain. The plain radiographs and Magnetic resonance imaging (MRI) showed destruction of the L1 and L2 end plates, with marked narrowing of the disc space, suggestive of infective discitis. CT-guided biopsy was performed by the right paravertebral approach. Methicillin-resistant Staphylococcus aureus (MRSA) was grown, following culture of the specimen. Twenty days later, the patient developed a palpable swelling in the right lumbar region, with worsening of the back pain. MRI and ultrasound imaging showed a 3 cm pseudoaneurysm within the right lumbar musculature. The pseudoaneurysm was successfully treated following percutaneous ultrasound-guided injection of thrombin.