Automatic pectoral muscle removal on medio-lateral oblique view of mammogram is an essential step for many mammographic processing algorithms. However, the wide variability in the position of the muscle contour, together with the similarity between in muscle and breast tissues makes the detection a difficult task. In this paper, we propose a two step procedure to detect the muscle contour. In a first step, the endpoints of the contour are predicted with a pair of support vector regression models; one model is trained to predict the intersection point of the contour with the top row while the other is designed for the prediction of the endpoint of the contour on the left column. Next, the muscle contour is computed as the shortest path between the two endpoints. A comprehensive comparison with manually-drawn contours reveals the strength of the proposed method.