By resolving the energy of the incident X-ray photons, spectral X-ray imaging with photon counting detectors offers additional material-specific information compared to conventional X-ray imaging. This additional information can be used to improve clinical diagnosis for various applications. However, spectral imaging still faces several challenges. Amplified noise and a reduced signal-to-noise ratio on the decomposed basis material images remain a major problem, especially for low-dose applications. Furthermore, it is challenging to construct an accurate model of the spectral measurement acquisition process. In this paper, we present a novel algorithm for projection-based material decomposition. It uses an empirical polynomial model that is tuned by calibration measurements. We combine this method with a statistical model of the measured photon counts and a dictionary-based joint regularization approach. We focused on spectral coronary angiography as a potential clinical application of projection-based material decomposition with photon counting detectors. Numerical and real experiments show that spectral angiography with realistic dose levels and gadolinium contrast agent concentrations are feasible using the proposed decomposition algorithm and currently available photon-counting detector technology.