Protein-ligand binding affinity prediction is a crucial and challenging task in the field of drug discovery. However, traditional simulation-based computational approaches are often prohibitively time-consuming, limiting their practical utility. In this study, we introduce a novel deep learning method, CPIScore, which leverages the capabilities of Transformer and Graph Convolutional Networks (GCN) to enhance the prediction of protein-ligand binding affinity. CPIScore utilizes the Transformer architecture to capture comprehensive global contexts of protein and ligand sequences, while the GCN component effectively extracts local features from small molecular graphs. Our results demonstrate that CPIScore surpasses both traditional machine learning and other deep learning models in accuracy, achieving a Pearson's r of 0.74 on our test set. Furthermore, CPIScore has been validated across multiple targets, proving its ability to discern inhibitors from a diverse compound library with high enrichment rates. Notably, when applied to a generated focused library of compounds, CPIScore successfully identified six potent small-molecule inhibitors of ATR, which were tested experimentally and four small molecules exhibited inhibitory activity below ten nanomoles. These results highlight CPIScore's potential to significantly streamline and enhance the efficiency of drug discovery processes.