The co-encapsulation of multiple bioactive components in a carrier may produce synergistic effects and improve health benefits. In this study, the interactions of β-lactoglobulin (β-LG) with epigallocatechin-3-gallate (EGCG) and/or piceatannol (PIC)/oxyresveratrol (OXY) were investigated by multispectroscopic techniques, isothermal titration calorimetry, and molecular docking. The static quenching mechanism of β-LG by EGCG, PIC and OXY was confirmed by fluorescence spectroscopy and UV-vis absorption difference spectroscopy. The binding sites of these three polyphenols in β-LG were identified by site marking fluorescence experiments and molecular docking. The thermodynamic parameters of the β-LG + EGCG/PIC/OXY binary complex and β-LG + EGCG + PIC/OXY ternary complex were obtained from fluorescence data and used to analyze the main driving force for complex formation. The exothermic binding process was further confirmed by isothermal titration calorimetry. The α-helical content, particle size and morphology of free and ligand-bound β-LG were determined by circular dichroism spectroscopy, dynamic light scattering and transmission electron microscopy, respectively. The effect of EGCG, PIC and OXY on the conformation of β-LG was studied by Fourier transform infrared spectroscopy. In addition, the maximum synergistic antioxidant activity between EGCG and PIC/OXY was obtained by response surface analysis. The effects of β-LG in the binary and ternary systems on the antioxidant activity, stability, solubility and cytotoxicity of the polyphenols were also studied. Finally, the different cytotoxicities of the complexes and nanoparticles of the binary and ternary systems were compared. The results of this study are expected to provide a theoretical basis for the development of β-LG-based carriers co-encapsulating a variety of bioactive components.