In this paper, we propose a portmanteau test for whether a graph-structured network dataset without replicates exhibits autocorrelation across units connected by edges. Specifically, the well known Ljung-Box test for serial autocorrelation of time series data is generalized to the network setting using a specially derived central limit theorem for a weakly stationary random field. The asymptotic distribution of the test statistic under the null hypothesis of no autocorrelation is shown to be chi-squared, yielding a simple and easy-to-implement procedure for testing graph-structured autocorrelation, including spatial and spatial-temporal autocorrelation as special cases. Numerical simulations are carried out to demonstrate and confirm the derived asymptotic results. Convergence is found to occur quickly depending on the number of lags included in the test statistic, and a significant increase in statistical power is also observed relative to some recently proposed permutation tests. An example application is presented by fitting spatial autoregressive models to the distribution of COVID-19 cases across counties in New York state.