In order to enhance the spectrum utilization, this paper uses the nonlinear dynamics theory for modeling and prediction of spectrum state duration. Firstly, the real spectrum state duration is investigated. Then, this study uses the directional derivative to accomplish the state-space reconstruction of the spectrum time series with the non-uniform time delays. Finally, the Scale-Dependent Lyapunov Exponent (SDLE) is used to determine the characteristics of chaos. Based on the Davidon-Fletcher-Powell-based Second Order of Volterra Filter (DFPSOVF) method, a novel Volterra model with adaptive coefficient adjusting using Limited storage Broyden-Fletcher- Goldfarb-Shanno quasi-Newton (L-BFGS) method is proposed. Furthermore, the proposed model is applied to predict the short-term spectrum with chaotic characteristics. To reduce the complexity of this new model, the useless filter coefficients are eliminated adaptively. The numerical simulations show that the new algorithm can reduce the complexity and guarantee prediction accuracy.