The information of the significant wave height is needed in the wind speed inversion from the sea echo of High Frequency Surface Wave Radar (HFSWR) by using the empirical model of wind waves. Therefore, the accuracy of the significant wave height has an effect on the wind speed inversion. Based on the empirical model of wind waves, a wind speed inversion empirical model for wind retrieval without the information of wave height is developed, which uses the relationship between the wind speed and ratio of second-order to first-order spectrum energies. The inversion model is applied to the wind speed extraction. And the data obtained from two radars with different frequency in different detecting area are analyzed. The results show that the proposed model can be used to extract the wind speed from the HFSWR sea echo and the results of which three-parameter model are better than the two-parameter model.