A new Inverse Synthetic Aperture Radar (ISAR) target recognition method with the fusion of Gabor magnitude and phase feature is proposed. Firstly, the corresponding Gabor Magnitude Maps (GMMs) and Gabor phase information are obtained by convolving the ISAR image with multi-scale and multi-orientation Gabor filters. Secondly, each GMM is divided into several non-overlapping rectangular units, and the histogram of unit is computed and combined as the magnitude histogram feature. Thirdly, the local Gabor phase pattern is obtained by combining quadrant bit coding with local XOR pattern, and the block histogram feature is extracted from the local Gabor phase pattern. Then, the fusion of the Gabor magnitude and phase feature is used as the feature of ISAR image. Finally, five-type aircraft models are classified by using a nearest neighbor classifier with2 as a dissimilarity measure in the computed feature space. The recognition method is tested on ISAR data simulated from Greco electromagnetic soft ware. Compared with other recognition methods, the numerical results show that the proposed method is effective and has higher recognition performance.