Abstract: At present, the book evaluation system is mostly based on external characteristics and lacks qualitative evaluation indicators, which makes it difficult to reflect the quality of books. This study attempts to introduce book review indicators that reveal the quality of books in the evaluation system, in order to make a useful exploration of the quality of Chinese books. Based on the high-frequency indicators of existing research, 16 secondary indicators are selected from traditional metrics, altmetrics, and book review indicators to construct an evaluation system consisting of three dimensions, namely, book utilization, book attention and book review. By using sentiment analysis to process book review data and by applying the CRITIC method to determine the weight of indicators, this paper makes an empirical analysis by taking psychology books as the sample. The reliability of evaluation system is analyzed by reliability and validity test. The empirical results show that the evaluation system thus constructed is reliable and can be used for comprehensive evaluation of psychology books; and that book review indexes can screen out books with high content quality and provide reference for bibliographic recommendations and other activities requring quality evaluation of books