Statistics Book Reviews - Best Textbooks for Stats

Statistics textbooks are usually divided into two levels: introductory books and mathematical statistics books (usually containing phrases like "mathematical statistics" or "statistical inference" in the title). If you're not sure whether you're looking for an introductory statistics book, this section explains.

Intro stats books do not require knowledge of calculus. The math required for them is minimal--basic arithmetic and a little bit of algebra is necessary for calculations, but no more.

Mathematical statistics books go more into the theory of the mathematics behind statistics. These books typically require a strong mathematical background, including calculus, set theory, and probability theory. Some of these books may require familarity with logic and proof. These books are quite diverse in difficulty level, but generally require calculus and good general math skills. Other books require extensive advanced topics like advanced calculus, measure theory, or even topology.

If you want to locate some books to master these subjects, check out my book reviews in calculus or probability theory.

With the exception of the two supplemental books, this page is arranged roughly in the order of level of mathematical sophistication required...the more basic books are shown first.

If you are not a math person, but are intelligent and a deep thinker, and want to develop a fairly thorough understanding of statistics, I would point to this book, Statistics in Plain English. It does not water down ideas, and emphasizes deep thinking

Pros: I think this book would make a good textbook for an introductory statistics book for a stat course that does not require prior math background. It would also be useful for self-study. This book is probably all one would need to obtain a working knowledge of statistical concepts, to where one could understand most basic statistical information and handle a few basic calculations.

Cons: Verbose. Strongly math-oriented people may want a more advanced text, and people who are slow readers may find the volume of writing a bit intimidating, especially if they have to work through this book at the pace of a course. Like most intro textbooks, this book does not cover much probability theory and does not provide a rigorous mathematical foundation for the subject.

Freedman is one of my favorite statistics authors. This book is very non-math oriented, and a bit goofy and off the wall. I recommend this but with a word of caution: this is a "love it or hate it" textbook that evokes strong opinions on both sides.

Pros: Highly accessible to people without a statistics background and with a weak math background. Funny and interesting. Helps develop accurate, rigorous thinking and sound reasoning without the need for rigorous mathematics.

Cons: As a book that is very wordy and humor oriented, this book tends to reach some students better than others. A common complaint I hear is that it is annoying and the authors don't get to the point soon enough. People who like math may find this book frustrating in how few equations it presents.

Rice's book Mathematical Statistics and Data Analysis is a book I would choose for an introduction to mathematical statistics for a student with a strong background in calculus.

Pros: Integrates data analysis in with presentation of theory, making the text ultimately more useful for preparing people for real world applications. Rich explanation of set theory, sample spaces, and basic probability theory, making this book self-contained and allowing it to be accessible to someone without the strongest mathematics background. Easy to skip around in. Includes some philosophical discussion, and explanation of where a lot of the mathematical structures and probability distributions come from.

Cons: Parts of this book would likely be tough to understand for people without a strong background in calculus. I also think the book might seem to go too fast for someone without at least some familiarity with basic probability theory.

Larry Wasserman's All of Statistics is subtitled A Concise Course in Statistical Inference, and I think it is just that. It is a mathy book, starting with abstract notions like sample spaces and events. If you're comfortable with set theory, summation notation, and have taken a course in calculus, this book will probably seem pretty accessible; otherwise it might be over your head.

Pros: Very concise, clear writing. Does a remarkable job of painting a complete picture of the field of statistics, covering topics that are often omitted, including nonparametric inference, Bayesian inference, decision theory, and even causal inference, as well as the usual topics like point estimation, hypothesis testing, and regression. Its combination of clarity and minimal presentation makes it an outstanding reference.

Cons: Omits proofs, extended explanations, and is sparse on examples. Mathematical notation can be dense at times. For self-study or a course textbook, I think this book is best supplemented by additional texts.

This book is widely used, but it is not one of my favorites. It was used in an introductory graduate-level course I took, and I thought it was a poor choice for that course. It does have some strengths, so I thought it important to review. I would exercise caution before using this book as a course textbook, however. Please read my comments below about the book's weaknesses before using it!

Pros: Excellent exposition of probability theory at the beginning. Clean and thorough use of mathematics throughout.

Cons: Completely theoretical, and divorced from applications: very little data is presented in the book and exercises, and there is very little discussion of applications (even the chapters on ANOVA and regression are highly theoretical) or real-world considerations. Barely mentions the Bayesian paradigm. Distributions presented without any discussion of how or why these distributions arise in the real world (i.e. no connection at all is made to the theory of stochastic processes). Very little discussion or explanation is given of the meaning or motivation behind the various equations. Both text and exercises heavily emphasize symbolic manipulation, without developing intuition. Exercises are very tough, but can usually be solved by mechanical / brute force work, without developing deeper understanding.

I used this as a supplementary reference and learning book when I was learning statistics--it was not used in one of my classes but I've read it in depth. This book presents a very classical view of the subject of Mathematical Statistics. I would recommend a prior course in probability, a strong background in calculus, and substantial exposure to abstract math, before tackling this book.

Pros: A clearly written book, accessible to a person with sufficient math background. Unlike many books, I find this book retains its clarity as it gets into later, more advanced chapters.

Cons: Old-fashioned in its choice of topics and presentation style. Does not adequately discuss Bayesian statistics.

When you are learning a subject for the first time, it is more important that the exposition of the material that you are reading is clear and thorough. To people who have not seen material before, a wordier explanation is often better. Also, if you are working the whole way through a book, it is often not a problem if one chapter depends on the previous ones in a logical way.

For a reference, however, it is more important that you are able to look up a given topic, such as a techniques, a probability distribution, or terminology, in a concise and self-contained manner.

This is why I prefer texts like the Wasserman as a quick reference for the shelf, and books like the Rice for initial learning.

This book on information theory and inference is highly atypical, as much among books about information theory as about books on statistical inference. It is quite unique in that it integrates the subjects, and presents both of them in a way that is quite different from any other presentation I've encountered.

Pros: Outstanding for self-study; written primarily for self-study. Very fun presentations. Rich, thorough explanations, but with ample challenge. Communicates outside-the-box thinking, nuanced thinking, and does an outstanding job of teaching intuition and rigorous reasoning in a casual framework. Contains applications to coding theory and neural networks, and integrates the Bayesian paradigm throughout the text.

Cons: I find this book follows a pretty linear course and is hard to jump around in. In spite of the clarity of explanations, this book covers tough material and presents a novel perspective, and thus can be deceptively challenging to work through. I find that it also increases in difficulty level as one works through it, which may be frustrating for some students.

This is one of my favorite books on statistics, and one that I wish every statistician would read. It is deep and philosophical, and I found it to be not only very easy to understand, but highly thought-provoking.

Pros: A terrible pun, but the best part of this book is its prose. This book is always philosophical and deep, and I find it is a book that can change the way you think (and much for the better). It gets at the question of subjectivity vs. objectivity. The book makes sparse use of mathematical equations, which will please people like me who love working with data and love reading and writing, but tire of working with long, tedious manipulations of symbols. I find this book to be extraordinarily accessible. Although the book is broken into sections, with Bayesian

Cons: Requires a very solid background in calculus and some prior exposure to probability theory to work through; probably best for students who have also already studied some mathematical statistics. I also found that the material on Bayesian Analysis was at times a tiny bit more mathematically tedious than the material on decision theory, but this is a very minor complaint--this book has a much more clean and accessible presentation of this material than other books.