Bishop: Pattern Recognition and Machine Learning
In my opinion the best introduction to modern machine learning methods. It explains ideas and algorithms very clearly and makes extensive use of probabilistic methods.
The book discusses some basic probabilistic methods and ideas that are needed and introduces some distributions that are useful. I feel it is not entirely self-contained when it comes to statistics and inference - but maybe that is not possible when one looks at the great range of algorithms that's covered here.
Many treatments include theoretical analysis and might be somewhat inaccessible at first but provide a lot of insight. This is definitely not a leisurely read (as Marslands book below) but rather a handbook for the researcher.
What I like in this book:
- Extensive use of Bayesian methods. After looking at all these algorithms from the Bayesian perspective, one really understands what Bayesian methods are all about.
- Graphical models. Message passing, exact inference, factor graphs. With proofs in many places.
- Mixture models, EM algorithm and Variational Inference. Very extensive examples and proofs. I implemented a variational Bayesian Gaussian mixture model using the description without much of a problem. And I understood what I was doing.
What I don't like about this book:
- No nonparametric methods
The Elements of Statistical Learning
An awesome book, written for people with a statistical background but very accessible if you don't have that (like me). As this is a classic, going into
its second edition and 5th printing, I decided going through it from cover to cover. It is really worth it. Also it is available online for free.
It contains a very comprehensive approach of many state of the art machine learning techniques, focusing on classification.
Richard Szeliski: Computer Vision: Algorithms and Applications
Unpublished and free.
On nearly 800 pages (not counting references) Szeliski gives a comprehensive account of classic computer vision techniques as well as summarizing recent developments.
This book is helpful as an introduction into the subject - or rather the subjects - of computer vision as well as an excellent pointer to recent works.
As so many applications are covered, most are not covered in a lot of depth. But the main ideas of different approaches are explained and with the provided pointers one can easily read up on the details.
Sebastian Nowozin, Christoph Lampert: Structured Learning and Prediction in Computer Vision
A great book covering many state of the art methods for structured problems in computer vision (and aren't they all?).
Covers basics of structured prediction, energy minimization, CRF and SSVM learning and inference. Many optimization and approximation methods for inference that are helpful in computer vision are covered. This book focuses more on application, no proofs are given. Instead, it contains many pointers to the current literature.