[D] Short survey of optimization methods

Submitted by medwatt t3_114f3p1 in MachineLearning

I have been trying to familiarize myself with the common techniques used in optimization theory so that I can follow some of the proofs I see in machine learning papers. I know that two of the goto books in this field are Boyd's and Bertsekas's books. However, these books require a significant amount of effort as they aim to teach you the finer details. Since my goal is to familiarize with the methods (and not go into the nitty-gritty details), I was wondering if there's a short book (say less than 100 pages) or some other resource whose goal is to provide the reader with a high level view of the field of the methods and techniques used in optimization theory. Is there such a book, lecture notes, video series, etc., that caters to such requirements?

13

Comments

You must log in or register to comment.

Academic-Poetry t1_j8x0owj wrote

Algorithms for Optimization by Mykel J. Kochenderfer and Tim A. Wheeler

Accessible introduction into a variety of methods, with code examples in Julia.

12

Dry_Obligation_8120 t1_j91vmb0 wrote

That book actually looks amazing. Nice visualizations, code in Julia to implement the algorithms and the exercises have solutions at the end. To top this off, its available for free to download.

I am impressed, thank you so much for the suggestion!

2

medwatt OP t1_j8xlq1b wrote

Thanks for the recommendation, but that's a very long book.

−1

TeamRocketsSecretary t1_j8ywdem wrote

Lol dude you wanna learn optimization, the details and length are what make the subject. If you want a high level overview look at a blog post. At the very least find an online offering of an optimization course with lecture videos and watch those and read the slides if you can’t be bothered to open a textbook.

All these low effort posts in this sub about people just looking to cut corners are depressing.

3

medwatt OP t1_j8yxl3j wrote

I'm neither a mathematician nor a computer scientist by trade. I don't have the need nor the time to go through the nitty gritty details of optimization theory. All I need is an overview of the main ideas in this field. Think of it like knowing how to use a limit without the need to go through the epsilon-delta definition. Hope my reply didn't offend your ego.

−1

TeamRocketsSecretary t1_j8z7pqs wrote

Given your reply I’m unsure of why you would want be able to follow the proofs then?

Some of the proofs in optimization are particularly rough so if you want to understand them the only way to is to wade through a book or the very least online lecture videos + slides.

7

Ulfgardleo t1_j92cwoy wrote

optimisation is a the worst field to skip the nitty gritty details. Optimisation is all about details.

​

your question is unspecified. "optimisation" in ML is a very different beast than optimisation in the math sense.

5