A conceptual and practical introduction to modern machine learning

• machine learning as an engineering approach, how it relates to other approaches to solving problems or "artificial intelligence", including a historical overview of machine learning, and how machine learning relates to computational complexity and cryptography.
• core concepts in supervised learning, form a conceptual, theoretical and practical perspective: approximation, estimation, empirical and structural risk minimization, complexity control, and the relationship to parameter estimation, probabilistic and Bayesian approaches,   The role of statistics, information and computation in machine learning.
• methods and models for supervised learning: Decision Trees, Linear Prediction, Support Vector Machines (SVMs) and Kernel methods, L1-methods, Boosting*, Nearest Neighbor methods, Differentiable Learning, Feedforward Neural Networks (The Multilayer Perceptron), Modern Deep Learning Architectures*
• supervised learning in practice: training, testing, validation and cross-validation, significance and working with data.
• generative models and learning samplers*
• beyond supervised learning*: structured and sequence models, transfer and semi-supervised learning, active learning, exploration and reinforcement learning.

*topics mentioned but not covered in depth.

---------------

Class Schedule:

Lectures: Mondays and Wednesdays 3-4:20 pm in TTIC 530.

Tutorials: Thursdays 3-4:20 pm in TTIC 529 and Fridays 3-4:20pm in TTIC 530 (two identical tutorials each week, need to only attend one).

---------------

Instructor: Nati Srebro. Office hours Mondays and Wednesdays 4:30-5pm at TTIC 503.

TAs:

Alec Sun. Office hours Tuesdays 3:30-4:30pm at TTIC 4th floor commons

Nirmit Joshi. Office hours Wednesdays 5-6pm at TTIC 4th floor commons

Shuo Xie. Office hours Wednesdays 1:45-2:45pm at TTIC 4th floor commons

Email course staff at: introml-winter25-staff@ttic.edu

Whom should I contact and where should I ask questions?

Questions about the course material, including questions about class, the slides or other reference material --> ask on Canvas, in person during TA or instructor office hours, or during lectures or tutorials.  Not by email.  If you ask by email, we will refer you to Canvas instead of answering by email.

Clarifications and technical problems with the homework, or general questions about course logistics (not specific to you) --> ask on Canvas (preferred) or during TA office hours.  Please post on Canvas instead of emailing us since others might have also encountered the same problem.

Help with the homework --> seek help during TA office hours.  You may also consult with other students.

Questions and concerns about grading and homework submission, including mistakes in grading, late submissions, extensions and other special arrangements or concerns that are specific to you --> email us at introml-winter25-staff@ttic.edu. Please use this email address instead of emailing a staff person individually.

Questions about registration, consent, auditing, dropping, canvas access --> email us at introml-winter25-staff@ttic.edu.

General comments or feedback  --> email introml-winter25-staff@ttic.edu (preferred) or directly to the relevant staff. 

Personally sensitive issue --> you may contact any staff member you are comfortable contacting. 

Recommended textbook:

We will not be following any particular text, and there are many others you might find useful.  The approach taken in this course is most similar to the following textbook

Understanding Machine Learning: From Theory to Algorithms Links to an external site.

See here for additional resources.

Homework:

The course will include weekly assignments.  About half of the assignments will be programming and experimentation assignments and half will be written assignments. 

The assignments are a required and integral part of the course.  Some topics will be introduced primarily through assignments, and many exam questions will be heavily based on the assignments.  Like lectures and other course components, the assignments are for your learning benefit.  You may collaborate on them freely, use any tools you would like, and seek help from others, including the TAs and other students in the class.  But ultimately, it is your responsibility to understand the material covered through the homework.  

Late HW policy: If you have a reason that justifies late homework submission, you may give yourself an extension on a self-approval basis, at your discretion, up to 10 total late days over the quarter, spread in any way over the assignments (eg a long extension on one assignment, or many short extensions). Each calendar day after a HW deadline counts as a late day.  Assignments submitted after grading starts might not be comprehensively graded.  

Quizzes:

We will have short online weekly quizzes on canvas.  Quizzes will be available Wednesday night, due by Monday noon, and cover the lectures and tutorials, as well as homework due during the week they are released.  In doing the quiz, you may consult course material (anything posted or directly linked to from canvas) and your own notes, but you may not collaborate, seek help from others, or use other tools.  Late quizzes will not earn credit, but we will drop the lowest two of the nine weekly quizzes.  

Grading: 

50% final exam

20% weekly quizzes

20% homework, but each homework grade will be replaced with the final exam grade if the exam grade is higher.

10% or more for at least one of:

• meaningful and productive participation in the discussion forums, including asking questions
• answering questions in the discussion forums
• class participation
• solving challenge problems on the homework
• creative, elegant or nicely presented homework solutions
• performing well on Kaggle-like competitions
• correcting or identifying mistakes from class, tutorial or the homework

Grade and Pass/Fail policy: After completing the course and the final exam, students will be informed of their earned letter grade.  They could then choose to take a letter grade, pass/fail, or withdraw.

Prerequisites: 

1) All the following core undergraduate courses: Probability, Linear Algebra, Multi-variate Calculus, Introduction to Algorithms, and Introduction to Programming

AND

2) One of the following: TTIC 31150 Mathematical Toolkit (recommended), or 25300/35300 Mathematical Foundations of Machine Learning, or STAT 34300 Applied Linear Statistical Methods or CAAM 30900 Matrix Computation.  Instructor consent without these pre-requisites will also be granted to graduate students with equivalent background or background in machine learning, and to students with a strong math background.  TTIC PhD students who have not yet taken TTIC 31150 may also register for the class.

AND

3) Passing a short online entrance quiz, covering basic probability, linear algebra, calculating gradients, understanding algorithms and runtime analysis of algorithms. 

The entrance quiz:  The quiz does not require calculations (beyond simple arithmetic without needing a calculator), touches only on core concepts, does not require creativity or coming up with solution approaches, and should take under 40 minutes to complete.  If you are comfortable with the pre-perquisites, you should not need to study for the quiz.  If the quiz takes you over an hour to complete, you are struggling with some of the questions, or you need to study in order to answer the questions, then you should reconsider applying to take the class.

How to Register

To request to take the course you must complete an online form and quiz Links to an external site..

You may complete the form once, and you must take the quiz immediately after completing the first part of the request form.  Completing the form will take you 10-15 minutes and completing the quiz another hour---only click the link above once you are ready to complete the form and take the quiz.

• You must use a @uchicago.edu google account in order to access the form.  Please contact us if you are not a TTIC or UChicago student and do not have a @uchicago.edu google account.
• Anyone wishing to take the course, or participate in it in any way, must complete the form and quiz.  Even if you contacted us by email, you must still complete the form and include all relevant information in the form.  I generally discourage auditing vs registering for the course, but in any case you must first complete the form and take the quiz before we can discuss further.

Requests will be reviewed weekly and if you meet the pre-requisites and do well on the quiz, you will receive an invitation to register.  To ensure a response before the Friday deadline for pre-registration or adding a course, please submit your request by Tuesday of the same week.  Response to requests requiring special consideration might be delayed---we might not be able to make a final decision until the first week of the quarter, but will keep you updated.