AIL 722: Reinforcement Learning (Fall 2024)

Sequential decision making under uncertainty is a key challenge for AI systems. Reinforcement Learning is a paradigm where an agent learns to make decisions by interacting with an environment. The agent takes actions in various states of the environment and receives rewards based on the outcomes of these actions. This approach has been used in diverse areas such as autonomous systems, game playing, robotics, and heathcare. This course will cover the basics of reinforcement learning as well as deep reinforcement learning — an exciting new area that combines deep learning techniques with reinforcement learning.

The course objectives are:

1. The course intends to familiarise students with the key features of reinforcement learning
2. Students will learn about fundamental RL algorithms and implement them in code
3. Given an application problem, students should be able to decide whether it should be formulated as an RL problem. If yes, they should be able to formulate the problem, decide which algorithm to use, and implement the algorithm.

• Slot: D
• Class Timings: Tue/Wed/Fri 9-10 am
• Venue: Bharti IIA Room 305


• Office hours
• Timing: Wednesdays, 4.30-5.30 pm
• Venue: 513-D, 5th Floor, Building 99-C

We will use piazza as the forum for students to ask questions about both the course material as well as logistics.
Piazza page

• Instructor: Raunak Bhattacharyya, Assistant Professor, Yardi School of Artificial Intelligence
• Teaching Assistants: Sanket Gandhi and Vaibhav Bihani, Ph.D. Students, Yardi School of Artificial Intelligence

Below is a tentative list of topics:

1. Markov Decision Processes
2. Value function based approaches
3. Approximations
4. Policy gradients
5. Actor critic methods
6. Exploration vs. exploitation

• [11^th August]
  
  Assignment 1 released (deadline: Sunday, Aug 25, 11:55 pm)
  [Problem Statement & Discussion Forum]
  [Overview Presentation]

• [25^th September]
  
  Assignment 2 released (deadline: Thursday, Oct 17, 11:55 pm)
  [Problem Statement & Discussion Forum]
  [Overview Presentation]

• [23^th October]
  
  Assignment 3 released (deadline: Sunday, Nov 24, 11:55 pm)
  [Problem Statement & Discussion Forum]

The weekly schedule will be updated with lecture slides and pointers to additional references as we progress through the course.

Grading policy:

1. Assignments: 40%
2. Exams: 50%
3. Paper presentation: 10%
4. Audit policy: Minimum 30% total score over all exams and minimum 30% total score over all assignments

Below are the guidelines for assignment submissions:

1. Collaboration: You are free to discuss the problems with other students in the class. However, the final solution/code that you produce should come through your individual efforts.
2. Submission Platform: The required code should be submitted using the Moodle Page.
3. Honor Code: Any cases of copying will result in a zero on the assignment. Additional penalties will be imposed based on the severity of the copying. Any copying cases run the risk of being escalated to the Department/DISCO.
4. Use of Code Generated from LLMs: Code generated from Language Models (LLMs) will be considered cheating. Any such code found in your submission will be treated as a violation of the honor code.
5. Copying Code from Online Resources: Copying code from any online resource is also considered copying. Ensure that all code submitted is your original work.
6. Late Policy: You are allowed a total of 5 buffer days (combined) for all programming assignments. There is no penalty if your submission stays within the limit of the 5 buffer days (total). For each additional day beyond the allowed 5 buffer days, you will lose 10% of the score for every late day in submission.