Description
Winter Quarter 2015
Instructor: Professor Benjamin Van Roy
Mondays and Wednesdays at 2:15PM-3:45PM
Room 160-329
Starting Monday January 5
Reinforcement learning concerns how an agent can learn over time to make increasingly effective decisions when uncertain about its environment and how and when its actions bear consequences. This course will provide an introduction and prepare students for research at the forefront of this area.
Topics covered:
o optimistic exploration in single-period optimization
o regret analysis
o Markov decision processes
o real-time dynamic programming
o Q-learning
o optimistic exploration in Markov decision processes
o generalization in reinforcement learning
o synthesis of optimistic exploration and generalization
o policy gradient methods
o exploration beyond optimism
Students are expected to have some background in optimization (e.g., MS&E 310) and stochastic processes (e.g., MS&E 321).
See the class website for more detailed information.
General Information
Announcements
Everyone should have submitted a 1-page project proposal for the class, the deadline was February 16th so try to get these in ASAP... definitely by class tomorrow.
This doesn't need to be a lot of work, we just want to see that you've formed a group and have a reasonable target lined up for the project.
Check out the resources now:
https://piazza.com/stanford/winter2015/mse338/resources
We will move this over to the class website after.
Check the class website:
https://web.stanford.edu/class/msande338/assignments.html
Please come and talk to me today after class about your project plans/groups. Alternatively email myself or Ben if you need another time to discuss this.
The coding assignment HW1 is available now on the class website.
https://web.stanford.edu/class/msande338/docs/msande338_hw1.pdf
You should complete it according to the instructions.
Hand in at the beginning of class on February 9th.
Hi everyone,
Just a reminder that the class website exists and can be found:
https://web.stanford.edu/class/msande338/
In particular I wanted to draw attention to the lecture schedule.
Some people are asking me when/if there are lectures, so please consult the full schedule that is listed on this site.
Also (while we're at it) make sure that you take note of which dates you are supposed to be presenting.
Ian
The paper presentations have been assigned and are available to view at the google document:
| Stanford ID | Assignment |
| imanol | 4 |
| csimoiu | 9 |
| sragain | 8 |
| rikel | 7 |
| milind | 3 |
| ruif | 6 |
| zyin | 1 |
| mmongia | 10 |
| rahulmj | 2 |
https://docs.google.com/spreadsheets/d/1wu-UxZ3GCSGmHxzK2d4N90gadjWy9OBtvpA-rYk27fo/edit#gid=0
Each presentation should be 15 minutes long with a short following session of Q&A. The schedule for presenting is available on the class website. If you have any problems with these dates make sure to contact iosband@ and bvr@ as soon as possible.
Please sign up for the class presentations using this google sheet and not by emailing directly:
https://docs.google.com/spreadsheets/d/1wu-UxZ3GCSGmHxzK2d4N90gadjWy9OBtvpA-rYk27fo/edit#gid=0
A link to this sheet has also been posted on the class website.
Hello everyone,
We have now got the class website up and running at:
https://web.stanford.edu/class/msande338/
This will be the main place to look for class materials, papers and lecture notes.
We will continue to use Piazza as the main avenue for class announcements and Q&A. Please make sure that you are signed up to Piazza by next class.
https://piazza.com/class/i41gqk8yzoe5rg
Cheers,
Ian