Reading list

Week 1 (Sep 6)

Introduction to the course. Class notes

Supplementary material

Stan Franklin and Art Graesser, Is it an Agent, or just a Program?: A Taxonomy for Autonomous Agents, Proc. 3rd Int'l Workshop on Agent Theories, Architectures, and Languages, published as Intelligent Agents III, Springer-Verlag, 1997, pp 21-35.
Chapter 2 of Russell and Norvig, "Artificial Intelligence. A modern approach" is devoted to agent architectures.

Week 2 (Sep 11-13)

Required readings on cooperation.

September 11
Jeffrey S. Rosenschein and Gilad Zlotkin, Designing Conventions for Automated Negotiation, AI Magazine, 15(3): Fall 1994, 29-46.
A classical paper that started research in how to design conventions to create desirable behaviors in agents. It also opened up the use of game theory in multi-agent systems.

September 13
MA Nowak, Five rules for the evolution of cooperation, Science 314:1560-1563, 2006.
A paper that tries to explain in terms of game theory how cooperation has evolved in biological systems.

Supplementary material (not required reading but useful to get a better understanding)

Vriend, Nicolaas (2000), "An Illustration of the Essential Difference Between Individual and Social Learning, and its Consequence for Computational Analyses" (pdf,245KB), Journal of Economic Dynamics and Control, Vol. 24, pp. 1-19. Focuses on the importance of the level of learning for computational agents. An agent is said to employ individual-level learning when it learns from its own past experiences, and to employ population-level learning when it learns from other agents, e.g., through mimicry of their observed behaviors.

Week 3 (Sep 18-20)

Required readings on on cooperation in large populations

September 18
Robert Axelrod and William D. Hamilton, The evolution of Cooperation. Science, vol 211, No. 4489. (Mar. 27, 1981), pp. 1390-1396.
Classical paper that introduces game theory in the study of evolution to explain cooperation. The paper talks about Prisoner Dilemma, the classical example in game theory to talk about cooperation and about strategies useful when playing a sequence of games with the same opponent.

September 20

Hauert C, A Traulsen, H Brandt, MA Nowak, K Sigmund, Via freedom to coercion: the emergence of costly punishment. Science 316: 1905-1907, 2007.
A paper that looks at the effect of punishment on societies. You need to read just the paper, but look at the appendix if you are interested in more details.

Supplementary material (not required but useful to get a better understanding )

The Axelrod's Iterated Prisoner Dilemma tournament. Software and information.
A paper describing the Southampton approach to the iterated prisoner dilemma tournament. Rogers, Alex, Dash, Rajdeep K., Ramchurn, Sarvapali D., Vytelingum, Perukrishnen and Jennings, N. R., Coordinating Team Players within a Noisy Iterated Prisoner’s Dilemma Tournament. Theoretical Computer Science, 377, (1-3), pp 243-259
for a short intro to game theory, read David K. Levine Game Theory or Game Theory: An Introductory Sketch
For a complete coverage of game theory read Essentials of Game Theory: A Concise Multidisciplinary Introduction by Kevin Leyton-Brown and Yoav Shoham, 2008. The book is available for free.

Week 4 (Sep 25-27)

Required readings on swarms

September 25
Eliseo Ferrante, Ali Emre Turgut, Edgar Duenez-Guzman, Marco Dorigo, and Tom Wenseleers, Evolution of Self-Organized Task Specialization in Robot Swarms, PLoS Comput Biol. 2015 Aug; 11(8): e1004273.

September 27
Melvin Gauci, Radhika Nagpal, and Michael Rubenstein. Programmable Self-Disassembly for Shape Formation in Large-Scale Robot Collectives, 13th Int'l Symposium on Distributed Autonomous Robotic Systems (DARS), 2016.

Supplementary material (not required reading but useful to expand the material):

Chih-Han Yu, Radhika Nagpal. Biologically-Inspired Control for Multi-Agent Self-Adaptive Tasks. Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 2010
Kirstin Petersen, Radhika Nagpal, Justin Werfel, TERMES: An Autonomous Robotic System for Three-Dimensional Collective Construction, Robotics: Science and Systems Conference, RSS. 2011
M. Dorigo & K. Socha, An Introduction to Ant Colony Optimization. T. F. Gonzalez (Ed.), Approximation Algorithms and Metaheuristics, CRC Press, 2007.
M. Dorigo, M. Birattari, T. Stützle, Ant Colony Optimization-- Artificial Ants as a Computational Intelligence Technique, IEEE Computational Intelligence Magazine, 2006.

Week 5 (Oct 2-4)

Required readings on game theory.

Monday October 2
Towards a Science of Security Games, Thanh H. Nguyen, Debarun Kar, Matthew Brown, Arunesh Sinha, Albert Xin Jiang, Milind Tambe, in New Frontiers of Multidisciplinary Research in STEAM-H (Book chapter) (edited by B Toni), 2016 .

Wednesday October 4
Game Theory-Based Opponent Modeling in Large Imperfect Information Games, by Sam Ganzfried and Tuomas Sandholm, AAMAS 2011.

Supplementary material (not required reading but useful to expand to other contributions):

Jain, Pita, Tsai, Kiekintveld, Rathi, Ordonez, and Tambe, Software assistants for patrol planning at LAX and Federal Air Marshals Service
Korzhyk, Yin, Kiekintveld, Conitzer, and Tambe, Stackelberg vs. Nash in security games: an extended investigation of interchangeability, equivalence, and uniqueness, JAIR, 41(2), pp 2970327, 2011.
Avrim Blum, Nika Haghtalab, and Ariel D. Procaccia, Learning to Play Stackelberg Security Games. Draft of Chapter 17 of the book Improving Homeland Security Decisions.
Tuomas Sandholm, The State of Solving Large Incomplete-Information Games and Application to Poker, AI Magazine, Winter 2010, pp 13-32.
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity by Pablo Hernandez-Leal, Michael Kaisers, Tim Baarslag, and Enrique Munoz de Cote. https://arxiv.org/abs/1707.09183

Week 6 (Oct 9-11)

Required readings

October 9
D. Bertsekas, Auction Algorithms, Encyclopedia of Optimization, Kluwer, 2001

October 11
Reserve Prices in Internet Advertising Auctions: A Field Experiment, Michael Ostrovsky and Michael Schwarz, working paper, 2016.

Supplementary material (not required reading but useful to expand to other contributions):

B. Edelman, M. Ostrovsky, and M. Schwartz, Internet advertising and the Generalized Second Price auction: Selling billions dollars worth of keywords, American Economic Review, 97(1), 2007.
Martin Pesendorfer, A Study of Collusion in First-Price Auctions, Review of Economic Studies, Volume 67, Number 3, pp 381-411, July 2000.
Paul Klemperer, How (Not) to Run Auctions: the European 3G Telecom Auctions, European Economic Review, 2002.

Week 7 (Oct 16-18)

Required readings on auctions

Monday October 16
S. Koenig, C. Tovey, M. Lagoudakis, V. Markakis, D. Kempe, P. Keskinocak, A. Kleywegt, A. Meyerson, S. Jain, The Power of Sequential Single-Item Auctions for Agent Coordination, Proc. of the AAAI Conference on Artificial Intelligence (AAAI), pp 1625-1629, 2006.
One of multiple papers by this group of authors on Sequential Single Item Auctions, a type of auction they have proposed which has desirable property such as low computational complexity and a bound on the quality of the solution.

Wednesday October 18

H.-L. Choi, L. Brunet, J. P. How, Consensus-Based Decentralized Auctions for Robust Task Allocation, IEEE Trans. on Robotics, Vol. 25, No. 4, pp. 912-926, August 2009.

Supplementary material (not required):

Week 8 (Oct 23-25)

Required readings on trust and negotiation

Monday October 23
Huynh, T.D., Jennings, N. R. and Shadbolt, N.R., An integrated trust and reputation model for open multi-agent systems, Autonomous Agents and Multi-Agent Systems, 13, (2), pp. 119-154, 2006.

Wednesday October 25
Negotiating Agents, by Catholijn M. Jonker, Koen V. Hindriks, Pascal Wiggers, and Joost Broekens, AI Magazine, 2012.

Week 9 (Oct 30 - Nov 1)

October 30
Ariel D. Procaccia, Cake Cutting Algorithms, Handbook of Computational Social Choice, (Brandt, Conitzer, Endriss, Lang, and Procaccia, eds.), chapter 13, 2016.

November 1
No Pizza for You: Value-based Plan Selection in BDI Agents, Stephen Cranefield, Michael Winikoff, Virginia Dignum, and Frank Dignum, IJCAI 2017.

Additional readings (not required but useful to get some background information): visit http://www.spliddit.org/, a site to test fair splitting on a variety of problems. Try it!

Week 10 (Nov 6-8)

Required readings on humans and robots/agents

Monday November 6
Localization and Navigation of the CoBots Over Long-term Deployments, Joydeep Biswas and Manuela Veloso. International Journal of Robotics Research, December 2013, 32(14), pp 1679-1694. Also available at http://repository.cmu.edu/compsci/2778.

Wednesday November 11
Sarvapali D. Ramchurn, Feng Wu, Wenchao Jiang, Human-agent collaboration for disaster response, Autonomous Agents and Multi-Agent Systems, January 2016, 30(1), pp 82-111.

Additional readings (not required but useful to get some background information):

Week 11 (Nov 13-15)

Required readings on agents and humans

Monday November 13
Can automated agents proficiently negotiate with humans? Raz Lin, Sarit Kraus Communications of the ACM, Volume 53 Issue 1, January 2010 Pages 78-88

Wednesday November 15
Computational rationality: A converging paradigm for intelligence in brains, minds, and machines, Samuel J. Gershman1, Eric J. Horvitz, Joshua B. Tenenbaum, Science 17 Jul 2015: Vol. 349, Issue 6245, pp. 273-278.

Additional readings (not required but useful for additional information):

Week 12 (Nov 20)

Required reading on deep reinforcement learning

Monday November 20
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller Playing Atari with Deep Reinforcement Learning, arXiv:1312.5602[cs.LG].

Week 13 (Nov 27-29)

Required reading on learning
Monday November 27
Cathy Wu, Aboudy Kreidieh, Eugene Vinitsky, and Alexandre M. Bayen, Emergent Behaviors in Mixed-Autonomy Traffic, 1st Annual Conference on Robot Learning (CoRL 2017).

November 29
Presentations of projects

Week 14 (Dec 4-6)

Presentation of projects

Week 15 (Dec 11-13)

Presentation of projects