CSci 8363 -- Fall 2017 -- Semester Plan
Plan for the semester and initial list of proposed papers.
General Information
- CSci 8363 - Mon Wed 4-5:15 in AkerH 227. (no UNITE).
Back to Class Web Page
- Some outside references may work only from "umn.edu" hosts.
Authentication or a VPN connection might be necessary to access them from
off-campus.
2017 PLAN
- Students are to present [at least] one research
or tutorial papers during the course of the
semester, by rotation.
Talks should highlight the main points, and summarize the theoretical and
experimental results present in the paper being presented. For very
theoretically technical papers, you should at least be able to explain what
the main results are and what they mean, even if the detailed derivations are
too complex for a short presentation.
Long papers may be split up into parts presented separately (e.g.,
basic results/algorithms and examples/applications).
- The papers listed here are a mix of historical foundational papers, recent
papers showing variations on the original ideas or how the abstract methods
have been applied to specific applications. Students should either select
from among these papers, or may select another similar relevant paper after
checking with first with the instructor.
-
Submit a weekly synopsis of each week's material, with your own reactions.
This should be limited to a paragraph or two pointing out what the main
take-away message you got out of each lecture.
- Also submit as separate item some comments on how the lecture
was presented, to be passed on to the speaker (without attribution).
- Develop and carry out a research project based on one or more recent
research papers devoted to topics studied in this class.
A research project can be a literature survey, an experimental study of some
methods proposed in a paper or of an application of one of the methods studied
in this class. To give an approximate scale of the effort required, you
should expect to devote about 50 hours of time during the course of the
semester. In about 3 weeks, you should submit a 1 page description of
your proposed project.
- Write a 10-15 page report on your research project at a level that would
be appropriate for publication in a workshop or conference.
- Give a short presentation on your project during the last 2 weeks of the
semester.
Your project will count toward the Project Requirements for a Plan C MS degree
in Computer Science.
- All submissions should be via
Moodle.
- The final grade will be derived from
- Presentation of Research Paper in class [c. 40%]
- Synopses submitted weekly [c. 10%]
- Project (including the proposal, the short presentation, and mainly the written report) [c. 50%]
Back to Class Web Page
Week 1 -- Intro: Basics of Eigenvalues, PCA definition
-
Intro - Examples.
classintro.pdf - Class Introduction (Boley)
- === alternate paper
A tutorial on Principal Components Analysis.
Lindsey I Smith
principal_components.pdf
-
REF ONLY Boley: Review of Linear Algebra
intro.htm - Introductory/Review slides (REF ONLY)
- === alternate paper
PCA by example, PCA<->SVD
PCA.pdf.
Week 2 -- Latent Semantic Indexing -- Text Mining
-
=== M 09/11 Yiyi Yin
Computational Methods for Intelligent Information Access. ;
by: M.W. Berry, S.T. Dumais, and T.A. Letsche. ; in: Proc. Supercomputing'95, San Diego, CA, December 1995. ;
note: for a talk, most useful to start with example.
http://dl.acm.org/citation.cfm?id=285569 ;
-
=== W 09/13 Yuchen Luo
Application of Dimensionality Reduction in Recommender System - A Case Study
Sarwar, Badrul ; Karypis, George ; Konstan, Joseph ; Riedl, John
UMN Computer Science Tech Report TR 00-043.
note: for a talk, most useful to start with example.
http://www.dtic.mil/docs/citations/ADA439541
Week 3 -- Tensor Decompositions
Week 4 -- Dimensionality Reduction -- Feature extraction
Week 5 -- Kernal Methods
Week 6a -- Dimensionality Reduction
Week 6b -- Multidimensional Scaling and non-linear dimensionality reduction
Week 7 -- Methods related to Spectral Graph Partitioning
Week 8 -- Importance Ranking in Graphs and Link Analysis
Week 9 -- Bounds on eigenvalues related to how well a graph can be cut
Week 10 -- Centrality Measures
Week 11 -- Convolutional Neural Network + Latent Variable Methods
-
=== M 11/13 Cameron Fabbri
Gradient-Based Learning APplied to Document Recognition
by: Yan LeCun, Leon Bottou Yoshua Bengio, Patrick Haffner
Proc. IEEE, November 1998.
http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf
ImageNet Classification with Deep Convolutional Neural Networks
by: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
NIPS 2012.
https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks
tutorial on CNNs: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/
tutoral on NN+ back-prop: https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/
-
=== W 11/15 Mengdie Wang
Latent Dirichlet Allocation.;
by: David M. Blei, Andrew Y. Ng, Michael I. Jordan. ; in: J. of Machine Learning Research 3:993-1022, 2003 ;
note: foundational paper on LDA.
http://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf;
- === alternate paper
Fast matrix computations for pairwise and columnwise commute times and Katz scores ;
by: F Bonchi, P Esfandiar, DF Gleich, C Greif
http://arxiv.org/pdf/1104.3791 ;
note: Katz scores using Lanczos and theory of moments. (refers to Fouss)
local link: FastKatz.pdf
http://arxiv.org/pdf/1104.3791 ;
=== alternate paper (or different date)
Week 12 -- Convolutional Neural Nets
Week 13 -- CNN (cont) + Graph based analysis
-
=== M 11/27 Yuting Sun
Convolutional Neural Networks on Graphs
with Fast Localized Spectral Filtering
Michaël Defferrard and Xavier Bresson and Pierre Vandergheynst
https://arxiv.org/abs/1606.09375
- === alternate paper
=== W 11/29 open
open (possible project presentation)
-
-
-
- === alternate paper
Sparse autoencoder
A Ng - CS294A Lecture notes, 2011
https://pdfs.semanticscholar.org/eb2f/e260af00818907fe82024203d8a5a1386777.pdf
- === alternate paper
An Autoencoder Approach to Learning Bilingual Word Representations,
by: Chandar A P, Sarath and Lauly, Stanislas and Larochelle, Hugo and Khapra, Mitesh and
Ravindran, Balaraman and Raykar, Vikas C and Saha, Amrita,
NIPS 27,
pp. 1853--1861,
2014.
http://papers.nips.cc/paper/5270-an-autoencoder-approach-to-learning-bilingual-word-representations.pdf
- === alternate paper
General Tensor Spectral Co-clustering for Higher-Order Data,
by: Wu, Tao and Benson, Austin R and Gleich, David F,
NIPS 29,
pp. 2559--2567,
2016.
http://papers.nips.cc/paper/6376-general-tensor-spectral-co-clustering-for-higher-order-data.pdf
- === alternate paper
Single Pass PCA of Matrix Products,
by: Wu, Shanshan and Bhojanapalli, Srinadh and Sanghavi, Sujay and Dimakis, Alexandros G,
NIPS 29,
pp. 2585--2593,
2016
http://papers.nips.cc/paper/6075-single-pass-pca-of-matrix-products.pdf
- === alternate paper
Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates,
by: Li, Yuanzhi and Liang, Yingyu and Risteski, Andrej,
NIPS 29,
pp. 4987--4995,
2016.
http://papers.nips.cc/paper/6417-recovery-guarantee-of-non-negative-matrix-factorization-via-alternating-updates.pdf
- === alternate paper
Personalized Hitting Time for Informative Trust Mechanisms Despite Sybils
by: Brandon Liu and David Parkes and Sven Seuken,
AAMAS,
2016,
http://dl.acm.org/citation.cfm?id=2937088
- === alternate paper
Sparse Representation for Computer Vision and Pattern Recognition. ;
by: J Wright ; Yi Ma ; J Mairal. ; G Sapiro ; T S Huang ; Shuicheng Yan ; in: Proc IEEE 98#6, 2010. ;
note: general intro: like a magazine art but more technical
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5456194&tag=1 ;
Week 14 -- Student Project Presentations
The following sign-ups are tentative.
- Projects:
=== M 12/04
- Rui Ma
- Mitchell Kinney
- Cameron Fabbri
- Yingxue Zhou
- Projects:
=== W 12/06
- Aditya Pakki
- Xiaochen Zhang
- Wenjun Lang
- Qun Su
Week 15 -- Student Project Presentations
- Projects:
=== M 12/11
- Mengqun Li
- Chaoran Chen
- Yue Bi
- Yuting Sun
- Projects:
=== W 12/13
- Yuchen Luo
- Yunpeng Shi
- Yiyi Yin
- Zhuliu Li
Alternate Papers
Large Text Document Collection
Information Theory
- === alternate paper
Information and Entropy;
by: Dan Boley
note: local notes on information theory and entropy.
Kernel Methods
Nonlinear Dimensionality Reduction
Graph Analysis
- === alternate paper
Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation ;
by: F. Fouss, A. Pirotte J.-M. Renders, and M. Saerens ; in: ;
note: follows title
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4072747&tag=1 ;
- === alternate paper
Lower Bounds for the Partitioning of Graphs ;
by: W E Donath and A J Hoffman. ; in: IBM J Res and Dev.1973. ;
note: short but dense.
http://ieeexplore.ieee.org/iel5/5288520/5391360/05391366.pdf?arnumber=5391366 ;
- === alternate paper
Expanders, Tropical Semi-rings, and Nuclear Norms. Oh My! ;
by: David Gleich ; in: The ACM Magazine for Students, 2013. DOI: 10.1145/2425676.2425688 XRDS * spring 2013 * Vol.19 * No.3 ;
note: magazine-style article. needs supplemental material
http://dl.acm.org/citation.cfm?id=2425688
https://www.cs.purdue.edu/homes/dgleich/publications/Gleich;
- === alternate paper
Learning from Labeled and Unlabeled Data on a Directed Graph. ;
by: D Zhou, J Huang, B Schoelkopf. ; in: ICML 2005. ;
note: have a partially labelled graph induce labels on remaining nodes. like symmetrizing a graph.
http://research.microsoft.com/en-us/um/people/denzho/papers/llud.pdf ;
- === alternate paper
Consensus Problems in Networks of Agents with Switching Topology and Time-Delays;
by: Reza Olfati-Saber and Richard M. Murray;
in: IEEE Trans AC:49, NO. 9, 1520:1533, 2004;
note: example of consensus computations;
tac04_ros_rmm.pdf
http://www.seas.ucla.edu/coopcontrol/papers/om03-tac.pdf.
- === alternate paper
Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems
by: Daniel A. Spielman, Shang-Hua Teng;
in: arXiv.
note: complexity;
TengSpielman.pdf
http://arxiv.org/abs/cs/0607105.
Regularization via Convex Optimization
- === alternate paper
Just Relax: Convex Programming Methods for Identifying Sparse Signals in Noise. ;
by: Joel A. Tropp. ; in: IEEE Transactions on Information Theory (2006). ;
note: one of the earlier papers on a guarantee of perfect reconstruction
http://www.acm.caltech.edu/~jtropp/papers/Tro06-Just-Relax.pdf ;
- === alternate paper
Least Squares Optimization with L1-Norm Regularization. ;
by: Mark Schmidt. ; in: ;
note: mostly an eclectic collection of methods
http://www.di.ens.fr/~mschmidt/Software/lasso.pdf ;
- === alternate paper
Robust Modeling with Erratic Data;
by: J F Claerbout; F Muir ; in: Geophysics 38:826-844, 1973 ;
note: first paper mentioning L1 regularization.
TBK1979.pdf
- === alternate paper
Devolution with the l1 norm.;
by: H L Taylor ; S C Banks ; J F McCoy; in: Geophysics 44:39-52, 1979 ;
note: first paper doing L1 regularization in a formal setting.
- === alternate paper
The In-Crowd Algorithm for Fast Basis Pursuit Denoising;
by: Patrick R. Gill, Albert Wang, Alyosha Molnar; in: IEEE Trans SP 59#10, 2011. ;
note: short succinct intro then with their specific algorithm
http://individual.utoronto.ca/patrickgill/Images/InCrowd.pdf ;
- === alternate paper
Tensor Sparse Coding for Positive Definite Matrices ;
by: Ravishankar Sivalingam, Daniel Boley, Vassilios Morellas, Nikolaos Papanikolopoulos ; in: IEEE PAMI 2013 ;
note:
http://www-users.cs.umn.edu/~boley/publications/papers/TensorSparse13.pdf
- === alternate paper
An ADMM Algorithm for a Class of Total Variation Regularized Estimation Problems. ;
by: Bo Wahlberg and Stephen Boyd and Mariette Annergren and Yang Wang ; in: ;
note: mainly uses ADMM. focus is on methods, but mentions fused lasso and is somewhat self-contained
http://www.stanford.edu/~boyd/papers/pdf/admm_tv_est.pdf ;
- === alternate paper
A Simpler Approach to Matrix Completion;
by: Benjamin Recht;
in: Journal of Machine Learning Research 12 (2011) 3413-3430;
note: early paper on topic;
http://dl.acm.org/citation.cfm?id=2185803.
- === alternate paper
Exact Matrix Completion via Convex Optimization ;
by: Emmanuel J. Candès, Benjamin Recht ; in: Found Comput Math (2009) 9: 717-772 ;
note: main focus: prove a guarantee on perfect reconstruction; more long-winded/readable then C-Tao.
http://link.springer.com/content/pdf/10.1007/s10208-009-9045-5.pdf ;
- === alternate paper
Atomic Decomposition by Basis Pursuit. ;
by: S. S. Chen, D. L. Donoho, and M. A. Saunders. ; in: SIAM Review 2001. ;
note: long survey on BP and sparse repres-s: BP in context ... denoising...
http://www.stanford.edu/group/SOL/papers/BasisPursuit-SIGEST.pdf ;
- === alternate paper
Matching Pursuits with Time Frequency Dictionaries ;
by: S G Mallat and ZF Zhang. ; in: IEEE TSP1993. ;
note:
http://www.cmap.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf;
- === alternate paper
Greedy Basis Pursuit ;
by: Patrick S. Huggins and Steven W. Zucker. ; in: IEEE TSP2007. ;
note:
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04244689 ;
- === alternate paper
Symmetric Nonnegative Matrix Factorization for Graph Clustering. ;
by: D Kuang, H Park, CHQ Ding ; in: SDM, 2012 - SIAM ;
note: relatively short paper using a varition of NMF for clustering.
http://epubs.siam.org/doi/pdf/10.1137/1.9781611972825.10 ;
- === alternate paper
Online Learning for Matrix Factorization and Sparse Coding ;
by: by J Mairal , Bach, Ponce, Sapiro ; in: Journal of Machine Learning Research 11 (2010) 19-60. ;
note: long paper: theory on reconstruction; scalable methods
http://www.jmlr.org/papers/volume11/mairal10a/mairal10a.pdf ;
- === alternate paper
The Power of Convex Relaxation: Near-Optimal Matrix Completion. ;
by: Emmanuel Candes, Terence Tao ; in: ;
note: incoherence param guarantees perfect reconstuction: heavy n the theory
http://arxiv.org/abs/0903.1476 ;
- === alternate paper
Recklessly Approximate Sparse Coding ;
by: Misha Denil, Nando de Freitas ; in: ;
note: dictionary encoding, here focus on represntation, but mentions also learning dict.
http://arxiv.org/abs/1208.0959
http://arxiv.org/abs/1208.0959v1;
- === alternate paper
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images. ;
by: Alfred M. Bruckstein and David L. Donoho and Michael Elad. SIAM Review 15:34-81, 2009. ; in: ;
note: long art: basic guarantees on reconstruction error; variations pursuit; methods; sensitivity and variations, signal proc.
http://epubs.siam.org/doi/abs/10.1137/060657704 ;
- === alternate paper
Decoding by Linear Programming. ;
by: Emmanuel Candes, Terence Tao ; in: ;
note:
http://arxiv.org/abs/math.MG/0502327 ;
- === alternate paper
Newton-Like Methods for Sparse Inverse Covariance Estimation ;
by: Peder A. Olsen, Figen Oztoprak, Jorge Nocedal, Steven J. Rennie. ; in: NIPS 2012 ;
note: mainly on methods, a sample survey
http://papers.nips.cc/paper/4601-newton-like-methods-for-sparse-inverse-covariance-estimation.pdf ;
- === alternate paper
Sparse Inverse Covariance Matrix Estimation Using Quadratic Approximation ;
by: Cho-jui Hsieh, Inderjit S. Dhillon, Pradeep K. Ravikumar, Màtyàs A. Sustik. ; in: NIPS 2011 ;
note: focus is mainly on their 1 method
http://papers.nips.cc/paper/4266-sparse-inverse-covariance-matrix-estimation-using-quadratic-approximation.pdf ;
- === alternate paper
Matrix Completion with Noise ;
by: Emmanuel J. Candès and Yaniv Plan ; in: ;
note:
http://www.eecs.berkeley.edu/~ehsan.elhamifar/EE290A/NoisyMatrixCompletion.pdf ;
- === alternate paper
Distributed optimization and statistical learning via the alternating direction method of multipliers.
by: Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011).
; in: Foundations and Trends in Machine Learning, 3, 1-122. ;
note: long survey paper: basic theory;
http://www.stanford.edu/~boyd/papers/admm/ ;
- === alternate paper
Spectral Regularization Algorithms for Learning Large Incomplete Matrices ;
by: R Mazunder; T Hastie; R Tibshirani (2010) ; in: ;
note: method mainly, mainly based on nuclear norm.
http://www.stanford.edu/~hastie/Papers/mazumder10a.pdf ;
- === alternate paper
Adaptive greedy approximations. ;
by: Davis, G., Mallat, S., & Avellaneda, M. ; in: Constructive Approximation, 13, 57-98. (1997). ;
note:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.6082 ;
- === alternate paper
When does non-negative matrix factorization give a correct decomposition into parts? ;
by: Donoho, D., & Stodden, V. ; in: S. Thrun, L. Saul and B. Schölkopf (Eds.), Advances in neural information processing systems 16. Cambridge, MA: MIT Press. (2004). ;
note:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.3590 ;
- === alternate paper
On the role of sparse and redundant representations in image processing. ;
by: Elad, M., Figueiredo, M., & Ma, Y. ; in: Proceedings of the IEEE, 98, 972-982. (2010). ;
note: modelling in image processing, focus is on image proc
http://yima.csl.illinois.edu/psfile/Sparse_Image_Processing.pdf ;
- === alternate paper
Tensor Dictionary Learning for Positive Definite Matrices ;
by: Ravishankar Sivalingam, Daniel Boley, Vassilios Morellas, Nikolaos Papanikolopoulos ; in: IEEE Image Proc 2015 ;
note:
http://www-users.cs.umn.edu/~boley/publications/papers/TensorDict15.pdf
- === alternate paper
An accelerated proximal gradient algorithm for nuclear norm regularized
linear least squares problems;
by: Kim-Chuan Toh, Sangwoon Yun; in: Pacific J. Optimization, 6 (2010), pp. 615--640;
note: a computational algorithm
http://www.math.nus.edu.sg/~mattohkc/papers/mc11.pdf
- === alternate paper
An Alternating Direction Algorithm for Matrix
Completion with Nonnegative Factors;
by: Yangyang Xu, Wotao Yin, Zaiwen Wen, Yin Zhang;
UCLA CAM report (20122)
note: a computational algorithm
ftp://ftp.math.ucla.edu/pub/camreport/cam11-07.pdf
Back to Class Web Page
