Machine Learning Foundations for Natural Systems

Summer Notes:

We start with Kernel Principal Components: Review material, Codes, prepare chapter and examples.

  1. Kernel Principal Components Part 2


Follow the course progress on Stellar by clicking here

12.S592, 54-1827, MWF 930-1030, Spring 2017

Sai Ravela (

The first part of an advanced two-semester sequence in Machine Learning that enables one to design Learning-based approaches for Natural Systems Science.

Natural Systems Science:

We are motivated at present  by problems in applications across the breadth of Nature pertaining to Estimation, Prediction, Parameterization, Characterization, Discovery and Decision Making.  Thre is a focus on Geophysics this term: Data Assimilation, Autonomous Environmental Mapping, Model Reduction, Uncertainty Quantification, Sensor Planning, Prediction and Predictability, Planning for Risk Mitigation, Convective Super-parameterization, Radiative-Convective Equilibrium, Nonlocal Operators, Teleconnections, Particle Detection and Sizing, Species Characterization, Paleoclimate, Event Detection and Tracking in X (Voclanoes, Earthquakes, Hurricanes, Tsunamis, Storms and Transits). Super-resolution/Downscaling, Coherent Structures in Turbulence,  Seismic Imaging and Geomorphology, Porus Media, Reservoirs, Exoplanets.

However, other Engineering, Science and Finance applications may be included depending on participant interest. Whilst our motivation and reach is broad, we will drill down each term to a few core applications that is set by participant interest.


Participants will be required to complete monthly assignments and a collaborative project.  This course is different from a typical course. It's hands-on, for both of us. Think about it as a "learnathon," we come together to solve problems/applications better than what either one of us can do alone. My job is to help tease you to the fundamentals and your job is to help take them to your application.  Prior experience with Probability, Linear Algebra, Statistics or Data Science courses would be helpful, or permission of the instructor.


The entire course is organized in three phases; Preparatory, Core and Advanced Topics.

Preparatory Topics (both terms): Classification, Regression, Clustering, Density Estimation,  Supervised vs. Unsupervised Learning, Loss functions, Training-Calibration-Validation, Sampling and Variational Inference, Nearest Neighbors, Generalization, Bias-Variance Dilemma, Model Selection, Feature Selection, Reproducing Kernel Hilbert Spaces, Regularization: Smoothness, Sparseness, Entropy and Information.

Core Material Term 1 Core Material Term 2
Non Parametric Bayesian Inference, Graphical Models, Regression Machines, Kernel Machines, Ensemble Learning, Manifold Learning, Transfer Learning, Recurrent and Deep Learning Basis and Dictionary Learning, Gaussian Processes. Stochastic Model Reduction, Abstraction, Segmentation and Grouping.  Markov Decision Processes, Reinforcement Learning, Incremental Online and Active Learning, Causal Learning, Information Theoretic Learning

Advanced Topics (both terms): Uncertain Probabilities and Ambiguity, Curse of Dimensionality, Sparse Data vs. Big Data, Rigged Hilbert Spaces, Parsimony Principles, Generic Chaining and Extremes, Predictability and Learnability, Model-based (Physically-based) Learning. As with applications, we will be selective with the Advanced Topics each term.


Follow the course on Stellar by clicking here