Advanced Data Analysis using Wavelets and Machine Learning

Welcome to my course on Machine Learning and Data Analysis, a course that will teach you how to use advanced algorithms to solve real problems with data. I am Emanuele, a mechanical engineer with a PhD in advanced algorithms, and I will be your instructor for this course.

This course consists of four main parts:

• Part 1: Overview on Fourier Analysis and Wavelets. You will learn the basics of these two powerful mathematical tools for analyzing signals and images in different domains.
• Part 2: Data Analysis with Fourier Series, Transforms and Wavelets. You will learn how to apply these methods to process and explore data efficiently and effectively, both in time and frequency domains.
• Part 3: Machine Learning Methods. You will learn how to use techniques that enable computers to learn from data and make intelligent predictions or decisions, such as linear regression, curve fitting, least squares, gradient descent, Singular Value Decomposition (and more).
• Part 4: Dynamical Systems. You will learn how to model and understand complex and nonlinear phenomena that change over time, using mathematical equations. We will also apply machine learning techniques to dynamical systems, such as the SINDy algorithm.

By the end of this course, you will be able to:

• Understand the principles and applications of Fourier analysis and wavelets
• Use Fourier series and transforms to analyze data in various domains
• Apply machine learning methods to different problems
• Extract features from data using wavelets
• Understand the importance of sparsity of natural data, as well as the revolutionary concept of compressed sensing, with realistic examples.
• Discover the governing equations of a dynamical system from time series data (SINDy algorithm).

I hope you enjoy this course and find it useful for your personal and professional goals.

————————————————————————————————————————————

Let’s provide some more details about the main parts of this course:

Part 1 constitutes a preliminary introduction to Fourier and Wavelet Analysis. Special focus will be put on understanding the most relevant concepts related to these fundamental topics.

In part 2, the Fourier series and the Fourier Transform are introduced. Although the most important mathematical formulae are shown, the focus is not on the mathematics. One of the key points of this part is to show one possible application of the Fourier Transform: the spectral derivative. Then, we introduce the concept of Wavelets more in detail by showing some applications of Multiresolution Analysis.

This is exemplified with Matlab, without using rigorous mathematical formulae. The student can follow and get the intuition even if they have no access to Matlab.

Another important achievement of this part is to convey a simple but thorough explanation of the well-known computational FFT method.

There are also some extras on the Inverse Wavelet Transform and the Uncertainty principle (here we see more mathematics, but this is an extra, if you want to skip it, just do it).

In part 3, some machine learning techniques are introduced: the methods of curve-fitting, gradient descent, linear regression, Singular Value Decomposition (SVD), feature extraction, classification, Gaussian Mixture Model (GMM). The objective in this part is to show some practical applications and cast light on their usefulness.

We will also focus on sparsity and compressed sensing, which are related concepts in signal processing. Sparsity means that a signal can be represented by a few non-zero coefficients in some domain, such as frequency or wavelet. Compressed sensing means that a signal can be reconstructed from fewer measurements than the Nyquist–Shannon sampling theorem requires, by exploiting its sparsity and using optimization techniques. These concepts are useful for reducing the dimensionality and complexity of data in machine learning applications, such as image processing or radar imaging.

Part 4 is a self-contained introduction to dynamical models. The models contained in this part are the prey-predator model, the model of epidemics, the logistic model of population growth.

The student will learn how to implement these models using free and open-source software called Scilab (quite similar to Matlab).

Related to Part 4, there is an application of machine learning technique called SINDy, which is an acronym for Sparse Identification of Nonlinear Dynamics. It is a machine learning algorithm that can discover the governing equations of a dynamical system from time series data. The main idea is to assume that the system can be described by a sparse set of nonlinear functions, and then use a sparsity-promoting regression technique to find the coefficients of these functions that best fit the data. This way, SINDy can recover interpretable and parsimonious models of complex systems.

Note: For some of the lectures of the course, I was inspired by S.L. Brunton and J. N. Kutz’s book titled “Data-Driven Science and Engineering”. This book is an excellent source of information to dig deeper on most (although not all) of the topics discussed in the course.