COLUMBIA UNIVERSITYDepartment of Electrical Engineering

The Fu Foundation School of Engineering and Applied Science

IN THE CITY OF NEW YORK

Signals, Information and Data

John Wright

Signals, Information and Data Faculty Shih-Fu Chang – Multimedia John Paisley – Bayesian nonparametrics Xiaodong Wang – Communications, genomic signal processing John Wright – High-dimensional data, images

Many faculty have connections, e.g., Dimitris Anastassiou – Computational Biology, Nima Mesgarani – neural signal processing, Aurel Lazar – neuroengineering…

Possible Course Sequences Consult the web version

SIGNALS, INFORMATION, DATA, LEARNING & CONTROL (Senior/graduate & Advanced graduate courses in EE)

Green = Senior/grad; Orange = Advanced graduate; Bold border = offered regularly; Updated AUG 2019

Recent topics for ELEN E688*, EECS E689*, EECS E669*, & other related topics courses: ELEN E4903 Topic: Machine Learning (Spring ‘16) EECS E6690 Topic: Stat. Learning in Bio. & Informa�on Sys. (Fall '19,'18, Spring '18)ELEN E6880 Topic: MIMO Wireless Communication (Spring ’14, ‘13, ‘12, ‘11) ELEN E6880 Topic: Rand. Matrix Theory with Eng & Data Sci App (Fall '19,'18) ELEN E6881 Topic: Multicarrier Resource Allocation (Fall '14) ELEN E6882 Topic: Visual Search Engine (Spring ‘12, ‘11) ELEN E6882 Topic: Mobile Sensing & Analysis (Spring ’16, ‘15) ELEN E6883 Topic: Detection & Es�mation (Fall ‘10, ‘09, ‘08, ‘06) ELEN E6883 Topic: An Introduc�on to Blockchain Technology (Spring '19)ELEN E6884 Topic: Data Compression (Spring ’14, ‘13, ‘12, ‘11) ELEN E6885 Topic: Reinforcement Learning (Fall '19 '18,‘'17) ELEN E6886 Topic: Sparse Rep. / High Dim. Geom. (Spring '19,’17, Fall ’15) ELEN E6887 Topic: Sta�s�cal Learning Theory (Spring ‘10, ‘09)

ELEN E6888 Topic: Broadband Wireless (Spring ’17, ’10 - ’16) ELEN E6889 Topic: Large Data Stream Proc. (Spring '19,’17, Fall ’15, Spring ’14, ‘10) EECS E6890 Topic: Visual Recognition and Search (Spring ’14, ‘13) EECS E6891 Topic: Reproducing Computa�onal Results (Spring ’14, ‘13) EECS E6892 Topic: Bayesian Models in Machine Learning (Fall ’15, Spring ‘14) EECS E6893 Topic: Big Data Analytics (Fall '19, ‘18 ’17, ‘16, ‘15, ‘14) EECS E6894 Topic: Deep Learn for Comp Visio & NLP (Fall ’18, Spring ’17, ‘15) EECS E6895 Topic: Adv. Big Data Analytics (Spring '19, ’17, ‘16, ‘15) EECS E6896 Topic: Quantum Computing and Comm. (Fall '19, ’17) EECS E6897 Topic: Distributed Storage Systems for Big Data (Fall '19)EECS E6898 Topic: From Data to Solutions (Fall ’17, ‘16, Spring ’16, Fall'12-'14)

ELEN E688* Topics in Sig. Processing

ELEN E6850 Visual Info. Systems

EEME E8601 Topics in Control Thy.

EEME E6601 Intro. to Control Thy.

EEME E4601 Digital Control

ELEN E6201 Linear System Theory

ELEN E4830 Digital Image Processing

ELEN E6820 Speech & Audio Proc.

ELEN E6860 Advanced Dig. Sig. Proc.

ELEN E4896 Music Signal Processing

ELEN E4810 Digital Signal Processing

ELEN E4815 Random Sig. & Noise

EEME E3601 Class. Control

ELEN E3201 Circuit Anal.

APMA E3101 Lin. Algebra

IEOR E3658 Probability

ELEN E3801 Sig. & Sys.

OR

OR EEME E6602 Modern Control Thy.

EEME E6610 Op�mal Control Thy.

OR

OR

OR

EECS E689* Topics in Info. Processing

EECS E669* Topics in Data Driven Anal.

EECS E6870 Speech Recogni�on

Probability & Sta�s�cs

ELEN E6873 Detec�on & Es�mation

ELEN E4835 Intro. Adapt. Signal Reps.

IEOR E3658 Probability

EEOR E4650 Convex Opt. for EE

ELEN E4511 Power Sys. Anal. & Cont.

ELEN E9501 EECS E9601 ELEN E9801

MATH V2030 ODEs

ELEN E3401 Elec. & Mag.

EECS E4750 Het. Comp. SP, Data Proc.

ELEN E3801 Sig. & Sys.

EECS E6720 Bayes. Model Mach. Learn.

Math & Prog

Prog

E4810 Digital Signal Processing John Wright, Mondays, 1:10-3:40

Digital filtering in time and frequency domain

Discrete-time signals and systems, sampling theory, transform analysis, system structures, IIR and FIR filter design

Discrete Fourier Transform, Fast Fourier Transforms.

E4750 Hybrid Computing for Signal and Data Processing Prof. Zoran Kostic, Thursdays 1:10-3:40

Deploying signal processing and communications algorithms on

contemporary mobile processors

Signal processing with heterogeneous computing infrastructures consisting of general purpose, graphics and digital signal processors

Programming languages such as OpenCL and CUDA for computational gains

Project/applications in audio, image and video processing and computational data analysis.

1

Electrical Engineering Department

New Course: ELEN E4750 Fall 2014 Thursdays 1:10pm-3:40pm

Signal Processing and Communications on Mobile Multicore

Processors (Applications of Parallel Computing) Target Audience: Open to SEAS Students interested in acquiring SW and systems skills in low-power parallel computing, …  skills  of  critical  importance  to  mobile   computing/communications industry in the next decade, applicable to research projects of CU EE/CS faculty. Course Description: Methods for deploying signal processing and communications algorithms on

contemporary mobile processors with heterogeneous computing infrastructures consisting of a mix of general purpose, graphics and digital signal processors. Using programming languages such as OpenCL and CUDA for computational speedup in audio, image and video processing and computational data analysis. Significant design project. Prof. Zoran Kostic zk2172 (at) columbia.edu Link to information: Course Link

ECBM 4040 Neural Networks and Deep Learning Prof. Zoran Kostic, Fridays 10:10-12:40

Introduction to neural networks and recent advances in deep learning

Focuses on models and intuition – feedforward networks, convolutional networks, recurrent networks, feature learning for classification

Analytical study and software / programming projects

Applications in speech and object recognition

Image – Wikipedia

E6601 Introduction to Control Theory Prof. Richard Longman, Wednesdays 7-9:30 PM

Introduction to classical and modern feedback control (graduate level)

Scalar and matrix differential equation models. Transfer functions, block diagram manipulations, closed-loop response.

Proportional, rate, and integral controllers, and compensators. Design by root locus and frequency response.

Controllability, observability. Luenberger observers, pole placement, and linear-quadratic cost controllers.

E6690 Statistical Learning in Biological and Information Systems Prof. Predrag Jelenkovic, Tuesdays 4:10-6:40 PM

Fundamental statistical (machine) learning techniques

Basics of statistics and optimization Supervised & unsupervised learning, inference and prediction, models,

regularization

High dimensionality, graphs, communities, ranking, association rules

Example

●

●

●

●

● ●

●

●

●

●

● ●●

●

●●

●

●

●

●

5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

training

x

y

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

testing

x

y

I More complicated models not always better - e.g., overfitting

I Amount of available data

I Interpretability

E6893 Topics in Info Processing: Big Data Analytics Prof. C. Y. Lin, Friday 7-9:30 PM

Analyzing Big Data: from acquisition and storage to processing

Platforms, including Hadoop, Spark

Uploading, distribute, and processing data, including HDFS, HBase, KV stores, document database, and graph database

Large-scale machine learning for big data

© 2014 CY Lin, Columbia UniversityE6893 Big Data Analytics – Lecture 1: Overview

How to Visualize Huge Static Graph

Challenging Task :

Squeezing millions and even billions of records into million pixels (1600 X 1200 ≈ 2 million pixels)

Tree of Life by Dr. Yifan Hu

76425 species

Facebook friendship graph by Paul Butler

500 million users

The information diffusion graph of the death of Osama bin Laden by Gilad Lotan

14.8 million tweets

34

© 2014 CY Lin, Columbia UniversityE6893 Big Data Analytics – Lecture 1: Overview

How to Visualize Huge Static Graph

Challenging Task :

Squeezing millions and even billions of records into million pixels (1600 X 1200 ≈ 2 million pixels)

Tree of Life by Dr. Yifan Hu

76425 species

Facebook friendship graph by Paul Butler

500 million users

The information diffusion graph of the death of Osama bin Laden by Gilad Lotan

14.8 million tweets

34

E9601 Bayesian Models in Machine Learning Prof. John Paisley, Wednesday 4:10-6:40

Advanced course on Bayesian approaches to machine learing

Mixed-membership models, latent factor models, Bayesian nonparametrics

Bayesian inference; mean-field variational methods

Applications to image processing, topic modeling, collaborative filtering, recommendation systems.

α z wθ

β

MN

Special topics (688X): ELEN E6880 Topic: Space-Time Coding / SP Wireless Comm. (Sp.‘08, ’07) ELEN E6880 Topic: MIMO Wireless Communication (Spring ‘13, ‘12, ’11) ELEN E6881 Topic: Video Coding and Communications (Spring ‘09, ’08) ELEN E6882 Topic: Stat. Methods for Video Index & Analysis (Fall ’07) ELEN E6882 Topic: Visual Search Engine (Spring ‘12, ’11) ELEN E6883 Topic: Detection & Estimation (Fall ‘10, ‘09, ‘08, ’06) ELEN E6884 Topic: Speech Recognition (Fall ’05) ELEN E6884 Topic: Data Compression (Spring ‘13, ‘12, ’11) ELEN E6885 Topic: Network Science (Fall ‘13, ‘12, ‘11, ’10) ELEN E6886 Topic: Multimedia Security Systems (Spring ’06) ELEN E6886 Topic: Sparse Rep. / High Dim. Geometry (Fall ‘12, ’11) ELEN E6887 Topic: Statistical Learning Theory (Spring ‘10, ’09) ELEN E6888 Topic: Intro. to LTE & WiMax Systems (Spring ‘13, ‘12, ‘11, ’10) ELEN E6889 Topic: Distributed Stream Processing and Analysis (Spring ’10) EECS E6890 Topic: Visual Recognition and Search (Spring ’13) EECS E6891 Topic: Replicating Computational Results (Spring ’13) EECS E6898 Topic: From Data to Solutions (Fall ‘13, ‘12)

E6896 Topics in Info. Processing: Quantum Computing and Communications Prof. Alexi Ashikhmin, Wednesdays 1:10-3:40 PM

E6880 Topics in Signal Processing:

Random Matrix Theory and Applications to AI and Data Science Prof. Ori Shantal, Fridays 4:10-6:40 PM

E6885 Topics in Signal Processing:

Reinforcement Learning Prof. Chong Li, Fridays 8-10:10 AM

[1] Diagram: https://upload.wikimedia.org/wikipedia/commons/thumb/1/1b/Reinforcement_learning_diagram.svg/500px-Reinforcement_learning_diagram.svg.png

Related course offerings (coding, communications, ect.) ELEN 4702 Digital Communications ELEN 4764 Internet of Things – Intelligent and Connected Systems ELEN 6761 Computer Communications Networks ELEN 6776 Topics: Content Distribution Networks ELEN 6718 Error Correcting Codes: Classical and Modern ELEN 6322 VLSI hardware architectures for signal processing and machine

learning ELEN 6950 Wireless and Mobile Networking