CS4234Optimiz(s)ation Algorithms
S1 AY21/22
Welcome!
Optimization Algorithms ('z'/American and 's'/British)– https://www.comp.nus.edu.sg/~stevenha/cs4234.html
Materials updated from 6 AYs ago, has branched out:– https://www.comp.nus.edu.sg/~gilbert/CS4234/
Instructor: Steven Halim– Office: COM2-03-37 (unless WFH)– Office Phone: not used, use email/Discord– Email: [email protected]– Discord: “the lecturer”
CS4234 Overview (1)
• Read my IOI stuff (IOI 2021 and IOI 2020)or my ICPC stuff (ICPC Jakarta Online 2020)or my achievements (this news, NDA18, ATEA, or this list)
• Social Media:– On Facebook (search my gmail),– On Instagram and Telegram
(steven_halim_7)• Has 1 wife (Grace) and
3 'J' kids (Jane, Joshua, Jemimah)• Wish for CS4234 S1 AY21/22:
– To restore my ≥ 4.3 teaching feedback• It fell to 4.1 due to a very busy IOI 2020 last year
CS4234 Overview (2)
Finally…
Tutor: Me again…– I am unable to secure any suitable TA…
• Predominantly due to my busyness with the IOI 2021pre/actual/post event activities in May/June/July 2021
• Hopefully any of you who are still year 3 now (or younger) nowand scored A- or above in CS4234 this semestercan keep this TA vacancy in mind for next S1 AY 2022/23…
– Slots: All F2F, at SR_LT19 next door (of this LT19)• T1: Mon, 11am-12noon• T2: Mon, 2-3pm• T3: Mon, 5-6pm
– If sick (MC), kena QO (self or immediate family member), etc• Show me a valid proof, you will be excused without penalty,
but read the modal answers carefullyand ask me personally if you have further questions
CS4234 Overview (3)
Yes I am in LT19 today and actually prefer to do F2F(hopefully most if not all of you)
Rough count, about 20/26 onsite today
Yes… but I still prefer online..(hopefully minority or best: 0)
(lectures are recorded)(but onsite tutorial attendance...??)
Rough count, about 6/26 onsite today
I am still overseas/SHN/LOA/others(so canNOT be in LT19 today)
(let me know when you will be onsite?)
Rough count, about 4 on SHN this week
CS4234 Overview – Survey (1)Among those in LT19…, raise your hand (obviously the bottom right cell cannot be filled in now)
Time to do attendance taking • https://inetapps.nus.edu.sg/ctr/HomeYou need to:• Declare your temperature regularly• Do first ART test soon (ignore if you have done so)• Should *not* be on SHN/LOA at the moment
Before I forget (check point 1)
Optimization:Find the minimum/maximum:
Discrete: a collection of items
Combinatorial: a collection of items generated
by counting, combining, and enumerating
Examples:Integers
Graphs and Trees
Similar structures…
Optimization Algorithms
• Searching• Shortest Paths
– 25=32 different (how to count?) (acyclic) simple paths from 010• With a correct algorithm, which is ____, we can solve this in O(V+E)
• Minimum Spanning Tree– 44-2 = 42 = 16 different (how to count?) spanning trees
• With a correct algorithm, which is _________,we can solve this in O(E log V)
Search Space Size != DifficultyA = [5, 4, 2, 7, 3, 6, 1], O(N) to find-min, how?
Sorted-A = [1, 2, 3, 4, 5, 6, 7], O(1), how?
Find the “best” item in a large set of items:Problem Set of items Size DifficultySearching List of integers Linear Easy
Shortest Path All paths in a graph Exponential Easy
Minimum Spanning Tree All spanning trees Exponential Easy
Steiner Tree All steiner trees Exponential Hard
Travelling Salesman All possible tours Exponential Hard
Matching All possible matchings Exponential Easy
Bipartite Vertex Cover All possible covers Exponential Easy
Vertex Cover All possible covers Exponential Hard
Maximum Clique All possible subsets Exponential Hard
Combinatorial Optimization
Find the “best” item in a large set of items:Problem DifficultyMaintain student records Easy
Exam timetable scheduling Hard
Program halting problem Impossible
Data compression Easy
VLSI chip layout Hard
Job assignment problem Easy
Computer deadlock problem Easy
Finding patterns in a database Easy
Combinatorial Optimization
Operations Research:How to make better decisions (e.g., maximize profit)
Project planning / critical path analysisFacility location: where to open stores / plantsFloorplanning: layout of factory or computer chipsSupply chain managementBerth assignment problem (BAP): port managementAssignment problems (e.g., weapon target assignment)Routing / transportation problems: buses, subways, trucking.Airline ticket pricing
Combinatorial Optimization
Optimization:Find the minimum/maximum:
Discrete: a collection of items
Combinatorial: a collection of items generated
by counting, combining, and enumerating
Continuous: given a function f(x), find the
vector x that maximizes f(x)Calculus stuffs, not the focus of CS4234
Optimization Algorithms
Optimization Algorithms
Goals: Algorithmic1. Design (problem solving)
2. Analysis (rigorous, deep understanding)
3. Implementation (able to put it to use)
CS3230
CS4234
“If you need your software to run twice as fast, hire better programmers.
But if you need your software to run more than twice as fast, use a better algorithm.”
-- Software Lead at Microsoft
Now the sixth iteration under Steven– First offered in S1 AY16/17, +ve feedback
– Quite OK also in S1 AY17/18, AY18/19, AY19/20– Poor in S1 AY20/21
• Mostly due to Steven missing for *most* part of September 2020 due to IOI 2020…
– https://ioi2020.sg/schedule/
– Subset of CS5234 (which has also evolved)• CS5234: Combinatorial and Graph Algorithms• More general: all sorts of optimization (not only graphs)• More specific: just optimization
CS4234 : Optimization Algorithms
Target students:– Advanced (3rd or 4th year) undergraduates
• There are (a few) master (5)/local? exchange (7) students– And also, ex CS3233 students (6)
– (Very) interested in algorithms– Interested in tools for solving hard problems
Prerequisites: – CS3230 (Analysis of Algorithms) CS2040 CS1231
– Mathematical fundamentals (currently MA1101R, but good performance in CS1231 is needed too)
CS4234 : Students
I took CS3233/join NUS ICPC before
1. Fidella Widjojo2. Gabriel Goh Kheng Lin3. Ganapathy Sanath Balaji4. Guan Yang Ze5. Lim Li6. Sherman Lim Jun HongYeah that’s all
I took CS2040C with you before(skip if you then also took CS3233)
1. Chen Chao
Do I miss anyone?
I know you from other activity/ies (elaborate)1. Agnes Natasya (from CS2040S)2. Audrey Felicio Anwar (helped with Moscow Workshop 2019)
Do I miss anyone?
CS4234 Overview – Survey (3)Do you know me before?
(just raise your hand and I roughly edit this slide myself)
I am currently in Kattis top 100https://open.kattis.com/ranklist
94. Ganapathy Sanath Balaji99. Guan Yang Ze
I am not in top 100 but top 50 herehttps://open.kattis.com/universities/nus.edu.sg
201. Koh Rayson211. Sherman Lim Jun Hong
I have more than 500 Kattis points actually…396. Fidella Widjojo437. Lim Li813. Audrey Felicio Anwar826. Gabriel Goh Kheng Lin
Do I miss anyone else?
CS4234 Overview – Survey (4)Top Kattis users
(just raise your hand and I roughly edit this slide myself)
Discord channel (I can be anonymous)
Many
Facebook post (no longer used)
From 2021 onwards
Weekly Zoom sessions with Steven (date/time TBC)
TBC… may not be needed
CS4234 Overview – Survey (-)e-Consultation Preference
(give a tick/anonymous is ok on your preferred means for this class)
You must already know these (use https://visualgo.net to review):• Data Structures (with Analysis)
– Linked Lists, Stacks, Queues,
– (Binary) Heaps and Priority Queues,
– Binary Search Trees, Balanced BSTs
• Algorithm Design Paradigms (with Analysis)– Standard Sorting and Searching algorithms,
– Complete Search, Divide-and-Conquer,Greedy Algorithms, Dynamic Programming,
– Graph traversal algorithms: DFS, BFS,
– MST Algorithms and Shortest Path Algorithms,
Pre-requisites (1)
You must already know these (use https://visualgo.net to review):• Analysis of Algorithms
– Expertise with Big-O, , notations
– Summation of series, Master Theorem
– Competent with Algorithmic Analysis:
Quicksort, Heapsort, Divide-and-Conquer algorithms
DFS, BFS, MST & Shortest Paths algorithms
Pre-requisites (2)
• https://nus.kattis.com/sessions/ng4cwg, graded this time (5% :O)– So that I can quickly be alerted on any “Prerequisites” issue involving some of you
– This is an elective level 4 algorithm module after all…• There is 1 CS2040C S2 AY20/21 exam question among the 5,
which is ______– I expect everyone to AC this by the end of Wk2, eventually
– It involves a simple data structure taught in CS2040 level, which is _______
– And a problem solving paradigm taught in CS2040+CS3230 level, which is _______• There are ≥ 1 tasks involving spanning trees (frequently used in CS4234):
– SSSP (also produces a spanning tree), which is/are _______• Use this algorithm _______
– MST (obviously produces a spanning tree), which is _______• Use this algorithm _______
– Graph Traversal (that produces a spanning tree), which is _______• Use this algorithm (or that algorithm) _______
PS1 – Prerequisites (1)
• There is 1 task involving Dynamic Programming (DP), which is ______– You should have learned DP in CS3230 and a prerequisite for the early topics of CS4234
– That task is also can be classified as an NP-hard (decision, not optimization) problem• That’s the last part of CS3230 and the topic of the first few weeks of CS4234
• The first 3 tasks are considered "high school level" tasks– They are ______, ______, and ______
– Please get AC(cepted) for at least these 3 out of 5 tasks
– Strongly reconsider taking CS4234 this sem if you struggled badly just to get these 3 ACs• The other 2 tasks are sufficiently more challenging
– Do not feel bad if they are still not AC after deadline, there will be partial marks!!
– But still of CS2040 and/or CS3230 level
– Not discussed today, but maybe during next Week 02 depending on the situation
PS1 – Prerequisites (2)
Four PS deadlines on Week 02, 04, 06, 09Must register by NOW (12 Aug 2021)! https://nus.kattis.com/courses/CS4234/CS4234_S1_AY2122/registerSecret registration key in the welcome email!
Mini project deadline on Week 13 Mid-term Test @ LT19 F2F :O, Week 08
Thursday, 07 October 2021, using lecture time Final Assessment @ ?? F2F (I asked for it)
Friday, 26 November 2021, 2.30-4.30 PM
CS4234 Important Dates
Grading20% Problem Sets15% Mini Project5% Tutorial Attendance + Participation Level25% Mid-term Test35% Final Assessment
Problem Sets– 4 sets (approximately every 2-3 weeks, 5% each)
– Algorithm design, analysis, with implementation• Using just one Online Judge : Kattis this time
CS4234 Grading
Mini-Project– Small project, triple? (41/3 = 14 groups, TBC)
– Idea: To put together some of the different ideaswe have used in the class
– Time scale: Last 3-4 weeks of the semester
– 15% Tutorial Attendance & Participation
– 5%, bell curve system, i.e.,more active in F2F class = more points
CS4234 Grading
Already ReleasedPS1-Prerequisites:
Released on Week 00 (Thursday, 05 August 2021, 08.00am) but will not due until the end of Week 02 (Saturday, 21 August 2021, 07.59am), so that you can use this information to retain/drop CS4234
As of 11am today, out of 41 students:– 24 have ≥ 3 ACs (considered safe)– 11 have registered but not much attempt (yet)– 6 MIA?
CS4234 Problem Sets
Problem Set GradingSimple scheme (graded with help of Online Judge):
5 : Perfect (coding style is ignored)
4 : Excellent, minor errors
3 : Satisfactory, mostly right
2 : Some mistakes
1 : Many mistakes (won’t be 0 if you submit)
0 : No submission (or caught by plagiarism checker)
CS4234 Problem Sets
What to submit (online judge version):C++ (17), Python (3), or Java (11) code that solves the optimization problem(s) --- I am not familiar with other languages :O
Especially if not AC at the end: Write additional comments inside source code (only the last submitted code will be read) – optional if AC:
1. Summary of the coded algorithm in English
2. Write a "proof of correctness" (hopefully your non AC is just due to coding bug) and "performance analysis" (hopefully your TLE is just due to constant factor weakness)
CS4234 Problem Sets
C++
About ~10 onsite
Python
About ~6 onsiteWarning: may TLE at some problems!
Use C++ if you can
Java
About ~3 onsite
Others (elaborate)
About ~2-3 onsite
CS4234 Overview – Survey (5)Your “mother tongue” programming language is
(just raise your hand, I will only do a rough count)
What to submit (paper version):Concise and precise answers:
Solutions should be rigorous, containing all necessary detail, but no more
Algorithm descriptions consist of:1. Summary of results/claims2. Description of algorithm in English3. Pseudocode, if helpful4. Worked example of algorithm5. Diagram / picture6. Proof of correctness and performance analysis
CS4234 Problem Sets
How to draw (graph) pictures?By hand:
Either submit hardcopy, or scan,Or take a picture with your phone!Or use a tablet / iPad…
Digitally: 1. VisuAlgo (graph) drawing feature2. Excel or Powerpoint (hmmm)3. ???
CS4234 Problem Sets
Policy on plagiarism:Do your work yourself:
Your submission should be unique, unlike anything else submitted, on the web, etc
You can discuss with other students (e.g., Discord), but…: 1. Discuss only general approach and techniques2. Just take general notes during such discussions3. Code the solution on your own4. List all collaborators in your submission
Do not ask for solutions on the web:Use web to learn techniques and to review material from class; students who ask CS4234stuffs in StackOverflow, Quora etc will be punished
CS4234 Problem Sets
Policy on plagiarism:Penalized severely:
First offense: Minimum of one letter grade lost on final grade for class (or referral to SoC disciplinary committee)
Second offense: F for the class and/or referral to SoC disciplinary committee
Do not copy/compare solutions!
PS: Do not store your coded solutions in publicrepository (e.g., GitHub, ideone.com, repl)with public setting
CS4234 Problem Sets
Introduction to Algorithms– Cormen, Leiserson, Rivest, Stein
– Strongly Recommended…
Textbooks
Algorithm Design– Kleinberg and Tardos
– Recommended…
Textbooks
Competitive Programming 4 - esp Book 2– Steven Halim, Felix Halim, and Suhendry Effendy
– Only a subset of these are relevant (especially Book 2 - Chapter 8+9)…• Also get Book 1 if you think your DS/BF/Greedy/DnC/DP/simple Graph concepts are still shaky
– Get from NUS co-op (Central Forum), or order online• https://www.nuscoop.sg/search?keywords=competitive%20programming
Textbooks
Stochastic Local Search: Foundations and Applications– Holger H. Hoos & Thomas Stützle
– For the last part of this course (a hard book to get)
Textbooks
Topics (90% same as last 5 AYs)Introduction to Combinatorial Optimization
Vertex Cover, Set Cover, Steiner Tree, TSP
Linear Programming
LPs, Relaxations, Rounding
Flows and Matching (slightly expanded)Maximum Flow (PR dropped – due to PH on Wk12),
(Bipartite) Matching (now plus weighted)
Local SearchGradient Descent, Meta-heuristics
CS4234 Topics
Time to do attendance taking • https://inetapps.nus.edu.sg/ctr/Home
You need to:• Declare your temperature regularly• Do first ART test soon (ignore if you have done so)• Should *not* be on SHN/LOA at the moment
Before I forget (check point 2)