SF2822 APPLIED NONLINEAR OPTIMIZATION Spring 2014 Course analysis • Course number, course name, credits, student group, period of study SF2822 Applied nonlinear optimization, 7.5hp, advanced level, elective for F, M, T, and master in mathematics, scientific computing and aerospace engineering, period 4. • Teachers Instructor and examiner: Anders Forsgren Exercise leader and project leader: Tove Odland • Course literature – Linear and Nonlinear Programming, second edition, by I. Griva, S. G. Nash and A. Sofer, SIAM, 2009. – Exercises in applied nonlinear optimization, 2013/2014. – Supplementary course material in applied nonlinear optimization, 2012/2013. – Lecture notes in applied nonlinear optimization, 2012/2013. – GAMS, A user’s guide. – GAMS. GAMS is installed in the Ubuntu computer rooms. It may also be downloaded from the web for use on a personal computer. – Two project assignments that are handed out during the course, March 27 and April 22 respectively. • Number of registered students (in Ladok): 34 of which 8 are PhD students • Number of students passing the projects: 33 of which 7 are PhD students • Number of students passing the first exam: 27 out of 33 The questions below are related to the LEQ course evaluation 1. How well does the course’s footprint agree with your own idea on the course’s learning environment? There is a good agreement. The setup of the course seems to go well with most students. 2. Which areas of strength can you identify in the course’s footprint? Are you able to find written comments that confirm your observation? The di↵erent moments in the course give a strength. Projects, theory and methods. “The projects and the exercise classes.”. The work in projects. “I liked the projects.”. 3. Which areas of development are you able to identify in the course’s foot- print? Are you able to find written comments that confirm your observa- tion? Some students would like to have had more time to reflect. This is difficult due to pressure from this course and others.
Transcript
SF2822 APPLIED NONLINEAR OPTIMIZATION Spring 2014Course analysis
• Course number, course name, credits, student group, period of studySF2822 Applied nonlinear optimization, 7.5hp, advanced level, elective for F, M, T,and master in mathematics, scientific computing and aerospace engineering, period4.
• Teachers
Instructor and examiner: Anders ForsgrenExercise leader and project leader: Tove Odland
• Course literature
– Linear and Nonlinear Programming, second edition, by I. Griva, S. G. Nashand A. Sofer, SIAM, 2009.
– Exercises in applied nonlinear optimization, 2013/2014.
– Supplementary course material in applied nonlinear optimization, 2012/2013.
– Lecture notes in applied nonlinear optimization, 2012/2013.
– GAMS, A user’s guide.
– GAMS. GAMS is installed in the Ubuntu computer rooms. It may also bedownloaded from the web for use on a personal computer.
– Two project assignments that are handed out during the course, March 27 andApril 22 respectively.
• Number of registered students (in Ladok): 34 of which 8 are PhD students
• Number of students passing the projects: 33 of which 7 are PhD students
• Number of students passing the first exam: 27 out of 33
The questions below are related to the LEQ course evaluation
1. How well does the course’s footprint agree with your own idea on thecourse’s learning environment?
There is a good agreement. The setup of the course seems to go well with moststudents.
2. Which areas of strength can you identify in the course’s footprint? Areyou able to find written comments that confirm your observation?
The di↵erent moments in the course give a strength. Projects, theory and methods.“The projects and the exercise classes.”.
The work in projects.“I liked the projects.”.
3. Which areas of development are you able to identify in the course’s foot-print? Are you able to find written comments that confirm your observa-tion?
Some students would like to have had more time to reflect. This is di�cult due topressure from this course and others.
4. What did the students like best about the course?
The projects and the setup of the course.
5. What do the students think might be improved?
Maybe restructure part of the course so as not to give impression of overlap.
6. Is there any useful advise for future students?
Take time for the projects. Repeat linear algebra prior to starting the course.
7. Is there anything else that should be noted from the material?
Maybe omit semidefinite programming from the contents.
8. What might be improved in the course based on your analysis?
9. What aspects of the teaching environment would you primarily like todiscuss with other teachers?
Projects. Teaching by lectures and exercise sessions. The structure of the exercisesessions. Now the exercise leader is active.
10. What changes do you plan prior to next course round?
Make a choice in what to add as last part of the course. Maybe omit semidefiniteprogramming.
11. Do you have any suggestions on how to improve the questionnaire?
Some questions are repeated. (Maybe on purpose.)
12. Do you have any suggestions on how to improve these questions?
Maybe one particular question related to this course.
1 23
4
5
6
7
8
9
10111213
14
15
16
17
18
19
20
2122
SF2822 Applied nonlinear optimization
Learning Experience Questionnaire Footprint
Antal enk tsvar: 16Meningsfullhet − R d sektorBegriplighet − Gr n sektorHanterbarhet − Bl sektor
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-The projects were really good. -The feedback on the homework assignments was incredibly valuable and was a great display of commitment on the partof the educators, too.
I regularly spent time to reflect on my work (S18)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
7/6/14 3:16 PM
Page 3 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-During the projects I did not have that much knowledge of the theory for the upcoming exam whic I would have liked tohave worked on better.-Not as much as I would have liked to due to time constraints-Most of the time I tried to spend time the reflect on my work but when I faced some pressure of other course I had tospend time on other course.-I would have liked to do this more but had several other courses with huge assignment requirements. The theoryquestions and preparation suggested for the lecture would then have been valuable.-Not so much on "my work" as on the implications and applications of what I had learned to different problems.
I investigated some topics on my own during the course (S2)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-For the projects I investigated a lot.
7/6/14 3:16 PM
Page 4 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
-I read a couple of scientific publications during the course that employed methods covered in class, with greaterunderstanding than previously possible.
My background knowledge was sufficient to follow the course (S9)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-It helped a bit that I also followed linear optimization-As I did not take the linear optimization course, at first it was little bit difficult to me.But I had previous knowledge on lotof related topic. -Some minor lapses in my recollection of calculus and linear algebra could be felt at times, but had no major impact on myability to follow the course.
I felt that I was part of a community (S5)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
7/6/14 3:16 PM
Page 5 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
Maximum number of choices: 1
Comment:
-Only the projects make one feel part of a group otherwise it is mostly individual work.-Teacher was really helpful.Students were quite friendly. -Thanks to the group projects and friendly lecturer and tutor!-I felt that there were some fraternal ties with the students with whom I collaborated on the homework assignments, butnot so much otherwise.
I received regular feedback on my work from peers or teachers (S15)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-I received regular feedback.-I do feel like the feedback sessions in groups after projects are a bit of a waste of time though. Especially because allgroups with the same project more or less did the same thing. -As regular as could be expected. There was no continuous monitoring of my progress since the work was done in 3 majorefforts: homework assignments 1 and 2, then studying for the final exam.
The course was challenging in a stimulating way (S4)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
answer choice -3 -2 -1 0 +1 +2 +3 X
7/6/14 3:16 PM
Page 6 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-For the second project,it was my own choice. -To some extent, since the tasks were quite open-ended, additional time and effort could be devoted to achieving"advanced" goals.
I understood what the teachers were talking about (S10)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
answer choice -3 -2 -1 0 +1 +2 +3 X
7/6/14 3:16 PM
Page 7 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-Unforntunately I wasn't as prepared as I wanted for the lectures-sometimes it was required to refresh certain topics again.-The lectures were clear.-Except for some of the final stuff on SDP - maybe use the feedback sessions as regular lectures to teach this as it isinteresting and there wasn't time to come to grasps with it.-In retrospect it all made sense (after discovering the conceptual links that unified the different parts of the theory). At thetime, some parts seemed overly formulaic and repetitive. "Haven't we seen this before?" and "What is the difference fromthis to the previous part?" were some questions that arose during the lectures
My understanding of key concepts was given high priority in the course (S12)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-Some times.-Intuitively understanding something was given as high priority as rigorous mathematical proofs. This strikes a goodbalance for different types of students who approach learning in different ways.
I could practice and learn from feedback before any final assessment was made (S16)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
7/6/14 3:16 PM
Page 8 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-yes-This would have been unquestionably true if I had managed to put time into self study, earlier on (since the teachers werealways available for answering questions). Since we did get thorough feedback on our homework assignments, however. Ifeel that this is still very true, regardless of my shortcomings.
The course activities helped me to reach the learning objectives efficiently (S13)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-The course activities were really helpful.-Exercise sessions are a really nice way of ensuring we stay up to speed with the practical aspects of the course andunderstand all concepts! Thanks Tove.
I was able to learn by collaborating and discussing with others (S21)
7/6/14 3:16 PM
Page 9 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
I was able to learn by collaborating and discussing with others (S21)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-yes-Although maybe some more applied examples would be nice - some sort of "real world" scenario where optimization isused. My background is engineering and I felt it would have been nice to see these concepts applied where "min f(x)"actually has meaning beyond just an abstract function.
I was able to get support from peers or teachers if I needed it (S22)(Scale: -3: fully disagree... 0: neutral... +3: fully agree)
Average (for numeric-answers): 016 has answered of 41 (39%)Maximum number of choices: 1
Comment:
-yes-Since the grade depends, ultimately, on an exam, result will be subject to variability in individual skill, endurance,previous mathematical familiarity, etc. Knowing a priori what should be learnt to get a particular grade is therefore notdeterministic.
Open questions
What was the best aspect of the course?
12 has answered of 41 (29%)
Comment:
-the projects-Projects-Very interesting projects and very good teaching and support from Anders and Tove.-assignments-The second project that was the implementation of an interior point method and sqp-I liked the projects. -Learn how nonlinear optimization methods work.-The Projects and the exercise classes.-Projects allowing us to apply the concepts and develop our own models. Friendly and approachable lecturer and tutor. -The best aspect of the course is the teacher. His clarity of expression, wit and humor is captivating and his approachabledemeanour invites students to ask questions about parts that are unclear to them. The homework assignments are set ona good level of difficulty and are open-ended enough to satisfy students with different levels of ambition.-Project applications.-The projects were fun and very stimulating. It was great to work in groups and solve real world situations for a changefrom the other classes' boring made up projects.
What would you suggest to improve? 10 has answered of 41 (24%)
Comment:
-the lessons-More calculation in class-I would prefer finishing the last project a bit earlier to allow more time for studies towards the exam. -na-The exams should have a formula paper helper. I believe that the point is not to remember the formulars by hart but youshould be able to use them properly. Thus we can have a harder exam questions but with the algorithms and corespondingformulas given.
7/6/14 3:16 PM
Page 14 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
formulas given.-I thought it was good the way it was. Maybe also an implementation option for the first project.-There should be two or more lectures on GAMS. -I found the book "Numerical Optimization by Nocedal & Wright" to be a more comprehensive source than the suggestedliterature.
Although I already am passionate about the subject of optimization, I feel that other students could have been motivatedfurther by the seemingly endless number of "cool" applications to real-world problems out there.
I felt that the part about semidefinite programming was added as a bit of an afterthought to the syllabus. It seemed likewe were done with the main course but oh by the way there is this other thing: SDP. If I was at a restaurant I'd be tellingthe waiter, no, no, you must be mistaken, I'm sure I didn't order this, you've probably got the wrong table...check please!But no, he'd insist that I did order it, and he'd even be prepared to stand guard there, making sure I didn't leave anythingon my plate.
Finally, I dislike the focus on GAMS in the course due to it's commercial nature which feels out of place in an academicenvironment. I dislike it because it encourages students to think of optimization as a black box and because it frames thesolution step as a strictly "offline" batch process.-Not much!-Give more application examples and results demonstration.
What advice would you like to give to future course participants?
9 has answered of 41 (21%)
Comment:
-follow the course regulary-Do not forget to find time to study the theory for the exam continously throughout the course.-start early with assignments, we did that and it was great-To keep up with the schedule. I didn't have time a few times and it made it harder in the end.-If two or more lectures added on GAMS it will be very helpful for the new comers.-It goes without saying that you should definitely attend the lectures and exercise sessions and treat the course as a (veryfun) job.
If given the choice for homework assignments, pick one related to solver implementation. The lessons you will learn therewill have much greater synergistic effects toward your studies for the final exam, whereas the modelling tasks are likely tobe too specific for the particular problem to be of use later in the course. Although learning how to best model a problemmay be more useful in the long term, depending on your field of study.
Before you even begin, get your favorite linear algebra book (I can recommend the online video lectures on youtube byGilbert Strang, for example and the Khanacademy.org website) and make sure to intimately understand concepts such asrank, range-space, null-space, inverse, transpose, norm, inner product, matrix-vector products, matrix-matrix products,determinant, orthogonality, Jacobian, Hessian. For symmetric matrices, in particular: positive (semi-) definiteness,Cholesky decomposition, and the symmetric part of a square matrix.
for implementation assignments:If you know C++, have a look at the Eigen library for linear algebra. If you know Python, take a look at NumPy. OtherwiseMatlab/Octave are pretty forgiving for beginners and have the linear algebra parts built in.
General advice:Use Linux, write your papers in LaTeX, wear a helmet when biking and have fun.-Take the time to answer the lecture preparation questions.-Read chapters from the textbook alongside the lecture notes.-To work hard throughout the semester. Working at the last minute didn't work out great for me.
Is there anything else you want to add?8 has answered of 41 (19%)
Comment:
-no-One of the best courses taken at KTH and I will strongly recommend it to others!-great course! -no-I think every thing is good.-Thanks for the course!- -It might be interesting to have a lecture given by someone from a company whose work is related to the course.
7/6/14 3:16 PM
Page 15 of 15https://bilda.kth.se/servlets/AZ/courseId/11104?frame=lespollsumv&mode=upper&pid=22132877&hasAccess=true
På svenska Print Search Contact
Optimization andSystems Theory
KTH / Engineering Science / Mathematics / Optimization and Systems Theory
(The course material is available at Bilda in the form of pdf files.)
Linear and Nonlinear Optimization, second edition, by I. Griva, S. G. Nash och A. Sofer,SIAM, 2009.Information on how to order the book can be found here.Exercises in applied nonlinear optimization, 2013/2014. Available at Bilda.Supplementary course material in applied nonlinear optimization, 2013/2014.Available at Bilda.Lecture notes in applied nonlinear optimization, 2013/2014. Can be downloadedfrom this web page, see the schedule below. Also available at Bilda.GAMS, A user's guide. May be downloaded from the GAMS web site.GAMS. GAMS is installed in the KTH linux computer rooms. It may also be downloaded fromthe GAMS web site for use on a personal computer.Two project assignments that are handed out during the course, April 2 and April 23respectively.
Additional notes that may be handed out during the course are also included.
Course goals
After completed course, the student should be able to:
explain fundamental concepts of nonlinear programming;explain how fundamental methods for nonlinear programming work;illustrate how these methods work by solving small problems by hand calculations;starting from a suitably modified real problem, formulate a nonlinear program; make amodel in a modeling language and solve the problem;analyze the solutions of the optimization problem solved, and present the analysis in writingas well as orally;interact with other students when modeling and analyzing the optimization problems.
Examination
The examination is in two parts, projects and final exam. To pass the course, the following isrequired:
Pass project assignment 1, with presence at compulsory presentation lecture on WednesdayApril 23, and precence at the following dicussion session.
Pass project assignment 2, with presence at compulsory presentation lecture on MondayMay 12, and precence at the following dicussion session.Pass final exam.
Course registration
Due to the project based nature of this course, students must register no later than March 31.Registration lists will be circulated at the initial lectures. Each student must give an e-mail addresswhere he/she can be reached.
Project assignments
The project assignments are performed in groups, where the instructor determines the division ofgroups. This division is changed between the two assignments. Assignment 1 is carried out usingthe modeling language GAMS. For project 2, there is a choice between a modeling assignment, tobe carried out using GAMS, or a method assignment, to be carried out using Matlab. The projectassignments must be carried out during the duration of the course and completed by the abovementioned presentation lectures. Presence at the presentation lectures is compulsory. For passingthe projects, the following requirements must be fulfilled:
At the beginning of the presentation lecture, each group must hand in a well-written reportwhich describes the exercise and the group's suggestion for solving the exercise. Suitableword processor should be used. The report should be on a level suitable for anotherparticipant in the course who is not familiar with the group's specific problem.When handing in the report, each student should append an individual sheet with a briefself-assessment of his/her contribution to the project work, quantitatively as well asqualitatively.At the presentation lecture, all assignments will be presented and discussed. Each student isexpected to be able to present the assignment of his/her group. In particular, each studentis expected to take part in the discussion. The presentation and discussion should be on alevel such that students having had the same assignment can discuss, and students nothaving had the same assignment can understand the issues that have arisen and how theyhave been solved.Each group should make an appointment for a discussion session with the course leaders.There is no presentation at this session, but these sessions are in the form of a 20 minutesquestion session, one group at a time. There will be times available the days after thepresentation session. One week prior to the presentation lecture, a list of available times fordiscussion sessions will be made available at Doodle, reachable from the course home page.Each group should sign up for a discussion session prior to the presentation lecture.Each participant in the course must contribute to the work of the group. Each group mustsolve their task independently. Discussion between the groups is encouraged, but eachgroup must individually solve the assignments. It is not allowed to use solutions made byothers in any form. If these rules are violated, disciplinary actions in accordance with theKTH regulations will be taken.
Each project assignment is awarded a grade which is either fail or pass with grading E, D, C, B andA. Here, the mathematical treatment of the problem as well as the report and the oral presentationor discussion is taken into account. Normally, the same grade is given to all members of a group.
Final exam
The final exam consists of five exercises and gives a maximum of 50 points. At the exam, thegrades F, Fx, E, D, C, B and A are awarded. For a passing grade, normally at least 22 points arerequired. At the exam, in addidion to writing material, no other material is allowed at the exam.Normally, the grade limits are given by E (22-24), D (25-30), C (31-36), B (37-42) and A (43-50).
The grade Fx is normally given for 20 or 21 points on the final exam. An Fx grade may beconverted to an E grade by a successful completion of two supplementary exercises, that thestudent must complete independently. One exercise among the theory exercises handed outduring the course, and one exercise which is similar to one exercise of the exam. These exercisesare selected by the instructor, individually for each student. Solutions have to be handed in to the
instructor and also explained orally within three weeks of the date of notification of grades.
The final exam is given Tuesday June 3, 8.00-13.00.
Final grade
By identitying A=7, B=6, C=5, D=4, E=3, the final grade is given as
round( (grade on proj 1) + (grade on proj 2) + 2 * (grade on final exam) ) / 4),
where the rounding is made to nearest larger integer in case of a tie.
Preliminary schedule
"L" means lecture, "E" means exercise session, "P" means project sesstion.
Type Day Date Time Room SubjectL1. Mon Mar 24 10-12 Q24 Introduction. Nonlinear programming models. (pdf)
L2. Wed Mar 26 13-15 Q24 Optimality conditions for linearly constrained problems.(pdf)
L3. Fri Mar 28 15-17 Q24 Optimality conditions for nonlinearly constrainedproblems. (pdf)
Unconstrained optimizationFundamental theory, in particular optimality conditions.Linesearch algorithms, steepest descent, Newton's method.Conjugate directions and the conjugate gradient method.Quasi-Newton methods.(Chapters 11, 12.1-12.3 and 13.1-13.2 in Griva, Nash and Sofer.)Constrained nonlinear optimizationFundamental theory, optimality conditions, Lagrange multipliers and sensitivity analysis.Quadratic programming.Primal methods, in particular active-set methods.Penalty and barrier methods, in particular primal-dual interior methods.Dulal methods, local duality, separable problems.Lagrange methods, in particular sequential quadratic programming.(Chapters 3, 14.1-14.7, 14.8.1, 15.1-15.5, 16.1-16.3 and 16.7 in Griva, Nash and Sofer.)Semidefinite programmingFundamental theory.(Chapter 16.8 in Griva, Nash and Sofer. Separate article in the supplementary coursematerial. Fundamental concepts only.)
Welcome to the course!
Course web page: http://www.math.kth.se/optsyst/grundutbildning/kurser/SF2822/.
Published by: Optimization and Systems Theory, KTHAnders Forsgren, [email protected]
