ECE 6397, Fall, 2012
Selected Topic in OptimizationSelected Topic in Optimization
Zhu Han
Department of Electrical and Computer Engineering
Class 1
Aug. 27nd, 2012
OutlineOutline
Instructor information Motivation to study optimization Course descriptions and textbooks What you will study from this course
Objectives Coverage and schedule Homework, projects, and exams
Other policies Reasons to be my students Background and Preview
Instructor InformationInstructor Information
Office location: Engineering 2 W302 Office hours: M 10am-2pm or by appointment Email: [email protected] or [email protected] Phone: 713-743-4437(o), 301-996-2011(c) Course website:
http://www2.egr.uh.edu/~zhan2/ECE6397/ Research interests:
http://www2.egr.uh.edu/~zhan2
Wireless Networking, Signal Processing, and Security
http://wireless.egr.uh.edu/
MotivationsMotivations
Optimization is the mathematical discipline which is concerned with finding the maxima and minima of functions, possibly subject to constraints.
Interdisciplinary• Architecture• Nutrition• Electrical circuits• Economics• Transportation
• Examples:• Determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition• Allocating available funds among various competing agencies• Deciding which route to take to go to a new location in the city
Course DescriptionsCourse Descriptions
What is the optimization framework? What are the major types?
Convex vs. non-convex Continuous vs. discrete Centralized vs. distributed Deterministic vs. stochatic
What are the theorems? What are the applications? What are the state-of-art research? Can I find research topics? How to conduct research and write technique paper
Textbook and SoftwareTextbook and Software
Require textbook:
1. Zhu Han, Dusit Niyato, Walid Saad, Tamer Basar, and Are Hjorungnes, Game Theory in Wireless and Communication Networks: Theory, Models and Applications, Cambridge University Press, UK, 2011.
2. Steven Boyd’s videos for convex optimization3. Handout for parts of book, Zhu Han and K. J. Ray Liu,
Resource Allocation for Wireless Networks: Basics, Techniques, and Applications, Cambridge University Press, 2008.
4. Other handouts Require Software: MATLAB
http://www.mathworks.com/ or type helpwin in Matlab environment
ScheduleSchedule
• Introduction to optimization• Convex optimization
• Steven Boyd’s classhttp://www.stanford.edu/~boyd/cvxbook/• 30% of the class• Need to watch videos as homework (17 videos for 1 hour 15 min each) • Watch the video before the class!!!• Class is just review
• Integer/Combinatorial optimization• Might based on Georgia tech class• 15%
• Stochastic optimization• Might based on UIUC class• 15%
• Game Theory• based on my book• 40%
Homework, Project, and ExamHomework, Project, and Exam
Homework Watch videos for convex optimization Some other homework
Projects: simple MATLAB programs Based on the simulation at the end of each chapter
Exams Two independent exams Grading policy
Participations Attendance and Feedback Quiz if the attendance is low
Teaching StylesTeaching Styles
Slides plus black board Slides can convey more information in an organized way Blackboard is better for equations and prevents you from
not coming. Course Website
Print handouts with 3 slides per page before you come Homework assignment and solutions Project descriptions and preliminary codes
Feedback Too fast, too slow Presentation, Writing, English, …
Other PoliciesOther Policies
Any violation of academic integrity will receive academic and possibly disciplinary sanctions, including the possible awarding of an XF grade which is recorded on the transcript and states that failure of the course was due to an act of academic dishonesty. All acts of academic dishonesty are recorded so repeat offenders can be sanctioned accordingly.• CHEATING• COPYING ON A TEST• PLAGIARISM • ACTS OF AIDING OR ABETTING • UNAUTHORIZED POSSESSION • SUBMITTING PREVIOUS WORK • TAMPERING WITH WORK • GHOSTING or MISREPRESENTATION • ALTERING EXAMS• COMPUTER THEFT
Reasons to be my studentsReasons to be my students
Wireless Communication and Networking have great market Usually highly paid and have potential to retire overnight Highly interdisciplinary Do not need to find research topics which are the most
difficult part. Research Assistant Free trips to conferences in Alaska, Hawaii, Europe, Asia… A kind of nice (at least looks like) Work with hope and happiness Graduate fast
Different Kinds of OptimizationDifferent Kinds of Optimization
Optimization Formulation and AnalysisOptimization Formulation and Analysis
We discuss how to formulate the problem as an optimization issue.
Specifically, we study what the objectives are, what the parameters are, what the practical constraints are, and what the optimized performances across the different layers are.
The tradeoffs between the different optimization goals and different users' interests are also investigated.
The goal is to provide the students a new perspective from the optimization point of view for variety of problems in engineering fields.
Mathematical ProgrammingMathematical Programming If the optimization problem is to find the best objective function
within a constrained feasible region, such a formulation is sometimes called a mathematical program.
Many real-world and theoretical problems can be modeled in this general framework.
We discuss the four major subfields of the mathematical programming: – linear programming,
– convex programming,
http://www.stanford.edu/~boyd/cvxbook/
– nonlinear programming,
– dynamic programming.
What do we optimize?What do we optimize?
A real function of n variables
with or without constrains– Without constraint
– With constraint
),,,(21 n
xxxf
22 2),(min yxyxf
2
2),(min
1,52
2),(min
0
2),(min
22
22
22
or
or
yx
yxyxf
yx
yxyxf
x
yxyxf
Lets OptimizeLets Optimize
Suppose we want to find the minimum of the function
What is special about a local max or a local min of a function f (x)?
at local max or local min f’(x)=0
f”(x) > 0 if local min
f”(x) < 0 if local max
Review max-min for Review max-min for 33
Integer/Combinatorial Optimization Integer/Combinatorial Optimization The discrete optimization is the problem in which the decision variables
assume discrete values from a specified set.
The combinatorial optimization problems, on the other hand, are problems of choosing the best combination out of all possible combinations.
Most combinatorial problems can be formulated as integer programs.
Integer optimization is the process of finding one or more best (optimal) solutions in a well defined discrete problem space.
The major difficulty with these problems is that we do not have any optimality conditions to check if a given (feasible) solution is optimal or not.
We listed several possible solutions such as – relaxation and decomposition,
– enumeration,
– knapsack problem
– cutting planes.
Example of Integer ProgramExample of Integer Program(Production Planning-Furniture (Production Planning-Furniture
Manufacturer)Manufacturer) Technological data:
Production of 1 table requires 5 ft pine, 2 ft oak, 3 hrs labor
1 chair requires 1 ft pine, 3 ft oak, 2 hrs labor
1 desk requires 9 ft pine, 4 ft oak, 5 hrs labor
Capacities for 1 week: 1500 ft pine, 1000 ft oak,
20 employees (each works 40 hrs).
Market data:
Goal: Find a production schedule for 1 week tomaximize the profit.
profit demand
table $12/unit 40
chair $5/unit 130
desk $15/unit 30
Production Planning-Furniture Production Planning-Furniture Manufacturer: modeling the problem Manufacturer: modeling the problem
as integer programas integer program
The goal can be achieved
by making appropriate decisions.
First define decision variables:
Let xt be the number of tables to be produced;
xc be the number of chairs to be produced;
xd be the number of desks to be produced.
(Always define decision variables properly!)
Production Planning-Furniture Production Planning-Furniture Manufacturer: modeling the problem Manufacturer: modeling the problem
as integer programas integer program Objective is to maximize profit:
max 12xt + 5xc + 15xd
Functional Constraints
capacity constraints:
pine: 5xt + 1xc + 9xd 1500
oak: 2xt + 3xc + 4xd 1000
labor: 3xt + 2xc + 5xd 800
market demand constraints:
tables: xt ≥ 40
chairs: xc ≥ 130
desks: xd ≥ 30
Set Constraints
xt , xc , xd Z+
Solutions to integer Solutions to integer programsprograms
A solution is an assignment of values to variables.
A feasible solution is an assignment of values to variables such that all the constraints are satisfied.
The objective function value of a solution is obtained by evaluating the objective function at the given point.
An optimal solution (assuming maximization) is one whose objective function value is greater than or equal to that of all other feasible solutions.
There are efficient algorithms for finding the optimal solutions of an integer program.
Game Theory Game Theory Game theory is a branch of applied mathematics that uses models to study
interactions with formalized incentive structures (“games").
It studies the mathematical models of conflict and cooperation among intelligent and rational decision makers.
Rational means that each individual's decision-making behavior is consistent with the maximization of subjective expected utility.
Intelligent means that each individual understands everything about the structure of the situation, including the fact that others are intelligent rational decision makers.
We have discussed four different types of games, namely, the non-cooperative game, repeated game, cooperative game, and auction theory.
Slideshttp://wireless.egr.uh.edu/research.htm
The basic concepts are listed and simple examples are illustrated.
Game Theory OverviewGame Theory Overview What is game theory?
– The formal study of conflict or cooperation– Modeling mutual interaction among rational decision makers– Widely used in economics
Components of a “game”– Rational players with conflicting interests or mutual benefit– Strategies or actions– Utility as a payoff of player’s and other players’ actions– Outcome: Nash Equilibrium
Many types– Non-cooperative game theory– Cooperative game theory– Dynamic game theory– Stochastic game– Auction theory
Rich Game Theoretical ApproachesRich Game Theoretical Approaches Non-cooperative static
game: play once
– Mandayam and Goodman (2001)– Virginia tech
Repeated game: play multiple times– Threat of punishment by repeated game. MAD: Nobel prize 2005. – Tit-for-Tat (infocom 2003):
Dynamic game: (Basar’s book)– ODE for state– Optimization utility over time – HJB and dynamic programming– Evolutional game (Hossain and Dusit’s work)
Stochastic game (Altman’s work)
Prisoner Dilemma Payoff: (user1, user2)
Auction TheoryAuction Theory
Book of Myerson (Nobel Prize 2007), J. Huang, H. Zheng, X. Li
Term ProjectTerm Project Forming the group, 2-3 people per group
– Similar research background
Formulation of problems– Is that a problem?
– What is the objective and constraints
– What is best optimization techniques
Simulation– Matlab
– Victim algorithm
Analysis
Writing– It will be a headache for everybody
HomeworkHomework Convex optimization I
– http://www.youtube.com/watch?v=McLq1hEq3UY
– Watch before Wed. class!!!
Form Term project group– 2-3 people per group
– Let me know in the next class for grouping and basic interests