45
LIMITATIONS OF COMPUTING Introduction to Computer Engineering 2015 Spring by Euiseong Seo
LIMITATIONS OF COMPUTING Introduction to Computer Engineering 2015 Spring by Euiseong Seo
Where are we? Chapter 1: The Big Picture Chapter 2: Binary Values and Number Systems Chapter 3: Date Representation Chapter 4. Gates and Circuits Chapter 5. Computing Components Chapter 6. Low-Level Programming Languages and Pseudocode Chapter 7. Problem Solving and Algorithms Chapter 8. Abstract Data Types and Subproblems Chapter 9. Object-Oriented Design and High-Level Programming languages Chapter 10. Operating Systems Chapter 11. File Systems and Directories Chapter 12. Information Systems Chapter 13. Artificial Intelligence Chapter 14. Simulation, Graphics, Gaming, and Other Applications Chapter 15. Networks Chapter 16. The World Wide Web Chapter 17. Computer Security Chapter 18. Limitations and Computing
Contents
¨ Hardware ¨ Software ¨ Problems
Limits on arithmetic
¨ Precision ¤ The maximum number of significant digits that can be
represented ¤ With 5 digits precision, the range of the numbers we
can represent is -99,999 through +99,999
Limits on arithmetic
¨ What happens if we allow one of these digits (let’s say the leftmost one, in red) to represent an exponent?
¨ For example,
represents the number +3,245 * 103
Limits on arithmetic
¨ The range of numbers we can now represent is much larger
-9999*109 to +9999*109
but we can represent only four significant digits
¨ Significant digits ¤ Those digits that begin with the first nonzero digit on
the left and end with the last nonzero digit on the right
Limits on arithmetic
¨ The four leftmost digits are correct, and the balance of the digits are assumed to be zero
¨ We lose the rightmost, or least significant, digits
Limits on arithmetic
¨ To represent real numbers, we extend our coding scheme to represent negative exponents
¨ For example, ¤ 4,394 *10-2=43.94
Or ¤ 22 *10-4=0.0022
Limits on arithmetic
¨ Let the current sign be the sign of the exponent and add a sign to the left to be the sign of the number itself
Limits on arithmetic
¨ Representational error or round-off error ¤ An arithmetic error caused by the fact that the precision of the
result of an arithmetic operation is greater than the precision of the machine
¨ Underflow ¤ Results of a calculation are too small to represent in a given
machine ¨ Overflow
¤ Results of a calculation are too large to represent in a given machine
¨ Cancellation error ¤ A loss of accuracy during addition or subtraction of numbers of
widely differing sizes, due to the limits of precision (1+0.000001234-1) = 0?
Limits on arithmetic
¨ There are limitations imposed by the hardware on the representations of both integer numbers and real numbers ¤ If the word length is 32 bits, the range of integer
numbers that can be represented is -2,147,483,648 to 2,147,483,647
¤ There are software solutions, however, that allow programs to overcome these limitations
¤ For example, we could represent a very large number as a list of smaller numbers
Limits on arithmetic
Limits on components
¨ Although most errors are caused by software, hardware components do fail
Limits on communications
¨ Error-detecting codes ¤ Techniques to determine if an error has occurred during
the transmission of data and then alert the system
¨ Error-correcting codes ¤ Error-detecting codes that detect an error has occurred
and try to determine the correct value
Limits on communications
¨ Parity bit ¤ An extra bit that is associated with each byte, used to
ensure that the number of 1 bits in a 9-bit value (byte plus parity bit) is odd (or even) across all bytes
¤ Parity bits are used to detect that an error has occurred between the storing and retrieving of a byte or the sending and receiving of a byte
Limits on communications
¨ Odd parity requires that the number of 1s in a byte plus the parity bit be odd
¨ For example, ¤ If a byte contains the pattern 11001100, the parity bit
would be 1, thus giving an odd number of 1s ¤ If the pattern were 11110001, the parity bit would be
0, giving an odd number of 1s
¨ Even parity uses the same scheme, but the number of 1 bits must be even
Limits on communications
¨ Check digits ¤ A software variation of the same scheme is to sum the
individual digits of a number and store the unit’s digit of that sum with the number
¤ For example, given the number 34376, the sum of the digits is 23, so the number would be stored as 34376-3
¨ Error-correcting codes ¤ If enough information about a byte or number is kept, it
is possible to deduce what an incorrect bit or digit must be
Contents
¨ Hardware ¨ Software
¨ Problems
Complexity of software
¨ Commercial software contains errors ¤ The problem is complexity ¤ Software testing can demonstrate the presence of bugs
but cannot demonstrate their absence n As we find problems and fix them, we raise our confidence
that the software performs as it should n But we can never guarantee that all bugs have been
removed
Software engineering
¨ Remember the four stages of computer problem solving? ¤ Write the specifications ¤ Develop the algorithm ¤ Implement the algorithm ¤ Maintain the program
¨ Moving from small, well-defined tasks to large software projects, we need to add two extra layers on top of these; software requirements and specifications
Software engineering
¨ Software requirements ¤ A statement of what is to be provided by a computer
system or software product
¨ Software specifications ¤ A detailed description of the function, inputs,
processing, outputs, and special features of a software product; it provides the information needed to design and implement the software
Software engineering
¨ Testing techniques have been a running thread throughout this book
¨ They are mentioned here again as part of software engineering
Software engineering
¨ Use of SE techniques can reduce errors, but they will occur
¨ A guideline for the number of errors per lines of code that can be expected ¤ Standard software: 25 bugs per 1,000 lines of
program ¤ Good software: 2 errors per 1,000 lines ¤ Space Shuttle software: < 1 error per 10,000 lines
Formal verification
¨ The verification of program correctness, independent of data testing, is an important area of theoretical computer science research
¨ Formal methods have been used successfully in verifying the correctness of computer chips
¨ It is hoped that success with formal verification techniques at the hardware level can lead eventually to success at the software level
Notorious software errors
¨ AT & T Down for Nine Hours ¤ In January of 1990, AT&T’s long-distance telephone
network came to a screeching halt for nine hours, because of a software error in an upgrade to the electronic switching systems
Notorious software errors
¨ Mariner 1 Venus Probe ¤ This probe, launched in July of 1962, veered off course
almost immediately and had to be destroyed ¤ The problem was traced to the following line of Fortran
code: n DO 5 K = 1.3
¤ The period should have been a comma n DO 5 K = 1,3
¤ An $18.5 million space exploration vehicle was lost because of this typographical error
Contents
¨ Hardware ¨ Software ¨ Problems
Comparing algorithms
¨ Many ways!
¨ We use a special notation to compare algorithms ¨ Big-O notation
¤ A notation that expresses computing time (complexity) as the term in a function that increases most rapidly relative to the size of a problem
Big-O analysis
¨ Function of size factor N: ¤ f(n) = N4 + 100N2 + 10N + 50
¨ Then, f(N) is of order N4 – or, in Big-O notation, O(N4)
¨ For large values of N, N4 is so much larger than 50, 10N, or even 100 N2 that we can ignore these other terms
Big-O analysis
¨ Common orders of magnitude ¤ O(1) is called bounded time
n Assigning a value to the ith element in an array of N elements
¤ O(log2N) is called logarithmic time n Algorithms that successively cut the amount of data to be
processed in half at each step typically fall into this category
n Finding a value in a list of sorted elements using the binary search algorithm is O(log2N)
Big-O analysis
¨ O(N) is called linear time ¤ Printing all the elements in a list of N elements is O(N)
¨ O(N log2N) ¤ Algorithms of this type typically involve applying a
logarithmic algorithm N times ¤ The better sorting algorithms, such as Heapsort, and
Mergesort, have O(N log2N) complexity
Big-O analysis
¨ O(N2) is called quadratic time ¤ Algorithms of this type typically involve applying a
linear algorithm N times. Most simple sorting algorithms are O(N2) algorithms
¨ O(2N) is called exponential time ¨ O(N!) is called factorial time
¤ The traveling salesperson graph algorithm is a factorial time algorithm
Big-O analysis
Turing machine
¨ Turing machine ¤ A model Turing machine was developed in the 1930s
and consists of a control unit with a read/write head that can read and write symbols on an infinite tape
Turing machine
¨ Why is such a simple machine (model) of any importance? ¤ It is widely accepted that anything
that is intuitively computable can be computed by a Turing machine
¤ If we can find a problem for which a Turing-machine solution can be proven not to exist, then the problem must be unsolvable
Halting problem
¨ The halting problem ¤ Given a program and an input to the program,
determine if the given program will eventually stop with this particular input
¤ If the program doesn’t stop, then it is in an infinite loop and this problem is unsolvable
Halting problem
¨ Assume that there exists a Turing-machine program, called SolvesHaltingProblem that determines for any program Example and input SampleData whether program Example halts given input SampleData
Halting problem
Halting problem
¨ Now let’s construct a new program, NewProgram, that takes program Example as both program and data
¨ And uses NewProgram the algorithm from SolvesHaltingProblem to write “Halts” if Example halts and “Loops” if it does not halt
Halting problem
¨ Let’s now apply program SolvesHaltingProblem to NewProgram, using NewProgram as data ¤ If SolvesHaltingProblem prints “Halts”, program
NewProgram goes into an infinite loop ¤ If SolvesHaltingProblem prints “Loops”, program
NewProgram prints “Halts” and stops ¤ In either case, SolvesHaltingProblem gives the wrong
answer ¤ Because SolvesHaltingProblem gives the wrong answer
in at least one case, it doesn’t work on all cases
Halting problem
Classification of algorithms
¨ Polynomial-time algorithms ¤ Algorithms whose order of magnitude can be
expressed as a polynomial in the size of the problem
¨ Class P ¤ Problems that can be solved with one processor in
polynomial time
¨ Class NP ¤ Problems that can be solved in polynomial time with as
many processors as desired
Classification of algorithms
¨ Let’s reorganize our bins, combining all polynomial algorithms in a bin labeled Class P
Classification of algorithms
¨ Are class NP problems also in class P?
Picture sources of today’s slides
¨ Jones & Barlett Learning’s slides
