TDC368 UNIX and Network Programming

TDC368UNIX and Network Programming

Camelia Zlatea, PhD

Email: [email protected]

Week 1:

Course Introduction

Overview of UNIX Operating Systems Architecture

UNIX process concepts

mailto:[email protected]







UNIX Network Programming – TDC368-901 Page 2Spring 2003

Course Outline

Overview of UNIX operating system architecture. UNIX process concepts. Process control primitives

(fork, exec, exit, wait, waitpid). Filters and I/O Redirection. Interprocess

Comunications (IPC): pipes, FIFO's. Asynchronous Events. Signals. Signal Functions

(kill, system, sleep, raise, alarm, pause, abort). UNIX system calls: C library functions, C++ classes,

POSIX API's. UNIX File System: files, catalogs (inodes; types,size,

permissions). UNIX File I/O: unbuffered I/O functions

(open ,close,seek,umask,read,write). Standard I/O buffering (fopen, fclose, fflush). File

Sharing (dup,dup2,fcntl).


Course Outline (cont.)

Interprocess Communication: Shared Memories, Semaphores, POSIX Semaphores, lock files.

Socket API (TCP, UDP). Design Methods for Client/Server Applications UNIX shells. C-shell commands, shell variables. C-Shell

Programming. Managing UNIX projects (make). Compilation and

Debugging. UNIX shells: I/O redirection, filters(grep, sed), awk.


Objectives

Upon completion of this course, students– will understand the fundamental concepts of UNIX operating

system and use them to create various multitasking applications.

– Will be familiar with UNIX programming environment, system call interface, ANSI C and ANSI/ISO C++ functions and classes

– Will be familiar with Client/Server methods and Socket APIs

– Will be proficient in UNIX shell programming


Prerequisites

CSC 309 Basic understanding of operating systems and

programming techniques C/C++ will be used to illustrate the concepts presented in

the lecture material and for the implementation of the programming assignments and the project.


Texts

Dave Curry, UNIX Systems Programming for SVR4, O'Reilly & Assoc., Sebastopol, CA, 1996.

W. Richard Stevens, UNIX Network Programming - Interprocess Communication, Volume 2, Second Edition, Prentice Hall PTR, Upper Saddle River, NJ, 1999.

W. Richard Stevens, Advanced Programming in the UNIX Environment, Addison-Wesley, Reading, MA, 1992.

John Shapley Gray, Interprocess Communications in Unix, The Nooks & Crannies, 2nd ed. Prentice Hall PTR, NJ, 1998.

HP-UX man pageshttp://docs.hp.com/hpux/onlinedocs/B2355-90682/B2355-90682.html (system calls)


Other References

Daniel Gilly, Unix in a Nutshell, O'Reilly and Associates, Inc., Sebastopol, CA, 1992.

K.A.Robbins and S. Robbins, Practical UNIX Programming - A Guide to Concurrency, Communication, and Multithreading , Prentice Hall PTR, NJ, 1996

Ellie Quingley, UNIX Shells by Example, Prentice Hall PTR, NJ, 1997

Prabhat K. Andleigh, UNIX System Architecture, Prentice-Hall, Englewood Cliffs, NJ, 1990.

Andrew Oran and Steve Talbot, Managing Projects with make, O'Reilly and Associates, Inc., Sebastopol, CA, 1991.

Brian W. Kernighan and Dennis M. Ritchie, The C Programming Language, Second Edition, Prentice-Hall, Englewood Cliffs, NJ, 1989.


Class Work and Grading

Programming Assignment 1 20%

due by April 22th


due by May 6th

Midterm 20% May 13th


due by May 27th

Final Project 20% due by June 10th


Programming Environment

All students will get an account on hawk.depaul.edu server (running HP-UX).

Optional, you can use Linux system

Project submissions via COL (dlweb.cti.depaul.edu )– use Submit Assignments link

Grading will be done on HP-UX or Linux– An experimental report is required for each programming project

» The the report should include the program design description and the test cases and test scenarios for the application.


TDC368 – Network Programming

Instructor: Camelia Zlatea, PhDOffice Hours: T 4:45-5:45PM; 9:00-9:30PM

Where: Rm. 429, ext. x26149 (during office hours)

or leave a message to setup a conference: +630-979-3621

Email: [email protected]

Web: http://condor.depaul.edu/~czlatea/TDC368/ (*)

Note:

* In addition, course materials will be available from dlweb.cti.depaul.edu


Class Communication

COL (dlweb.cti.depaul.edu) will be used primarily for:– Links to course materials/assignments

– Grades

– Announcements

– Email/Communication

– Newsgroup

Asking questions – Send me an email at [email protected] OR

– Use dlweb.cti.depaul.edu and post questions to class forum

• Anybody may reply (including your classmates)

• Everyone benefits from common issues


UNIX - Brief History

1969, Ken Thompson, Bell-Labs.

– UNIX OS written in Assembly on PDP-7 (DEC) 1973, K.Thompson&D.Richie

– UNIX re-written in C 1975, Bell-Labs. offered UNIX to educational

institutions, at minimal cost


UNIX Features

Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis


UNIX Major Versions

AT&T UNIX version V

– 1983 /SV, 1986 /SVR3.0 AT&T and Sun Microsystems

– 1988 /SVR4.0 - tries to unify Berkeley UNIX & other Berkeley UNIX, CSRG at U. of California

– BSD (Berkeley Software Distribution)


UNIX Standards

System V Interface Definition (SVID), AT&T Portable Operating System Interface for Computer

Environments (POSIX), based on SVID, IEEE ANSI C, American National Standard Institute ANSI/ISO C++ Standard (draft)


UNIX Implementations

Solaris - Sun Microsystems

– SunOS (later called Solaris), Solaris 2.x based on SVR4

HP-UX, Hewlett-Packard, SVR2 Linux, Linus Torvalds, free distribution, PC-based AIX, IBM, similar to SVR4 IRIX, Silicon Graphics, SVR4


UNIX Implementations

UNIX-like– same facilities as SV, does not use entirely the same

code– Mach (CMU), kernel is based on OO model server– Next (implementation inspired by Mach with MacOS

UI) UNIX-alike

– portability is no longer supported – attributes not compliant with SV or BSD– OS/9(MicroWare, Motorola 680xx, RT kernel)– QNX(Quantum Software, Intel, RT processing)


UNIX Architectural Overview

Hardware Architecture

Kernel UNIX

System Calls Interface

Shell’s Executable Programs

pipe, filters Commands

Applications Network ApplicationsDBMS Utilities


UNIX Features

Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis


Multi-process/Multi-user architecture

Virtual Machine– timesharing OS

– process, process quantum, process states

Kernel, base OS– manages all HW dependent functions

– users have no direct access to it

System Calls Interface– service routine performing user requests


UNIX Processes

A process may create sub-processes– fork();

A process may terminate– exit();

A process may put itself to sleep temporarily– sleep(20);

– pause();

– wait();

Processes– synchronization mechanisms

– communication mechanisms


UNIX Threads

Multiple Processes - concurrency at OS level Multiple Threads - concurrency at process level

– thread = flow of control in a process

– multiple threads (stream of instructions) are executed within the same process

– threads share code & data (address space)

– threads have their own execution stack, PC, register set and states

– context switches are avoided

– efficient mapping on multi-processor machines


Some Unix Terms

TRAP INSTRUCTION– switches a system call from user to kernel mode

– handles error conditions

– synchronous execution

– Ex: division by zero/overflow

INTERRUPT– used by peripherals to request services from the devices handles

– asynchronous execution

SIGNALS software notification of an event example:

– when typing “ctrl-c” an interrupt is generated for the keyboard driver, which notifies the process by sending a signal


UNIX Kernel - model

Hardware

System Call Interface (Library Routines)

Process Memory File System I/O Services Mgmt. Mgmt.

Scheduler Device Drivers I/O Buffers

User Processes

Kernel

Space

User

Space


Apps./Utilities

System Calls

Hardware InterruptsUNIX KERNEL

Hardware


UNIX Kernel - low level

Process representation, scheduling, dispatching Memory allocation and de-allocation Interrupt handling Low level device control Disk Mgmt., data buffering Process synchronization and IPC


Kernel - services level

Maps user-level requests with device driver actions A user system call is translated to a call of the kernel routine,

providing that requested service Type of Services:

– process creation and termination

– I/O services

– UNIX file system services

– terminal handling services


Kernel - services level

User program:

system(“ps -e”);

System calls

actions:

fork();

//process creation service

execl(“/bin/ps”, “ps -e”);

//code execution

Executable Programs

Library SystemFunctions Calls

Kernel

Hardware


System Call Interface level

A user mode process is translated into a protected kernel mode process

Now, program can call kernel routines


User Processes level

User processes running:– shells

– Unix commands

– utilities

– application programs


UNIX and POSIX API Library Functions ANSI C Standard ANSI/ISO C++ Standard

UNIX Standards


UNIX and POSIX API

UNIX API - system calls UNIX API are called by

– C library functions and

– C++ standard classes

Example: iostream class

Program confirms POSIX.1?#define _POSIX_SOURCE

or

% CC -D_POSIX_SOURCE *.C


UNIX and POSIX API

API set to perform:– determine system configuration and user information

– file management

– process creation and management

– inter-process communication

– network communication


UNIX and POSIX API

User Process (User Mode of Execution)

Kernel mode of execution

UNIX API’s level

an API is invoked

API executed

in protected mode

API execution

completed


UNIX and POSIX API

Context Switch from user to kernel mode – more overhead than library functions, for the same task

– I/O lib.functions are buffered


UNIX and POSIX API

UNIX API - system calls UNIX API are called by

– C library functions and

– C++ standard classes

Example: iostream class

Program confirms POSIX.1?#define _POSIX_SOURCE

or

% CC -D_POSIX_SOURCE *.C


UNIX and POSIX API

Development elements: Headers

– <unistd.h>

– <sys> (/usr/include/sys directory)

– <stdio.h>

function perror - system diagnostic message for an API execution failure

Object Code – libc.a and libc.so (on most systems)

– other special libraries, libsocket.a and libsocket.so


UNIX and POSIX API

#include <stdio.h>

void perror(const char *s);

#include <errno.h>

int errno;

#include <string.h>

char *strerror(int errnum);


Library Functions

Library of functions

– object code of the functions stored in archives (library files)

– run-time library routines UNIX archive ar utility manage library files.

Displaying the table of contents of the standard library:

ar -t /usr/lib/libc.a


ANSI C Standard

Standard C library functions:

– memory management

– get system information

– get/set time information

– data conversion, processing Advantages:

– portability

– maintainability

– cost effective for application design Disadvantages:

– duration of execution

– weak support for time-critical apps.


Standard C library functions

Header files directory: /usr/include Header files:

- <stdio.h> - <stdlib.h>

- <string.h>

- <memory.h> - <malloc.h>

- <time.h>

- <assert.h> - <stdarg.h>

- <getopt.h> - <setjmp.h>

Object code directory: /usr/lib– libc.a , libc.so



<stdio.h> - declares FILE data type functions/macros -manage stream files

open/close stream files: fopen/fclose

read/write block data: fread/fwrite

read/write lines: fgets/fputs

write formatted data: fprintf

positions/returns stream file location:fseek/ftell

macros: feof, ferror

unnamed pipes: popen/pclose


Example: ls -l|grep tdc368

#include <stdio.h>int main(){ /* execute the command */

FILE *cmd = popen(“ls -l|grep tdc368”,”r”); if (!cmd) {

perror(“popen”); return 1; }

char answer[256];/* read grep process execution output */while(fgets(answer, sizeof(answer),cmd))

fputs(answer, stdout); /* echo */pclose(cmd);return 0;

}



<stdlib.h> - declares set of functions for data conversions, get/set env. variables, shell command execution, process termination– atoi, atof, atol, strtod, strtol

– rand, srand

– getenv, putenv

– system

– exit


Example: ls -l|grep tdc368

#include <stdlib.h>

int main()

{ /* execute the command */

if (system(“ls -l|grep tdc368”)== -1) {

perror(“system”); return 1;

}

return 0;

}


Example: getenv/putenv

#include <iostream.h>

#include <stdlib.h>

int main()

{

char *env = getenv(“REMOTEHOST”);

cout << “\REMOTEHOST\” is:”<<env<<‘\n’;

if (putenv(“CC=c++”))

cer<<“putenv of CC failed\n”);

return 0;

}


Example: <malloc.h>

//C style

char* adr1 = (char *)malloc(4096);

free(adr1);

//C++ style , no use of malloc !

char* adr2 = new char[4096];

delete adr2;



<time.h> - declares set of functions for system clock query

time - #sec since 01/01/70 (UNIX birth)

ctime - current local time

localtime

gmtime

asctime

mktime

clock - (ANSI C) , #microsec since the process first called clock


Example: <time.h>

// evaluate process execution time

#include <iostream.h>

#include <time.h>

main() {

time_t tick = CLOCKS_PER_SECOND;

clock_t start_time = clock(); // start timer

/* process code here */

clock_t duration = clock() - start_time;

cout <<“Duration:”<<(duration/tick)<<endl;

}



<assert.h> - declares a macro to assert conditions or invariants of the program– easier program debugging

– compiler option -DNDEBUG take them out

or

#ifndef NDEBUG

#define assert(p) {if (!(p)) {\

fprintf(stderr,”Assertion failed: \

file:\”%s\”,line %d\n”,\

_FILE_, _LINE_); exit(1); }

#endif


Example: <assert.h>

#include <fstream.h>

#include <string.h>

#include <assert.h>

int main(int argc, char * argv[])

{

assert(argc >1); // should have 1 arg

ifstream ifs(argv[1])

assert(ifs.good()); // should be opened OK

char *adr = new char[strlen(argv[1])+1];

assert(adr); // should not be NULL

}


ANSI/ISO C++ Standard

1980, Bjarne Stroustrup, C++ 1994, ANSI/ISO C++ standard (draft) C++ compiler (AT&T C++ V3.0) support:

– C++ classes, derived classes

– virtual functions, operator overloading

– templates classes, functions

– exception handling

– iostream classes


Compilation

cc -o file file.c

file

Man Pages

man cc

man sys_call

man shell_cmd


Processes

Process - a program in execution– process - active entity

– program - passive entity (binary file)

Address Space - list of memory locations from where a process reads/writes (code/data/stack)

Set of registers (PC, SP, ...) Process Table - linked list of structures associates w/ processes System Calls - interface between OS and User process


Process Control Block (process attributes)

Process State – new, ready, running, blocked, terminated

Process Image Map Pointer Process ID

– assigned at process creation

Program Counter (PC) – address of next instruction to be executed

CPU Registers (saved process context) List of Open File Descriptors I/O Devices Attached CPU Scheduling Info (priority)


Process Image Map

Proc. n

Proc. 1Process Image

Process Table

Text/Code Segment

Data Segment

Stack Segment

Process Control Block


Process Memory Map

High Address

System Memory Map

process 0process 1.....

process i....

program text segment

initialized static data

uninitialized static data

heap

stack

argc argv; environment

System Process Info.System Stack

User addressablearea

Systemaddressablearea

Low Address

UNIX Data Structures

u area

Buffers

Kernel code

Low level device drivers


Example:

/* Display Segment Address Information

*/

#include <stdio.h>

extern int etext,edata,end;

void main(void) {

printf(“etext: %6X\t edata: %6X \t end: %6X \n”, &etext, &edata, &end);

}


Process States

New - process created ( Ex: fork(); ) Ready - process is waiting to be

assigned to processor (inserted in ready queue) Running - instructions are being executed Blocked - wait for events to occur (inserted

in queue) Ex: wait(); pause(); Terminated - normal/abnormal termination

(exit();)


Process Model

New

Ready

Blocked/Suspended

RunningUser Mode

Terminated

exitsleep

wakeup

dispatch

created

QuantumExpired

RunningKernel Mode

System Call

InterruptReturn

InterruptInterrupt return


Context of a Process– process state (defined by it’s code)

– value of u-area

– values of registers the process uses

– contents of user and kernel stacks

– is associated with process image map

Context Switching– system executes a process in the context of the process

– when the kernel decides to execute another process, it does context switching

– kernel saves enough information such that it can later switch back to the first process and resumes its execution

Mode Switching– moving from user to kernel mode

– kernel save information to return to user mode


User mode– processes in use mode can access their own instructions and data;

NOT kernel or other process’s code or data

Kernel mode– process can access system(kernel) code and data and user addresses

– Kernel is part of each process

– Kernel executes on behalf of the process

P1 P2 P3 P4

Kernel Mode

User Mode

K K

U U

OSHW


Context Switching

P1 P2

Save state in PCB1

Reload state from PCB2

Save state in PCB2

Reload state from PCB1

OS


Switching the CPU to another process by saving the state of the old process (PCB) and load the state of the new process (PCB)

Pure Overhead Performance Bottleneck Avoid Overhead of Context Switching by introducing new

structures:

THREADS

Context Switching


Multitasking

Sequential Execution

Context Switching

Date post:	30-Jan-2016
Category:	Documents
Upload:	darci
View:	47 times
Download:	0 times

TDC368 UNIX and Network Programming

Documents