The Operating System Project

Start Here

Version 4.10: September 2014

2

Table of ContentsWhat to do in Week 1 – Test0• Brushing up your C programming skills.• Compiling the program.• Understanding how the program flows.• What does Test0 do?• What do you need to change?• Step by step example.What you need to do - Test1a• What’s the goal of Test1a?• Summary information.• Where do you find resources to help you?

3

Table of Contents(2)What you need to do - Test1b• What’s the goal of Test1b?• Summary information• Where do you find resources to help you?

What you need to do – Test2a• What’s the goal of Test2a?• Summary information• Where do you find resources to help you?

4

Brushing up your C programming skills.

There are many places you can review C programming. The document listed below looks at some of the differences between C and Java. The concept of pointers is where most java programmers have difficulty. If you need more information, the web is your friend.

C_By_Example.ppt can be found on the Project Home Page – the same place you found this Start_Here.ppt.

5

Compiling the program.The first thing you need to figure out is the environment where you will be doing this project.

If you’re a LINUX fan, then life is easy since gcc is installed on every computer. I would recommend that as a good way to get started. If you’ve been programming on Windows, then I’d recommend the free Visual Studio version designed for students – it’s great though may be a bit formidable to start with.

I've built this code with Eclipse on Windows, with gcc installed on Windows, and with a standard gcc on Linux. It worked for these three environments.

The first thing you need to do is define whether you will be building on Windows or Linux. In the file global.h you will find the following two lines:

// #define       NT#define       LINUX

One of these lines should be commented out so that the other type of system is used; This is necessary because the threads implementations on the two operating systems are very different.

After you’ve moved the files from the webpage into a new directory, compile the program:

>gcc –g *.c -lm -o os Windows

>gcc –g *.c -lm –lpthread -o os Linux

This will create an executable called os. Because of the –g, you can debug this program.

6

Compiling the program.Executing the program will give the following output:

This is Simulation Version 4.00 and Hardware Version 4.00.

Program called with 2 arguments: C:\Users\jb\workspace\Z502\Debug\Z502.exe test0Calling with argument 'sample' executes the sample program.This is Release 4.00: Test 0SVC handler: get_time Arg 0: Contents = (Decimal) 4300384, (Hex) 419E60Time of day is 0SVC handler: term_procArg 0: Contents = (Decimal) -1, (Hex) FFFFFFFFArg 1: Contents = (Decimal) 4300388, (Hex) 419E64ERROR: Test should be terminated but isn't.ERROR: Simulation did not end correctly

If you get this result, you know your compilation was successful. Your task now for Test 0 will be to make this code work right, so it doesn’t produce the errors you see here.

7

Compiling the program.

Native Hardware Platform(Intel I5, etc.)

Native Operating System(Windows 7 , Linux, etc.)

Hardware Simulator(z502.c)

Operating System(base.c, state_printer.c)

The executable you just compiled

Test.ccontains test0 test1a test1b … test2a ...

All elements inside the heavy box are ina single process, running several threadsof execution.

All I/O devices in the program are simulatedentities. This includes the timer deviceand the disk devices.

Try to treat the Hardware Simulatoras a “black box” and use the architecturespecification instead.

You just compiled the program in the solid box. It includes a hardware simulator, the beginnings of an operating system that you will expand, and test cases that drive your development of the OS.

8

Understanding how the program flows.The next slides describe the starting code that’s given to you (what you’ve already compiled). It shows how the program flows. The important actions are:

Test0 in test.c contains system calls – requests for service from the Operating System.

Those system calls come to the routine svc( ) in base.c. This is in the OS – you’re writing the OS so you own this code.

In svc, you call (make a subroutine call) to the hardware in order to implement the action requested by Test0.

9

The Execution of test0

test.c

Hardware Z502.c

base.c

Z502Clock(Memory IO) Z502SwitchContext

SVC

OS_Init

test0 main

Z502Halt

4

10

9

7

8

6

5

3 2

1

10

The Execution of test0

4

9

8

7

6

5

3

2

1 All C programs start in main(). A temporary context is created and the simulation starts by requesting to run on that context.os_init is a routine in the operating system. For right now, all it does is create a context that will allow test0 to run.

We go out to test0. It is time to run the user code.

Test0 does a system call get_time_of_day. A system call produces a software interrupt that causes execution to go to svc, the software service routine.svc must get the time in order to service the system call. It calls the hardware to do that. It passes by reference a variable in which the time can be placed.Z502_CLOCK is a hardware routine that keeps track of the time. It passes back this time to svc.svc passes back the time to test0. test0 prints out that time as part of its code.

test0 does a terminate_process system call – it’s all done with its job. It makes this call and again the execution ends up back in svc.svc must handle this terminate_process request. Eventually this code will be more complicated, but for right now, since there’s nothing else for the OS to do, it simply ends the simulation by halting the processor.

11

What does Test0 do?void test0(void) { printf("This is Release %s: Test 0\n", CURRENT_REL); GET_TIME_OF_DAY(&Z502_REG1);

printf("Time of day is %ld\n", Z502_REG1); TERMINATE_PROCESS(-1, &Z502_REG9);

// We should never get to this line since the TERMINATE_PROCESS call // should cause the program to end. printf("ERROR: Test should be terminated but isn't.\n");} // End of test0

There are two system calls:

GET_TIME_OF_DAY( &Z502_REG1 ); Get the time the hardware thinks it is. This is NOT in any normal units like seconds or whatever. Note that following the C convention, we’re passing the ADDRESS of the variable Z502_REG1 (that’s what the “&” does.) Then in the next line

printf( "Time of day is %d\n", Z502_REG1 );

the value that’s in the variable is used in the printf statement.

TERMINATE_PROCESS( -1, &Z502_REG9 ); has two arguments. The “-1” says terminate the current process. The &Z502_REG9 gives the address of a variable that the OS can use to return an error.

Appendix C is where you will find a description of the system calls. Syscalls.h contains the macros that implement these system calls.

12

Test0 - What do you need to change?To make Test0 work, “all” you need do is change code in svc(). Let’s start by looking at the original code:

void svc( SYSTEM_CALL_DATA *SystemCallData ) { short call_type; static short do_print = 10; short i;

call_type = (short)SystemCallData->SystemCallNumber; if ( do_print > 0 ) { printf( "SVC handler: %s\n", call_names[call_type]); for (i = 0; i < SystemCallData->NumberOfArguments - 1; i++ ){ //Value = (long)*SystemCallData->Argument[i]; printf( "Arg %d: Contents = (Decimal) %8ld, (Hex) %8lX\n", i, (unsigned long )SystemCallData->Argument[i], (unsigned long )SystemCallData->Argument[i]); } do_print--; }} // End of svc

• SystemCallData - a data structure containing everything we know about this system call.• Call_type– a variable contains the type of system call that’s being passed to svc. In svc,

this variable, as well as the arguments requested by the system call in test0, are printed out so you can see them.

• The do_print variable is here simply to do some initial printout, but then not clutter up printouts when there are many system calls. You can see how it works from the code.

13

Test0 - Step by step example.void svc SYSTEM_CALL_DATA *SystemCallData ) { short call_type; static INT16 do_print = 10; INT32 Time;

call_type = (short)SystemCallData->SystemCallNumber; if ( do_print > 0 ) { // same code as before } switch (call_type) { // Get time service call case SYSNUM_GET_TIME_OF_DAY: // This value is found in syscalls.h CALL( MEM_READ( Z502ClockStatus, &Time ) ); *(INT32 *)SystemCallData->Argument[0] = Time; break; // terminate system call case SYSNUM_TERMINATE_PROCESS: Z502Halt(); break; default: printf( "ERROR! call_type not recognized!\n" ); printf( "Call_type is - %i\n", call_type); } // End of switch} // End of svc

Remember that the call_type is which type of system call is being executed. The switch statement means a different case handles each call.

This is easy – all I did was find the code in sample.c that does this same call to the hardware. Then I copied it here! At this point, it’s magic.

We’re returning the time to the caller (in test0). The Argument[0] could be pointing to 32 bits or 64 bits. We cast it to 32 since the data value is 32 bit. Then the “*” on the front says this is a pointer. (This is not obvious stuff if you’re new to C).

In this test, when Test0 says it wants to terminate, there’s nothing more to do, so we simply call the hardware to say we’re done. Note how this is in a different case statement from the GET_TIME call.

If a bogus system call number comes in here, we want to know about it and report an error.

14

Test0 - Step by step example.

This is Simulation Version 4.00 and Hardware Version 4.00.

Program called with 2 arguments: C:\Users\jb\workspace\Z502\Debug\Z502.exe test0Calling with argument 'sample' executes the sample program.This is Release 4.10: Test 0SVC handler: get_time Arg 0: Contents = (Decimal) 4300384, (Hex) 419E60Time of day is 45SVC handler: term_procArg 0: Contents = (Decimal) -1, (Hex) FFFFFFFFArg 1: Contents = (Decimal) 4300388, (Hex) 419E64Hardware Statistics during the SimulationContext Switches = 1: CALLS = 13: Masks = 0The Z502 halts execution and Ends at Time 50Exiting the program

Here’s what the execution looks like after the code has been changed.Note that the time of day is reported as “65” in this case (you’re number may be different. Note also that the simulator says that the test ended happily.

15

Test1a - What you need to do• What does Test1a do?• Summary information.• Starting Architecture of the Simulator

Environment; the interrupt_handler• Implementation of Test1a:– Step 1– Step 2– Step 3Complete one step at a time. This will get you ready

for test1b.

16

What does Test1a do?void test1a(void) { static INT32 SleepTime = 100; static INT32 time1, time2;

printf("This is Release %s: Test 1a\n", CURRENT_REL); GET_TIME_OF_DAY(&time1);

SLEEP(SleepTime);

GET_TIME_OF_DAY(&time2);

printf("Sleep Time = %d, elapsed time= %d\n", SleepTime, time2 - time1); TERMINATE_PROCESS(-1, &Z502_REG9);

printf("ERROR: Test should be terminated but isn't.\n");} // End of test1a

Let’s look at this code. A lot is the same as test0. There are two calls to GET_TIME_OF_DAY, and one call to TERMINATE_PROCESS. The only new piece is the SLEEP.

There is one new system call:SLEEP( TimeToSleep ); With this call, we’re not getting a value returned to us – we’re simply passing to the OS, the amount of time we want to “sleep”. We don’t want control to come back to this code for at least TimeToSleep time units.

Appendix C is where you will find a description of the system calls. Syscalls.h contains the macros that implement these system calls.

17

Test1a – Summary Information Hardware Registers

Name Bits Usage

Z502_REG1 … Z502_REG9 32 General purpose

Z502 _PAGE_TABLE_ADDR 32 Points to page table

Z502 _PAGE_TABLE_LENGTH 32 Length of page table in 32 bit entries

Z502 _CURRENT_CONTEXT 32 Handle for current context

Only the REG are of interest to you right now.

18

Test1a – Summary Information Interrupt Handling

An OS is just a program waiting for someone to give it something to do. It’s the hardware that transfers control into the OS. There are three ways to do this.– Interrupts (starts executing at interrupt_handler in base.c)

• TIMER_INTERRUPT from the delay timer• DISK_INTERRUPT from disk 1, 2, ...

– Faults (starts executing at fault_handler in base.c)• INVALID_MEMORY fault• CPU_ERROR fault• PRIVILEGED_INSTRUCTION fault

– Software Traps (starts executing at svc in base.c)• SOFTWARE_TRAP for each system call

19

Test1a – Summary Information System Modes

Modes have to do with privileges. The code executing in User mode has access to the code in Test.c and access to data associated with the test. In Kernel Mode, the code can see, touch, smell, and modify ANYTHING!

– User Mode • Address space for user programs is divided into

– C code “program” memory for instructions and for local variables. – User “data” memory, referenced through a virtual address space, and called MEMORY. You

don’t need to know this until Test2a.

– Kernel Mode• Instruction set includes C language instructions, plus

– access to all the Z502 registers– access to the privileged instructions of the Z502 hardware instruction set

» I/O primitives» memory primitives» context switching primitives

– These are all available through provided macros

20

Test1a – Summary Information Hardware Actions on Interruption

1. User registers are saved in Z502 Hardware Context – this is done by the hardware so you don’t have to worry about it.

2. Execution mode is set to kernel – after all, we’re now running in the OS!

3. Hardware begins execution at interrupt_handler when the hardware has something to communicate (i.e., it took an error, it’s successfully completed its work, etc.)

4. The interrupt_handler queries the hardware to find out about the interrupt. There are three requests to the hardware. These are explained in excruciating detail in Appendix A, the Architecture Specification – see Section 5.3.

5. The MEM_READ() and MEM_WRITE() hardware references:a) ask for the device that caused the interrupt b) say that future hardware queries will reference that device, and then c) get the status of that device.

21

Test1a – The interrupt_handlervoid interrupt_handler( void ) { INT32 device_id; INT32 status; INT32 Index = 0; static BOOL remove_this_in_your_code = TRUE; /** TEMP **/ static INT32 how_many_interrupt_entries = 0; /** TEMP **/

// Get cause of interrupt – which device caused the interrupt MEM_READ(Z502InterruptDevice, &device_id ); // Set this device as target of our query MEM_WRITE(Z502InterruptDevice, &device_id ); // Now read the status of this device – you need to interpret this to // make sure no error occurred. MEM_READ(Z502InterruptStatus, &status );

/** REMOVE THE NEXT SIX LINES **/ how_many_interrupt_entries++; /** TEMP **/ if ( remove_this_in_your_code && ( how_many_interrupt_entries < 20 ) ) { printf( "Interrupt_handler: Found device ID %d with status %d\n", device_id, status ); } // Clear out this device - we're done with it MEM_WRITE(Z502InterruptClear, &Index );} /* End of interrupt_handler */

22

Test1a – The interrupt_handlervoid interrupt_handler( void ) { INT32 device_id; INT32 status; INT32 Index = 0; static BOOL remove_this_in_your_code = TRUE; /** TEMP **/ static INT32 how_many_interrupt_entries = 0; /** TEMP **/

// Get cause of interrupt – which device caused the interrupt MEM_READ(Z502InterruptDevice, &device_id ); // Set this device as target of our query MEM_WRITE(Z502InterruptDevice, &device_id ); // Now read the status of this device – you need to interpret this to // make sure no error occurred. MEM_READ(Z502InterruptStatus, &status ); // Your code will go here. It should include a switch statement for // each device – you have only the timer right now. // Check that the timer reported a success code. // Call a routine you write that does the work described later. MEM_WRITE(Z502InterruptClear, &Index );} /* End of interrupt_handler */

23

Test1a – Summary Information Hardware Context

• The context is the state of the executing CPU; essentially its registers.

• The Hardware context is really just the set of registers , the current state of the process, plus an entry address.

• Your OS only deals with the handle(address) to a context. You store this in the process control block. You don’t EVER need to know what’s in that context. (This is an example of data hiding.)

• Z502 Operations for manipulating contexts– Z502MakeContext(handle, start address, kernel flag)– Z502DestroyContext(handle)– Z502SwitchContext(save/destroy flag, handle)

24

Writing Test1aWrite this test in multiple stages – get each stage working before you start

the next one; take baby steps.• Stage 1: In svc for the SLEEP system call, you should:

a) Start the timer (see sample.c for an example of this – see also Appendix A for the API for the timer.

b) Call Z502_IDLE()c) Control will not pass back from IDLE to it’s caller until the timer has completed its delay.

• Stage 2: Write a routine called OSCreateProcess (to be called by osInit)a) In OSCreateProcess(), create OS structures , for instance a Process Control Block that

allows you to know/record everything about the process. One of the elements in this PCB is a pointer to the context for the process. Ask the hardware for a Context for the process you’re creating.

• Stage 3: Timer Queue is an object that contains an ordered list of the processes waiting for or currently being handled by the timer.

a) Your svc() calls AddToTimerQueue() and then gives up the CPUb) Your Interrupt_handler calls TimerInterrupt calls RemoveFromTimerQueue();

25

The Execution of test1a (Stage 1)

test.c

Hardware

base.c

Z502Clock(Memory IO) Z502SwitchContext

SVC

OS_Init

test1a main

Z502Idle

4

10/11

9

7

8

6

5

3 2

1

Z502DelayTimer(Memory IO)

12/13

26

Writing Test1aWhy Stage 1 isn’t enough

So, you say, I get test1a working after implementing Stage 1. Why should I do more?

Because you are building a multithreaded operating system. When a process calls SLEEP(), you don’t want to just sit there waiting for the sleep to finish; you want to be doing other things with the processor.

This means the timer operation becomes semi-independent, and processing of other operations can occur simultaneously.

Stages 2 and 3 prepare you for this.Stage 2 gives a process the record-keeping ability to give the processor to

another process ( by having the Context).Stage 3 sets up the timer processing ability for the system.

27

Components In The Starter Code – you don’t need all these right now!

Test.c

z502.c

O.S.

Z502Clock(Memory IO)

Z502MakeContext

Z502DestroyContext

Z502SwitchContext

Z502DelayTimer(Memory IO)

Z502Idle

SVC

fault_handler

osInit

test0 test1a test1b test2go o o o o o o o o

interrupt_handler

main

28

OS Components – What you need to Build getting ready for Stage 3

Test.c

z502.c

O.S.

SVC

Interrupt_HandlerTimerQueue

StartTimer

osInit

August, 2013

Z502Clock(Memory IO)

Z502MakeContext

Z502DestroyContext

Z502SwitchContext

Z502DelayTimer(Memory IO)

Z502Idle

main

OS_Create_Process

29

The Execution of test1a (Stage 3 – the Sleep() call. )

test.c

z502.c

base.c

SVC

osInit

August, 2013

test1a main4

10

9

7

8

5

SLEEP

3a

2

1OS_

Create_Process

Interrupt_Handler

TimerQueue

Start_Timer

11

12

13

14

Z502Clock(Memory IO)

Z502SwitchContext

Z502DelayTimer(Memory IO)

Z502Idle Z502MakeContext

3b

New Process Starts Here

6

30

The Execution of test1a (Stage 3)

4

10

9

8

7

6

5

3

2

1 The program starts in main(), and passes control to osInit.

osInit figures out what test you want to run. It passes the identifier for that test to os_create_process.We come to os_create_process, a routine YOU write. Here we ask the hardware for a context(Z502MakeContext) , create the PCB, and then call Z502SwitchContextT.

Z502SwitchContext causes control to be passed to a new thread which transfers control to test1a.

Note: Test1a does various system calls, but we’re looking only at SLEEP in this picture. Test1a does a system call SLEEP transferring control to svc.svc hands control of the SLEEP request to start_timer, a routine YOU write.

start_timer, enqueues the PCB of the running process onto the timer_queue.You will want a routine AddToTimerQueue()

Start_timer calls the Z502_DELAY_TIMER to give the request for a future interrupt. The timer starts thinking about the time, but interrupts in the future!! Start_timer realizes there’s nothing else to do and so calls Z502_IDLE. This routine says to idle the processor until an interrupt occurs.The timer has expired. This means that the time you requested has occurred. The hardware thus generates an interrupt so that the OS knows the hardware (timer) operation has finished.

31

The Execution of test1a (Stage 3)

In the interrupt handler, call TimerInterrupt() that you write. This routine takes the PCB off the timer queue. You will want a routine RemoveFromTimerQueue() to do this. This is the process that has been sleeping!When you return from the interrupt_handler, execution returns back to start_timer, to the line AFTER your call to Z502_IDLE.

Start_timer returns to svc.

svc returns to test1a.

11

12

13

14

32

Test1b - What you need to do• What’s the goal of Test1b?• Summary information.• Where do you find resources to help you?

• Architecture of the Simulator Environment• Z502 Hardware Organization and Architecture• Generic Operating System Structure

33

OS Components – What you need to Build for test1b

Test.c

z502.c

O.S.

SVC

Interrupt_HandlerTimerQueue

StartTimer

ReadyQueue

GiveUpCPU

Dispatcher

Make_Ready_To_Run

osInit

Z502Clock(Memory IO)

Z502MakeContext

Z502DestroyContext

Z502SwitchContext

Z502DelayTimer(Memory IO)

Z502Idle

main

34

Test2a - What you need to do• What’s the goal of Test2a?• Summary information.• Where do you find resources to help you?

• Architecture of the Simulator Environment• Z502 Hardware Organization and Architecture• Generic Operating System Structure

35

The Execution of test2a and test2b

test.c

z502.c

base.c

Z502MemoryRead

SVC

OS_Init

August, 2013

test2a main4

10

9

7

8

6

5 GET_PROCESS_ID

3a

21

OS_Create_Process

Interrupt_Handler

Process_Management

Z502MemoryWrite

11

12

Fault_Handler

Dispatch

Z502SwitchContext

Z502MakeContext

New Process Starts Here

3b

36

The Execution of test2a and test2b

4

10

9

8

7

6

5

3

2

1

The test may do system calls as we saw in test1*. The example we see here is GET_PROCESS_ID.

svc hands control of the system call to the appropriate handler.

The test does a Memory Request (either read or write). That request ends up in the hardware. If the hardware can handle it, you’re done. If hardware can NOT handle the call, then a page fault is generated. You do the work in your fault handler to make the memory access successful.After completing the page_fault work, always call your dispatcher to schedule the same or a new process. NEVER return from the fault handler.Reads and writes are handled the same way.

The program starts in main(), and passes control to osInit.

osInit figures out what test you want to run. It passes the identifier for that test to os_create_process.We come to os_create_process, a routine YOU write. Here we ask the hardware for a context(Z502MakeContext) , create the PCB, and then call Z502SwitchContextT.

Z502SwitchContext causes control to be passed to a new thread which transfers control to test1a.