Linux and Advanced Operating Systems

Linux and Advanced Operating Sytems (L3 IBIOM / M2.1 BioInforMatique)

Linux and Advanced Operating Systems(L3 - IBIOM)

(M 2.1 - BioInforMatique)

L. Mouchard

[email protected]

1st Term 2012-2013

[email protected] LITIS EA4108 - Faculty of Sciences 1/27

mailto:[email protected]


Linux: so easy...



References

Historical references

A brief history of Unix/Linux/MacOS Operating Systems (OS)

1969 Bell Laboratories (Thomson) Multics → Unix written in low-level assembler;

1971 Unix written in C language (designed by Ritchie et Thomson for this specificpurpose);

1974-1977 Unix freely avalaible for Higher Education;

1978-1983 Several versions of Unix are simultaneously developed (BSD, System V);

1984 X Window system (DEC-IBM-MIT Athena Project);

1989 GNU Project is launched (GNU means: GNU’s Not Unix);

1991-... Linux (Thorvald) writes a PC version of Unix, partially based on previous workby A. Tannenbaum;

1995 Linux takes off;

1999 MacOS X (a descendant of BSD d’Unix);

2009 Milestone: one hundred Linux distributions are available (including Slackware,Ubuntu, Fedora, Gentoo, Mandriva, Knoppix, Backtrack...).

To be compared with Microsoft c© history



References

Main features Multi-tasking/multi-user from the crave

Several remarks

Linux/Unix/MacOS X operating systems are:

multi-tasking OS several programs can be executed simultaneously onthe same computer (even with a single processor);

multi-user OS several users can work simultaneously on the samephysical computer;

file systems these OS partially capitalize on the way they handlefile systems and their associated online commands;

shells these OS partially capitalize on the way they can in-terpret and combine online commands: their nativeinternal interpreters (handling functions, loops andtests) are named shells.



References

Main features Multi-tasking OS

(We will use, from now on, Linux rather than Linux/Unix/MacOS X)

A real multi-tasking OS or a time-shared system?

Time (1ms)

Mono-processor

p1 p2 p3 p1 p2 p1 p2 p1 p4 p2

Multi-processor: tasks are allocated to different processors;

mono-processor: CPU time is shared for simulating multi-tasks;

process: it is a program the system is in charge of (states are: running, waiting,zombie, ...);

Linux has to keep each user and each process separate and has to properly handleaccesses to all possible devices (CPUs, memory, hard drives...);

a process can be interrupted, suspended and resumed later if needed, can be priori-tized...;



References

Main features Multi-user

More than one user on the same physical computer?

Linux has been designed from the beginning as a network-oriented OS;

it is possible to access a remote computer from a local one and use itsCPU/Memory/Storage as if it was local;

Authentication process (login name/password):

several graphical interfaces for local access: kdm, gdm, xdm...online (text) commands for remote access such as (but not limited to):ssh, sftp, ....

User/group/Others.

Login process: from (login name,password) to actual connection;

Why is your password so important and how to choose a good one?



References

Main features Some graphical applications

Examples (freewares)

Terminals to be used for online commands (xterm, console, gnome-terminal...);

many web browsers (mozilla firefox, konqueror, dolphin, Opera...);

office suites (libreoffice, abiword...);

development kits for (programming languages such as) Java, C,Python, R, HTML...);

RDBMS (Relational DataBase Managment Systems) such as Oracle,MySQL, PostgreSQL...;

text formatter LATEX, compiler and visualisers;

plotters (R, Gnuplot, pgfplot...);

automatic image processing (gimp, ImageMagick...).



References

Online commands

Why should I use a text terminal rather than a file browser?

Linux strength is mainly based on its ability to handle, parse and processonline commands operating on its file systems;

Linux philosophy 1: keep commands as simple and modular as possible;

Linux philosophy 2: provide a mechanism for gathering simple commands andcreate complex ones;

some examples: create a (series of) directory(ies), rename or move a directory;(simple) parse a file to display all lines containing a pattern, (simple) countthe number of lines in a file, and (complex) gather these two commands forcounting in a file the number of lines that contain a given pattern.

plan the future execution of a given command;

schedule (with a specific periodicity: each night, monday, first Wednesday ofthe month at 4am) the forthcoming executions of a given command.



References

Online commands Examples: the Gnome terminal and the konsole (among others)

Some applications for typing in online commands

gnome-terminal and konsole;

tuning the menu/look and feel;



References

Our very first commands

Command formatcommand [option[s]] [argument[s]]

command command to be executed.spaces are used for separating the command from the pos-sible options and the possible options from the possible ar-guments. Spaces can be either one or several <space>symbols, one or several <tab> symbols or a mix.

[option[s]] Brackets mean optional: zero, one or several options. Onlyfor explanations, they are not supposed to be typed in.

[argument[s]] zero, one or several arguments.If one wants to display the current date, the command date

expects no argument, if a specific format is to be used, thecommand date expects exactly one argument.Similarly, if one wants to copy a file into a directory, thecommand cp expects two arguments, while if one wants tocopy several files into a directory, the command cp expectsprobably more than two arguments.



References

Our very first commands

Environment and help commands

date current datedate +format current date using a specific format

cal [month] [year] calendar [for month] [for year]hostname computer nameuname [options] linux name and versiontty terminal namelogname login nameid user identificationwhoami litteraly: who am I?who list of connected usersfinger user[s] informations on user[s], possibly disactivatedman command[s] manual pages for command[s]whatis command[s] overview of command[s]apropos pattern[s] commands associated to pattern[s]



Files

Types Regular, Directory and Special

Types

ordinary/regular files containing data (text, image, sound, video, libreoffice document)

directory file containing a list of references to subdirectories or/and files: it istechnically speaking a file that contains pairs (filename, inode number).

special all other files, see below.

Special files

link link to another file (sort of shortcut) on the same or a different file system.block file associated to an Input/Output block device (USB stick, internal or

external hard drive...)character file associated to a Input/Output character device (keyboard, screen...).socket file associated to a network-oriented application (providing data exchange

mechanisms)tube file associated to the automatic execution of a command (providing a data

spooling mechanism for automatic data processing)



Files

File systems Files

How are the files organised?

The file systems store files in an arborescent and hierarchical way:



Files

File systems Files

How are the files organised?

Relative filename: image.gif;

Asolute filenames : /home/lm/Enseignement/ ... 2012 2013/Cours/image.gifand /home/lm/ ... / wordpress/img/image.gif;

Home directory, working (current) directory.



Files

File systems Naming convention

Absolute and relative names, other important notations

/ at the begining of a filename, it is the root of the file system tree.Every absolute name starts with /;

/ within a filename, it separates two directory names.. the current directory, whatever its name is;.. the parent directory, whatever the current directory is: the parent of

/ is /.absolute name path from the root of the file system (/) to the file itself, directory

names are separated by /.relative name path from the working directory to the file itself, directory names are

separated by /.;

Remarks

No major constraint on the length of the filenames;

made of letters, digits and “ ” (some other symbols are accepted but not recom-mended);

choose explicit names.



Files

File systems Metacharacters (wildcards, jokers)

Filenames and metacharacters (depend on the Shell that is used)

* any (possibly empty) series of characters;? any character, exactly one;{a,d,f} any character in the set: a or d or f;[a-c] any character in the interval: a or b or c;

Some examplesimg1.jpg, img2.jpg, img3.jpg, img4.jpg, img5.jpg, img6.jpg, img7.jpg, img8.jpg,img9.jpg, img10.jpg, img11.jpg.

img?.jpg all files but img10.jpg and img11.jpg;img1?.jpg files img10.jpg and img11.jpg;img1*.jpg files img1.jpg, img10.jpg and img11.jpg;img{3,6}.jpg files img3.jpg and img6.jpg;img[4-6].jpg files img4.jpg, img5.jpg and img6.jpg.



Commands

Directories

Commands operating on directories

pwd print working directory;cd [~] change working directory to default home directorycd ~john change working directory to john’s home directorycd dirname change working directory to dirname

cd - change working directory to previous working directorypushd dirname current working directory is memorized, new working

directory is dirname

popd change working directory to memorized directory

mkdir dirname[s] make directory(ies) dirname[s]

mkdirhier dirname[s] make directory(ies) dirname[s] recursivelyrmdir dirname[s] remove empty directory(ies) dirname[s]

mv dirname[s] existdirname move dirname[s] into existdirname/dirname

mv dirname nonexistdirname move (rename) dirname[s] into nonexistdirname

cp -R dirname[s] existdirname copy dirname[s] into existdirname/dirname

cp -R dirname nonexistdirname copy dirname into nonexistdirname

ls dirname[s] list the content of dirname[s]ls -l dirname[s] list (verbose) the content of dirname[s]



Commands

ls -l

Detailed description of the content of a directory: ls -l

[laurent] ls -l

total 5068

-rw-r--r-- 1 laurent teaching 4938937 2012-08-23 02:48 chapter01.pdf

lrwxrwxrwx 1 laurent teaching 18 2012-08-22 13:34 guide.pdf -> ../guide.pdf

-rw-r--r-- 1 laurent teaching 68096 2012-07-29 00:05 Term.ppt

drwxr-xr-x 3 laurent teaching 4096 2012-07-11 23:03 GI2

-rw-r--r-- 2 laurent teaching 114688 2012-06-01 09:36 L3 GI2 2012 2013.xls

1 2 3 4 5 6 7 8

1 - regular file (-), link (l), directory (d), block (b), character (c),pipe (p), socket (s).

2 rw-r--r-- owner (rw-), group (r--) and others (r--).modalities : read (r), write (w), execute (x) nothing (-)

3 1 number of links (the filename has only one filename)4 laurent owner (login name, see /etc/passwd or similar mechanism)5 teaching group (group name, see /etc/group or similar mechanism)6 4938937 size of the file (bytes)7 2012-08-23 02:48 last modification8 chapter01.pdf filename



Commands

Inodes

Files: what is an inode?

Inode: physical and logical contents of a file

It contains:

user and group id;

filetype and access modes;

dates: creation, last modification, last access;

number of physical links;

exact size and number of blocks on the storage device;

list of blocks where the file is to be found.

It does not contains:

the filename!



Commands

Inodes

All the informations associated to a given file

You can simply add -i to ls, that is ls -i -l or simply ls -il:

922928 -rw------- 1 laurent laurent 4216 Aug 20 06:29 cours10 GI2 2012 2013.tex

922929 -rw-rw-r-- 1 laurent laurent 1017 Aug 29 00:12 cours GI2 2012 2013.aux

922921 -rw-rw-r-- 1 laurent laurent 52182 Aug 29 00:12 cours GI2 2012 2013.log

922941 -rw-rw-r-- 1 laurent laurent 6502 Aug 29 00:12 cours GI2 2012 2013.nav

922927 -rw-rw-r-- 1 laurent laurent 1161 Aug 29 00:12 cours GI2 2012 2013.out

922930 -rw-rw-r-- 1 laurent laurent 911221 Aug 29 00:12 cours GI2 2012 2013.pdf

922943 -rw-rw-r-- 1 laurent laurent 0 Aug 29 00:12 cours GI2 2012 2013.snm

922988 -rw------- 1 laurent laurent 4468 Aug 29 00:03 cours GI2 2012 2013.tex

inode numbers appear on the left;

these numbers are not necessarly consecutive: it depends mainly onthe workload and the number of connected users;

the concept of inode is important when one wants to understand howthe cp, mv, rm commands work...



Commands

File systems

What is a file system?

So far, we consider the simple case of a unique file system (tree);

we might have one or several file systems (partitions) on the same storage device;

a file system corresponds to a specific region of the tree we described before;

several types of file systems:

ext4 performant and reliable file system which is able to handle very largefiles;

vfat, ntfs Microsoft c© file systems;iso660 CD-ROM file system conforming to ISO9660 standard;nfs, smb network file systems.

a file system is attached to a root file system (/home, /public/Data);

if a file system is full, most of the time, it does not prevent the system fromworking;

one inode table per file system;



Commands

File systems

Commands

mount list and properties of active (mounted) file systems;df disk usage for all mounted file systems;df file[s] disk usage for the file systems that contain file[s] ;du file[s] disk usage for file[s];du -s dir[s] disk usage for dir[s];quota size and number of files you can use;stat file[s] inode informations for file[s];

whereis com path to binary, source or manual page for com;which com order in which Linux finds com in its file systems;locate patt commands corresponding to pattern *patt*;find ... files that (for example): are larger than 50MB, older than 2

months, haven’t been accessed for the last 20 days etc (manfind for a complete description).



Commands

Files

Commands operating on files

touch nexist file create an empty file nexist file;touch exist file modify the last access date for file exist file;cp file[s] exist dir copy file[s] into exist dir/file[s];cp src dst copy file src to dst;rm file[s] remove definitely file[s];mv file[s] exist dir move file[s] into exist dir;mv src dst rename file src to dst;

Attention: cp + rm 6= mv

ln src dst hard link between src (existing) and dst (new name forunique physical contents): (dst,inode src) is created inaddition to existing (src,inode src)

ln -s src dst symbolic link between src (existing) and dst (new name).Used for cross-device link: (dst,inode for src) is cre-ated, the inode corresponds to a disk block which containsthe filename (src).



Commands

File content

From files to file content

We would like to be able to:

display everything, one page at a time, beginning, end...

display only lines (not) containing a given pattern;

display selected parts of each line: characters, columns;

sort, remove duplicates;

gather the contents of several files into a unique file;

split the content of a unique file into several independent files;

(un)compress a file or a set of files.



Commands

File content

Displaying the content of a file

cat file[s] whole content;more file[s] one page at a time;less file[s] one page at a time;head -n file[s] only n first lines;tail -n -k file[s] only k last lines;tail -n +k file[s] only from line k till the end;

sort file[s] sort;uniq file[s] remove consecutive duplicates;grep pattern file[s] only the lines containing pattern;cut -c... file[s] only specified characters;cut -d... -f... file[s] only specified fields (-f) with specified separator (-d);

paste file1 file2 gather lines from file1 and file2;join file1 file2 join lines from file1 and file2;split size file split a file into chunks of size Bytes;csplit pattern file split a file according to a given pattern;

gzip file compress file into file.gz (zip or bzip2)gunzip file.gz uncompress file.gz into file (unzip or bunzip2)tar cf file.tar dir archive dir in one unique file (file.tar in our example);tar xf file.tar extract all files contained in archive file.tar.



Processes

What is a process?

What is a process?

a process is a program that is currently executing;

different possible states: execution, stopped, sleeping, zombie,waiting...;

processes and priorities: differential use of CPU resources;

foreground process: the corresponding command/program prevents allother commands to be executed;

tree-like structure: parent, fork, exec;

/proc directory and its content.



Processes

commandes

Commandes oprant sur les processusLa gestion des processus

commande option[s] argument argument[s]

ps liste des processus grs par le systmepstree arborescence des processuspgrep motif liste des processus dont le nom correspond motif

top liste ordonne des processus avec mise jour rgulirepmap proc affiche l’utilisation mmoire de proc

chrt proc affiche/modifie la priorit de proc

taskset proc affiche/modifie l’utilisation des processeurs pour procnice cmd lance l’excution de la commande cmd suivant une pri-

orit donnerenice proc modification de la priorit d’un processus

Quelques signaux : SIGHUP, SIGINT, SIGKILL, SIGTERM,

SIGCONT, ...

kill sig proc envoi d’un signal sig donn proc

pkill sig motif envoi d’un signal sig donn aux processus dont le nomcorrespond motif

killall sig proc envoi d’un signal sig [email protected] LITIS EA4108 - Faculty of Sciences 27/27

Date post:	12-Sep-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Linux and Advanced Operating Systems

Documents