Unix 3.4 Advantages and Disadvantages of Buffer Cache

ADVANTAGES AND DISADVANTAGES OF BUFFER CACHE

ADVANTAGES

• The use of buffers allows uniform disk access. It simplifies system design.
• The system places no data alignment restrictions on user processes doing I/O. By copying data from user buffers to system buffers (and vice versa), the kernel eliminates the need for special alignment of user buffers, making user programs simpler and more portable.
• Use of the buffer cache can reduce the amount of disk traffic, thereby increasing overall system throughput and decreasing response time.
• The buffer algorithms help insure file system integrity.

DISADVANTAGES

• Since the kernel does not immediately write data to the disk for a delayed write, the system is vulnerable to crashes that leave disk data in an incorrect state.
• Use of the buffer cache requires an extra data copy when reading and writing to and from user processes. When transmitting large amounts of data, the extra copy slows down performance.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 3.3 Structure of the Buffer Pool

STRUCTURE OF THE BUFFER POOL 

Structure of the Buffer Pool

The kernel caches the least recently used data into the buffer pool. Once a back from buffer pool is allocated for a file of the system this block cannot be used for any other file’s data. The kernel also maintains a free list of buffers. The free list is a doubly circular list of buffers.
When kernel wants to allocate any buffer it removes a node from the free list, usually from the beginning of list but is could take it from middle of the list too. When kernel frees a node from the buffer list it adds this free node at the end of the free list.

When kernel want to access the disk it searches the buffer pool for a particular device number-block number combination (which is maintained in the buffer header).
The entire buffer pool is organized as queues hashed as a function of device number-block number combination. The figure down below shows the buffers on their hash queues.

The important thing to note here is that no two nodes in the buffer pool can contain the data of same disk block i.e. same file.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 3.2 Buffer Header

BUFFER HEADER 

Buffer Header

When the system initializes the kernel allocates the space for the buffer cache. The buffer cache contains two regions/arts. One for the data/files that will be read from the disk, second the buffer header.

The data in the buffer cache corresponds to the logical blocks of the disk block of file system. The buffer cache is “in memory” representation of the disk blocks. This mapping is temporary as the kernel may wish to load some other files’ data into the cache at some later stage.

There will never be a case when the buffer has two entries for the same file on disk as this could lead to inconsistencies. There is only and only one copy of a file in the buffer.
The buffer header contains the metadata information like device number and the block number range for which this buffer holds the data. It stores the logical device number and not the physical device number. The buffer header also contains pointer to a data array for the buffer (i.e. pointer to the data region) .

The buffer header also contains the status of the buffer. The status of the buffer could be :

• The buffer is currently locked or unlocked
• The buffer contains valid data
• The kernel must write the buffer contents to disk before reassigning the buffer(write delay)
• The kernel is currently reading or writing the contents of the buffer to disk,
• A process is currently waiting for the buffer to become free.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 3.1 The Buffer Cache

THE BUFFER CACHE

The files are stored on the hard drive and the processes can access these files and create new files on the disk. When a process requests for a file the kernel brings the file into the main memory where user process can change, read or access the file.

The kernel read the super block to figure out the details about the hard drive and the inode table to find the meta data information of any file. So the kernel reads the inode into the memory whenever any process want to access the data and write it back onto the hard disk when the process is done using the file.

The kernel could read and write the file directly from the hard disk and put it in memory and vice versa but the response time and throughput will be very low in this case because of disks sow data transfer speed. 

To minimize the frequency of disk usage/access the kernel keeps a buffer to store the recently accessed files and/or frequently accessed files. This buffer is called the buffer cache.

When the process want to read a file the kernel attempts to read this file in the buffer cache, if the data is found in the buffer cache the data/file is sent to the process. If the file is not found in the buffer cache then the file is read from the disk and then kept in the buffer cache so that it can be made available to the process.


To minimize the disk access frequency the kernel may also implement the pre-caching or write delay functionalities.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 2.5 Kernel Data Structures and System Administration

KERNEL DATA STRUCTURES AND SYSTEM ADMINISTRATION 

Kernel Data Structures

Most kernel data structures occupy fixed-size tables rather than dynamically allocated space. The advantage of this approach is that the kernel code is simple, but it limits the number of entries for a data structure to the number that was originally configured when generating the system.

If, during operation of the system, the kernel should run out of entries for a data structure, it cannot allocate space for new entries dynamically but must report an error to the requesting user.

If, on the other hand, the kernel is configured so that it it is unlikely to run out of table space, the extra table space may be wasted because it cannot be used for other purposes. 

Algorithms typically use simple loops to find free table entries, a method that is easier to understand and sometimes more efficient than more complicated allocation schemes.


System Administration

Administrative processes are loosely classified as those processes that do various functions for the general welfare of the user community. 

Such functions include disk formatting, creation of new file systems, repair of damaged file systems, kernel debugging, and others. 

Conceptually, there is no difference between administrative processes and user processes. They use the same set of system calls available to the general community. They are distinguished from general user processes only in the rights and privileges they are allowed. 

Internally, the kernel distinguishes a special user called the superuser, endowing it with special privileges. A user may become a superuser by going through a login-password sequence or by executing special programs.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 2.4 Process States

PROCESS STATES

The lifetime of a process can be divided into a set of states, each with certain characteristics that describe the process.

State 1
The process is initially in the User mode. The user initialize the process to transit into Kernel mode for execution via system calls.

State 2
The process is executed in Kernel mode. Several processes can execute simultaneously in a time-shared manner and they may all run simultaneously in kernel mode. The kernel maintains consistency of processes by prohibiting arbitrary context switches and controlling the occurrence of interrupts. It raises the processor execution level around critical regions of code to prevent interrupts that could cause inconsistencies.
The kernel allows a context switch only when a process moves from the state "kernel running" to the state "asleep in memory." The system prevents inconsistency situations by disallowing context switches when a process executes in kernel mode. If a process goes to sleep, thereby permitting a context switch, kernel algorithms are encoded to make sure that system data structures are in a safe, consistent state. The kernel protects its consistency by allowing a context switch only when a process puts itself to sleep and by preventing one process from changing the state of another process.

State 3
The process is Sleeping.Processes go to sleep because they are awaiting the occurrence of some event, such as waiting for I/O completion from a peripheral device, waiting for a process to exit, waiting for system resources to become available, and so on. Processes are said to sleep on an event, meaning that they are in the sleep state until the event occurs, at which time they wake up.

State 4
The process after wake up is still not executing, but it is ready to run. The scheduler schedules the process and send it to kernel mode for execution. The scheduled processes are executed by the kernel one at a time in order to use the system resources.

State 5
The executed process returns backs to the user after getting completed.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 2.3 Process Sub-system

PROCESS SUB-SYSTEM 

A process is the execution of a program and consists of a pattern of bytes that the CPU interprets as machine instructions, data, and stack. A process executes by following a strict sequence of instructions that is self-contained and does not jump to that of another process. Processes communicate with other processes and with the rest of the world via system calls.

A process in a UNIX system is the entity that is created by the fork system call. Every process except process 0 is created when another process executes the fork system call. The process that invoked the fork system call is the parent process, and the newly created process is the child process. Every process has one parent process, but a process can have many child processes. The kernel identifies each process by its process number, called the process ID (PID). Process 0 is a special process that is created "by hand" when the system boots; after forking a child process (process 1), process 0 becomes the swapper process. Process 1, known as init, is the ancestor of every other process in the system and enjoys a special relationship with them.

A user compiles the source code of a program to create an executable file, which consists of several parts:
• a set of "headers" that describe the attributes of the file,
• the program text,
• a machine language representation of data that has initial values when the program starts execution, and an indication of how much space the kernel should allocate for uninitialized data.
• other sections, such as symbol table information.

Every process has an entry in the kernel process table, and each process is allocated user area that contains private data manipulated only by the kernel. 

The process table contains a per process region table, whose entries point to entries in a region table. 

A region is a contiguous area of a process's address space. Region table entries describe the attributes of the region.  

The u area contains information describing the process that needs to be accessible only when process is executing. The important fields are :
• a pointer to the process table slot of the currently executing process,
• parameters of the current system call, return values and error codes,
• file descriptors for all open files,
• internal I/O parameters,
• current directory and current root,
• process and file size limits.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 2.2 File Sub-system

FILE SUB-SYSTEM 

The internal representation of a file is given by an inode, which contains a description of the disk layout of the file data and other information such as the file owner, access permissions, and access times. Every file has one mode, but it may have several names, all of which map into the mode. Each name is called a link. When a process refers to a file by name, the kernel parses the file name one component at a time, checks that the process has permission to search the directories in the path, and eventually retrieves the mode for the file.When a process creates a new file, the kernel assigns it an unused mode.

The kernel contains two other data structures, the file table and the user file descriptor table. 

The file table is a global kernel structure, but the user file descriptor table is allocated per process. When a process opens or creates a file, the kernel allocates an entry from each table, corresponding to the file's mode. Entries in the three structures user file descriptor table, file table, and mode table maintain the state of the file and the user's access to it. The file table keeps track of the byte offset in the file where the user's next read or write will start, and the access rights allowed to the opening process. The user file descriptor table identifies all open files for a process. 

The kernel returns a file descriptor for the open and create system calls, which is an index into the user file descriptor table. When executing read and write system calls, the kernel uses the file descriptor to access the user file descriptor table, follows pointers to the file table and mode table entries, and, from the Mode, finds the data in the file. 

A file system has the following structure :

• The boot block occupies the beginning of a file system, typically the first sector, and may contain the bootstrap code that is read into the machine to boot, or initialize, the operating system. Although only one boot block is needed to boot the system, every file system has a (possibly empty) boot block.

• The super block describes the state of a file system — how large it is, how many files it can store, where to find free space on the file system, and other information.

• The Mode list is a list of modes that follows the super block in the file system. Administrators specify the size of the mode list when configuring a file system. The kernel references Modes by index into the mode list. One Mode is the root Mode of the file system: it is the mode by which the directory structure of the file system is accessible after execution of the mount system call.

• The data blocks start at the end of the mode list and contain file data and administrative data. An allocated data block can belong to one and only one file in the file system.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 2.1 Architecture of Unix Operating System

ARCHITECTURE OF UNIX OPERATING SYSTEM

There are three levels: user, kernel, and hardware. 

The system call and library interface represent the border between user programs and the kernel. System calls look like ordinary function calls in C programs, and libraries map these function calls to the primitives needed to enter the operating system.

User Level
The libraries are linked with the programs at compile time and are thus part of the user program. Programs frequently use other libraries such as the standard I/O library to provide a more sophisticated use of the system calls. 

Kernel Level
The set of system calls are partitioned into those that interact with the :
 1. file subsystem 
 2. process control subsystem. 

The file subsystem manages files, allocating file space, administering free space, controlling access to files, and retrieving data for users. Processes interact with the file subsystem via a specific set of system calls, such as open (to open a file for reading or writing), close, read, write, stat (query the attributes of a file), chown (change the record of who owns the file), and chmod (change the access permissions of a file).

The file subsystem accesses file data using a buffering mechanism that regulates data flow between the kernel and secondary storage devices. The buffering mechanism interacts with block I/O device drivers to initiate data transfer to and from the kernel. Device drivers are the kernel modules that control the operation of peripheral devices. Block I/O devices are random access storage devices alternatively, their device drivers make them appear to be random access storage devices to the rest of the system.

The process control subsystem is responsible for process synchronization of inter-process communication, process scheduling, and memory management. The file subsystem and the process control subsystem interact when loading a file in the memory for execution, the process subsystem read executable files into memory before executing them.

The scheduler module allocates the CPU to processes. It schedules them to run in turn until they voluntarily relinquish the CPU while awaiting a resource or until the kernel preempts them when their recent run time exceeds a time quantum. The scheduler then chooses the highest priority eligible process to run; the original process will run again when it is the highest priority eligible process available.

The memory management module controls the allocation of memory. If at any time the system does not have enough physical memory for all processes, the kernel moves them between main memory and secondary memory so that all processes get a fair chance to execute.

Hardware Level
The hardware control is responsible for handling interrupts and for communicating with the machine. Devices such as disks or terminals may interrupt the CPU while a process is executing. If so, the kernel may resume execution of the interrupted process after servicing the interrupt. Interrupts are not serviced by special processes but by special functions in the kernel, called in the context of the currently running process.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 1.5 Assumptions About Hardware

ASSUMPTIONS ABOUT HARDWARE

The execution of user processes on UNIX systems is divided into two levels: user and kernel. 

When a process executes a system call, the execution mode of the process changes from user mode to kernel mode: the operating system executes and attempts to service the user request, returning an error code if it fails.

The differences between the two modes are :- 
• Processes in user mode can access their own instructions and data but not kernel instructions and data. Processes in kernel mode, however, can access kernel and user addresses.
• Some machine instructions are privileged and result in an error when executed in user mode.

Interrupts and Exceptions
The UNIX system allows devices such as I/O peripherals or the system clock to, interrupt the CPU asynchronously. On interruption, the kernel saves it as current context, determines the cause of the interrupt, and services the interrupt. After the kernel services the interrupt, it restores its interrupted context and proceeds as if nothing has happened. When the kernel services an interrupt, it blocks out lower priority interrupts but services higher priority interrupts.
An exception condition refers to unexpected events caused by a process, such as addressing illegal memory, executing privileged instructions, dividing by zero, etc. They are distinct from interrupts, which are caused by events that are external to a process. Exceptions happen "in the middle" of the execution of an instruction, and the system attempts to restart the instruction after handling the exception. The UNIX system uses one mechanism to handle interrupts and exception conditions.

Processor Execution Levels

Computers typically have a set of privileged instructions that set the processor execution level in the processor status word. Setting the processor execution level to certain values masks off interrupts from that level and lower levels, allowing only higher-level interrupts. If the kernel masks out disk interrupts, all interrupts except for clock interrupts and machine error interrupts are prevented. If it masks out software interrupts, all other interrupts may occur.

Memory Management
The kernel permanently resides in main memory as does the currently executing process. When compiling a program, the compiler generates a set of addresses in the program that represent addresses of variables and data structures or the addresses of instructions such as functions. The compiler generates the addresses for a virtual machine as if no other program will execute simultaneously on the physical machine. When the program is to run on the machine, the kernel allocates space in main memory for it, but the virtual addresses generated by the compiler need not identical to the physical addresses that they occupy in the machine. The kernel coordinates with the machine hardware to set up a virtual to physical address translation that maps the compiler generated addresses to the physical machine addresses. The mapping depends on the capabilities of the machine hardware.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 1.4 Operating System Services

OPERATING SYSTEM SERVICES

The operating system performs various primitive operations on behalf of user processes to support the user interface described above. Among the services provided by the OS are :-

• Controlling the execution of processes by allowing their creation, termination or suspension, and communication.

• Scheduling processes fairly for execution on the CPU. Processes share the CPU in a time-shared manner: the CPU executes a process, the kernel suspends it when its time quantum elapses, and the kernel schedules another process to execute. The kernel later reschedules the suspended process.

• Allocating main memory for an executing process. 
The kernel allows processes to share portions of their address space under certain conditions, but protects the private address space of a process from outside tampering. 
If the system runs low on free memory, the kernel frees memory by writing a process temporarily to secondary memory, called a swap device, lithe kernel writes entire processes to a swap device, the implementation of the UNIX system is called a swapping system; if it writes pages of memory to a swap device, it is called a paging system.

• Allocating secondary memory for efficient storage and retrieval of user data.
This service constitutes the file system. The kernel allocates secondary storage for user files, reclaims unused storage, structures the file system in a well understood manner, and protects user files from illegal access.

• Allowing processes controlled access to peripheral devices such as terminals, tape drives, disk drives, and network devices.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 1.3 File System

UNIX FILE SYSTEM

The UNIX file system is characterised by :

• a hierarchical structure,
• consistent treatment of file data,
• the ability to create and delete files,
• dynamic growth of files,
• the protection of file data,
• the treatment of peripheral devices (such as terminals and tape units) as files.

The file system is organised as a tree with a single root node called root, every non-leaf node of the file system structure is a directory of files, and files at the leaf nodes of the tree are either directories, regular files, or special device files.

Each file or directory is uniquely identified by its name, the directory in which it resides, and a unique identifier, typically called an inode.

The directories have specific purposes and generally hold the same types of information for easily locating files.

/ - This is the root directory which should contain only the directories needed at the top level of the file structure

/bin - This is where the executable files are located. These files are available to all users

/dev - These are device drivers

/etc - Supervisor directory commands, configuration files, disk configuration files, valid user lists, groups, ethernet, hosts, where to send critical messages

/lib - Contains shared library files and sometimes other kernel-related files

/boot - Contains files for booting the system

/home - Contains the home directory for users and other accounts

/mnt - Used to mount other temporary file systems, such as cdrom and floppy for the CD-ROM drive and floppy diskette drive, respectively

/proc - Contains all processes marked as a file by process number or other information that is dynamic to the system

/tmp - Holds temporary files used between system boots

/usr - Used for miscellaneous purposes, and can be used by many users. Includes administrative commands, shared files, library files, and others

/var - Typically contains variable-length files such as log and print files and any other type of file that may contain a variable amount of data

/sbin - Contains binary (executable) files, usually for system administration. For example, fdisk and ifconfig utlities

/kernel - Contains kernel files

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 1.2 System Structure

SYSTEM STRUCTURE

The structure of UNIX system is a set of well defined layers, each performing different functions according to their levels.

Hardware
The hardware provides the operating system with basic services. The operating system directly interacts with the hardware, providing common services to programs.

Kernel
The kernel is a set of system programs which build up the operating system. The kernel acts as a mediator between user programs and hardware.
Viewing the system as a set of layers, the operating system is commonly called the system kernel, emphasising its implementations of the UNIX system, the operating system interacts with a native operating system that, in turn, interacts with the underlying hardware and provides necessary services to the system.

Unix Programs
Programs such as the shell and editors (ed and vi) shown in the outer layers interact with the kernel by invoking a well defined set of system calls. The system calls instruct the kernel to do various operations for the calling program and exchange data between the kernel and the program. 
Several programs shown in the figure are in standard system configurations and are known as commands, but private user programs may also exist in this layer as indicated by the program whose name is a.out, the standard name for executable files produced by the C compiler.

Other Application Programs
Other application programs can build on top of lower-level programs, hence the existence of the outermost layer in the figure. 
For example, the standard C compiler, cc, is in the outermost layer of the figure: it invokes a C pre-processor, two-pass compiler, assembler, and loader (link-editor), all separate lower programs.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.

Unix 1.1 System Characteristics

SYSTEM CHARACTERISTICS

The Unix system is divided into two parts. 

1. The first part consists of programs and services that are readily apparent to users, including such programs as the shell, mail, text processing packages, and source code control systems. 
2. The second part consists of the operating system that supports these programs and services.

Properties / Characteristics / General Overview

• The system is written in a high-level language, making it easy to read, understand, change, and move to other machines.
• It has a simple user interface that has the power to provide the services that users want.
• It provides primitives that permit complex programs to be built from simpler programs.
• It uses a hierarchical file system that allows easy maintenance and efficient implementation.
• It uses a consistent format for files, the byte stream, making application programs easier to write.
• It provides a simple, consistent interface to peripheral devices.
• It is a multi-user, multi-process system; each user can execute several processes
• simultaneously.
• It hides the machine architecture from the user, making it easier to write
• programs that run on different hardware implementations.

Friends, if you find this post useful please comment below. If you want quick notes for any topic please mail us at hardikpanchal551@gmail.com, we will try to provide notes if possible. Thanks for reading.