Virtual Memory, File-System Interface

Background Virtual memory – separation of user logical

memory from physical memory. Only part of the program needs to be in

memory for execution Logical address space can therefore be much

larger than physical address space Allows address spaces to be shared by several

processes Allows for more efficient process creation

Virtual memory can be implemented via Demand paging

2

Virtual Memory Larger Than Physical Memory

3

Demand Paging Bring a page into memory only when it is

needed Less I/O needed Less memory needed Faster response More users

Page is needed reference to it invalid reference abort not-in-memory bring to memory

Lazy swapper – never swaps a page into memory unless page will be needed Swapper that deals with pages is a pager

4

Page Table When Some Pages Are Not in Main Memory

5

Handling a Page Fault

6

Process Creation Virtual memory allows other benefits Copy-on-Write (COW): more efficient

process creation allows both parent and child processes to

initially share same pages in memory If either process modifies a shared page, only

then is the page copied

7

What happens if there is no free frame?

8

Page replacement – find some page in memory, but not really in use, swap it out Goal –minimize

number of page faults

Only modified pages are written to disk to reduce overhead of page transfers

Basic Page Replacement

1. Find the location of the desired page on disk

2. Find a free frame: - If there is a free frame, use it - If there is no free frame, use a page replacement algorithm to select a victim frame

3. Bring desired page into the (newly) free frame; update the page and frame tables

4. Resume the process

9

Page Replacement Algorithms

Want lowest page-fault rate Evaluate algorithm by running it on a

particular string of memory references (reference string) and computing the number of page faults on that string

In all our examples, the reference string is

1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

10

First-In-First-Out (FIFO) Algorithm Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 3 frames (3 pages can be in memory at a time per

process)

4 frames

Belady’s Anomaly: more frames more page faults

1

2

3

1

2

3

4

1

2

5

3

4

9 page faults

1

2

3

1

2

3

5

1

2

4

5 10 page faults

44 3

11

Optimal Page Replacement

12

Replace page that will not be used for longest period of time

Used for measuring how well your algorithm performs

Least Recently Used (LRU) Page Replacement

13

Every page entry has a counter; every time page is referenced through this entry, copy the clock into the counter

When a page needs to be changed, look at the counters to determine which to change

LRU Algorithm (Cont.)

Stack implementation – keep a stack of page numbers in a double link form: Page referenced:

move it to the top requires 6 pointers to be changed

14

Use Stack to Record The Most Recent Page References

15

keep a stack of page numbers in a double link form: Page referenced move it to the top, requires 6

pointers to be changed

Counting Algorithms Keep a counter of the number of references

that have been made to each page

Least Frequently Used (LFU) Algorithm: replaces page with smallest count

Most Frequently Used (MFU) Algorithm: based on the argument that the page with the smallest count was probably just brought in and has yet to be used

16

Frame Allocation Equal allocation – For example, if there are

100 frames and 5 processes, give each process 20 frames.

Proportional allocation – Allocate according to the size of process

mSs

pa

m

sS

ps

iii

i

ii

for allocation

frames of number total

process of size

5964137127

56413710

127

10

64

2

1

2

a

a

s

s

m

i

17

Thrashing

If a process does not have “enough” pages, the page-fault rate is very high. Thrashing a process is busy swapping pages in

and out

Demand paging works because of locality model Process migrates from one locality to

another Localities may overlap

Why does thrashing occur? size of locality > total memory size

18

Working-Set Model working-set window a fixed number of

page references Example: 10,000 instruction

WSSi (working set of Process Pi) =total number of pages referenced in the most recent (varies in time) if too small will not encompass entire locality if too large will encompass several localities if = will encompass entire program

D = WSSi total demand frames if D > m Thrashing Policy if D > m, then suspend one of the

processes

19

Move on to File System To explain the function of file systems To describe the interfaces to file systems To explore file-system protection

20

File Concept A named collection of related information

that is stored on secondary storage The smallest allotment of secondary storage A sequence of bits, bytes, lines or records… Types:

Datanumericcharacterbinary

Program

21

File Structure None - sequence of words, bytes Simple record structure

Lines Fixed length Variable length

Complex Structures Formatted document Relocatable load file: executable files, library files Indexed file: for fast access to data

Can simulate last two with first method by inserting appropriate control characters

22

Example of Index and Relative Files

23

File Attributes Name – only information kept in human-readable form Identifier – unique tag (number) identifies file within

file system Type – needed for systems that support different

types Location – pointer to file location on device Size – current file size Protection – controls who can do reading, writing,

executing Time, date, and user identification – for

creation/last modification/access, used for protection, security, and usage monitoring

Information about files are kept in directory structure, which is maintained on the disk

24

File Operations File is an abstract data type with operations

such as: Create Write Read Reposition within file Delete Truncate Open(Fi) – search the directory structure on disk for

entry Fi, and move the content of entry to memory Close (Fi) – move the content of entry Fi in memory

to directory structure on disk

25

Open Files Several pieces of data are needed to manage open

files: File pointer: pointer to last read/write location,

per process that has the file open File-open count: counter of number of times a

file is open – to allow removal of data from open-file table when last processes closes it

Disk location of the file: cache of data access information

Access rights: per-process access mode information

26

Open File Locking Provided by some operating systems and

file systems Mediates access to a file Mandatory or advisory:

Mandatory – access is denied depending on locks held and requested

Advisory – processes can find status of locks and decide what to do

27

File Types – Name, Extension

28

Access Methods

Sequential Accessread nextwrite next resetno read after last write(rewrite)

29

Simulation of Sequential Access on Direct-access File

30

Direct Access, n = relative block numberread nwrite nposition to nread nextwrite next rewrite n

Directory Structure Directory: a collection of nodes containing

information about all files

F 1 F 2F 3

F 4

F n

Directory

Files

Both the directory structure and the files reside on disk

31

Disk Structure

Disk can be subdivided into partitions also known as minidisks, slices

Disks or partitions can be protected against failure using: RAID (Redundant Array of Independent Disks)

Disk or partition can be used raw – without a file system, or formatted with a file system

Entity containing file system known as a volume Each volume containing file system also tracks that

file system’s info in device directory or volume table of contents

general-purpose file systems vs special-purpose file systems

32

A Typical File-system Organization

33

Operations Performed on Directory

Search for a file Create a file Delete a file List a directory Rename a file Traverse the file system

34

Organize the Directory (Logically) to Obtain

Efficiency – locating a file quickly Naming – convenient to users

Two users can have same name for different files

The same file can have several different names

Grouping – logical grouping of files by properties, (e.g., all Java programs, all games, …)

35

Single-Level Directory

A single directory for all users

Unique naming problem

Grouping problem

36

Two-Level Directory Separate directory for each user

Path name

Can have the same file name for different user

Efficient searching

No grouping capability

37

Tree-Structured Directories

38

Tree-Structured Directories (Cont)

Efficient searching

Grouping Capability

Current directory (working directory) cd /spell/mail/prog

39

Tree-Structured Directories (Cont)

Absolute or relative path name Creating a new file is done in current

directory Delete a file

rm <file-name> Creating a new subdirectory is done in

current directorymkdir <dir-name>

Example: if in current directory /mailmkdir count

mail

prog copy prt exp count

Deleting “mail” deleting the entire subtree rooted by “mail”

40

Acyclic-Graph Directories Have shared subdirectories and files

41

Acyclic-Graph Directories (Cont.)Issues: A file can have more than one path (aliasing

problem) If dict deletes list dangling pointer

Solutions: Backpointers, so we can delete all pointers Count number of references to a file

Implement shared files / directories: New directory entry type:

Link – another name (pointer) to an existing file Resolve the link – follow pointer to locate the file

42

General Graph Directory

43

General Graph Directory (Cont.)

How do we guarantee no cycles? (avoid infinite loops) Allow only links to files, not subdirectories Garbage collection: delete items that have no

reference to it Traverse file system and mark everything that

can be accessed Collected everything that is not marked onto a

list of free space Every time a new link is added, use a cycle

detectionalgorithm to determine whether it is OK

44

File Sharing in Multiple User System Sharing of files on multi-user systems is desirable

Sharing may be done through a protection scheme

Identify users User IDs identify users, allowing permissions and

protections to be per-user

Group IDs allow users to be in groups, permitting group access rights

45

Protection File owner/creator should be able to control:

what can be done by whom

Change owner user or group chgrp: change group associated with file chown: change owner of file

Types of access Read Write Execute Append Delete List

46

Access Lists and Groups

chmod 761 prog1.out Mode of access: read, write, execute, setuid, setgid Three classes of users

RWXa) owner access 7 1 1 1

RWXb) group access 6 1 1 0

RWXc) public access 1 0 0 1

chmod: change access modes

47

Digit Permissions Binary Meaning

0 --- 000 All types of access are denied

1 --x 001 Execute access is allowed only

2 -w- 010 Write access is allowed only

3 -wx 011 Write and execute access are allowed

4 r-- 100 Read access is allowed only

5 r-x 101 Read and execute access are allowed

6 rw- 110 Read and write access are allowed

7 rwx 111 Everything is allowed

setuid, setgid access right

Mode of access: read, write, execute, setuid, setgid setuid, setgid: Unix access rights flags that allow users to run

an executable with permissions of the executable's owner or group.

Used to allow users to run programs with temporarily elevated privileges in order to perform a specific task.

When an executable file has been given setuid attribute, normal users who have permission to execute this file gain the privileges of the user who owns the file (commonly root) within the created process. When root privileges have been gained within the process, the application can then perform tasks on the system that regular users normally would be restricted from doing.

E.g. passwd, chsh commands for changing password or login shell

Need to modify system file /etc/passwd Another example: program you used for submitting programs

49

File Sharing – Remote File Systems Network allow file system access between

systems Manually via FTP Automatically, seamlessly using distributed file

systems Semi automatically via world wide web

Client-server model allows clients to mount remote file systems from servers Server can serve multiple clients Client and user-on-client identification is insecure or

complicated NFS is standard UNIX client-server file sharing

protocol CIFS is standard Windows protocol Standard operating system file calls are translated

into remote calls Distributed Information Systems (distributed

naming services) such as LDAP, DNS, NIS, Active Directory implement unified access to information needed for remote computing50

File Sharing – Consistency Semantics

Consistency semantics specify how multiple users are to access a shared file simultaneously Similar to Ch 7 process synchronization algorithms

Tend to be less complex due to disk I/O and network latency (for remote file systems

Andrew File System (AFS) implemented complex remote file sharing semantics

Unix file system (UFS) implements: Writes to an open file visible immediately to other users

of the same open file Sharing file pointer to allow multiple users to read and

write concurrently AFS has session semantics

Writes only visible to sessions starting after the file is closed

51

File System Mounting A file system must be mounted before

it can be accessed A unmounted file system is mounted at

a mount point

52

Mount Point

53