unix file systems acknowledgement : soongsil univ. presentation materials 단국대 최종무...

UNIX File Systems

Acknowledgement : Soongsil Univ. Presentation Materials 단국대 최종무 교수님 강의노트

(based on Chap 4. in the book “the design of the UNIX OS”)

Hard Disk Drive Structure

Characteristics Seek time, rotational latency, transmission time Disk Scheduling?, cylinder group?, Buffering?

2

Flash Memory

limitation Block erasure : block 단위로 지워짐 . Memory wear : # of erase limiation Read disturb

Seek optimization 이 불필요 Require strong reliability (e.g. power failure recovery)

3

File System

the system in which files are named and where they are placed logically for storage and retrieval

컴퓨터에서 파일이나 자료를 쉽게 발견 및 접근 (read/write) 할 수 있도록 보관 / 조직하는 체제

Properties Hierarchical structure Ability to create, read/write, and delete files Dynamic growth of files Protection of file data

File System 하는일

Support file interfaces to user File interface: open, close, read, write, … Directory manipulation: mkdir, cd, rmdir,… File system interfaces: mount, mkfs, fsck, …

File management i-node, FAT Access control, attribute control, ..

Storage management Disk management (Block allocation, I/O handling, …)

5

OS 별 File Systems

UNIX File System LINUX : EXT2, EXT3, EXT4 Windows : NTFS OS X (Apple Mac OS) : HFS+ Traditional OS (MS-Windows 9x) : FAT32

6

FAT

Size limitation of FAT32 32GB 보다 큰 파티션을 만들수 없다 4GB 초과하는 파일을 만들수 없다

단순 / 빠르고 호환성이 좋음 ( 대부분의 OS 에서

사용가능 )

7

FAT

File Allocation Table

8

FAT

9

NTFS

MS-Windows 계열 OS 기본 file system 보안 ( 사용 권한 및 암호화 ) 신뢰성 ( 파일시스템 오류 자동 복구 ) Storage growth Support for Large volume and file sizes

( 대용량 파일 (16EB)/ 볼륨 지원 ) 호환성 떨어짐 (Mac OS 에서는 read 만 지원 ,

LINUX 에서는 배포판 / 버전에 따라 불분명 )10

UNIX File System Overview

Boot block : 부팅에 필요한 정보 저장 Super block : file system 전체 정보 저장 i-list : 각 file 당 하나의 i-node 할당 , file 정보 저장 Data blocks : 실제 파일 내용을 블록 단위 저장

Block : 데이터를 한번에 읽고 쓰는 단위 ( 주로 1KB~4KB)

File System Layout

• Root directory 의 i-node 는 미리 정해져 있음 .

Sample File System

File Sharing

Each user has a “file descriptor table” (or “per-user open file table”) Each entry in the channel table is a pointer to an entry in the system-

wide “open file table” Each entry in the open file table contains a file offset (file pointer) and

a pointer to an entry in the “memory-resident i-node table” If a process opens an already-open file, a new open file table entry is

created (with a new file offset), pointing to the same entry in the memory-resident i-node table

If a process forks, the child gets a copy of the channel table (and thus the same file offset)

14

UNIX File System Overview

EXT2 file system LINUX 기본 file system Ext2 특징

block group 도입 : 같은 파일은 같은 block group ( 하드 디스크상 같은 cylinder) 을 가능한 사용하여 성능 향상 (disk seek time, fragmentation )

Ext3 특징 Journaling 기능 도입 : Logging and Fast Recovery

Ext4 특징 Large size 파일 지원

16

UNIX File System

17

18

Table of Contents

Inodes Structure of a regular file Directories Conversion of a path name to an Inode Super block Inode assignment to a new file Allocation of disk blocks Other file types Summary

19

Summary

Inode is the data structure that describes the attributes of a file, including the layout of its data on disk.

Two versions of the inode Disk copy : store the inode information when file is not in use In-core copy : record the information about active files.

Inode disk blocks Directories : files that correlate file name components to inode

numbers namei : convert file names to inodes Inode assignment Disk block assignment

20

Table of Contents

Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Summary

21

Definition of Inodes

Every file has a unique inode Contain the information necessary for a process

to access a file Exist in a static form on disk Kernel reads them into an in-core inode to

manipulate them.

22

Contents of Disk inodes

File owner identifier (individual/group owner) File type (regular, directory,..) File access permission (owner,group,other) File access time Number of links to the file Table of contents for the disk address of data in a file

(byte stream vs discontiguous disk blocks) File size * inode does not specify the path name that access the file

23

Sample Disk Inode

File owner identifier File type File access permission File access time Number of links to the file Table of contents for the

disk address of data in a file File size

Owner mjb

Group os

Type regular file

Perms rwxr-xr-x

Accessed Oct 23 1984 1:45 P.M

Modified Oct 22 1984 10:3 A.M

Inode Oct 23 1984 1:30 P.M

Size 6030 bytes

Disk addresses

24

Distinction Between Writing inode and File

File change only when writing it. Inode change when changing the file, or when

changing its owner, permisson,or link settings. Changing a file implies a change to the inode, But, changing the inode does not imply that the file

change.

25

Contents of The In-core copy of The Inode

Fields of the disk inode Status of the in-core inode, indicating whether

Inode is locked Process is waiting for the inode to become unlocked In-core inode differs from the disk copy as a result of a change to

the data in the inode In-core inode differs from the disk copy as a result of a change to

the file data Inode number (linear array on disk, disk inode does not need

this field) Reference count

26

Inode Lock and Reference Count

Kernel manipulates them independently Inode lock

Set during execution of a system call to prevent other processes from accessing the inode while it is in use.

Kernel releases the lock at the conclusion of the system call

Reference count Kernel increase/decrease when reference is active/inactive Prevent the kernel from reallocating an active in-core inode

27

Table of Contents


28

Direct and Indirect Blocks in Inode

direct0

direct1

direct2

direct3

direct4

direct5

direct6

direct7

direct8

direct9

single indirect

double indirect

triple indirect

Inode Data Blocks

29

Byte Capacity of a File

System V UNIX. Assume that Run with 13 entries 1 logical block : 1K bytes Block number address : a 32 bit (4byte) integer

1 block can hold up to 256 block number (1024byte / 4byte) 10 direct blocks with 1K bytes each=10K bytes 1 indirect block with 256 direct blocks= 1K*256=256K bytes 1 double indirect block with 256 indirect blocks=256K*256=64M bytes 1 triple indirect block with 256 double indirect blocks=64M*256=16G

Size of a file : 4G (232), if file size field in inode is 32bits* Refer to the link on our class webpage for the case of current UNIX file system.

30

Byte Offset and Block Number

Process access data in a file by byte offset. The file starts at logical block 0 and continues to a

logical block number corresponding to the file size Kernel accesses the inode and converts the logical file

block into the appropriate disk block

31

Conversion of Byte Offset to Block Number Algorithm bmap /* block map of logical file byte offset to file system block */Input : inode, byte offsetOutput: (1)block number in file system, (2)byte offset into block, (3)bytes of I/O in block, (4)read ahead block number

calculate logical block number in file from byte offset;calculate start byte in block for I/O; /* output 2 */calculate number of bytes to copy to user; /* output 3 */check if read-ahead applicable, mark inode; /* output 4*/determine level of indirection;while(not at necessary level of indirection)

calculate index into inode or indirect block from logical block number in file;

get disk block number from inode or indirect block;release buffer from previous disk read, if any (algorithm brelse);if(no more levels of indirection) return (block number);read indirect disk block (algorithm bread);adjust logical block number in file according to level of indirection;

32

Block Layout of a Sample File and Its inode

4096

228

45423

0

0

11111

0

101

367

0

428

9156

824

331 3333

331Single indirect9156

Double indirect

3333Data block

367Data block

0 75

Byte 9000 in a file -> 8block 808th byte

Byte 350,000 in a file

0

88

11

(10K+256K)

816th byte

33

Block Entry in the Inode is 0

Logical block entry contain no data. Process never wrote data into the file at that byte

offset No disk space is wasted Cause by using the lseek and write system call

34

Table of Contents


35

Directories A directory is a file Its data is a sequence of entries, each consisting of an

inode number and the name of a file contained in the directory

Path name is a null terminated character string divided by “/”

Each component except the last must be the name of a directory, last component may be a non-directory file

36

Directory Layout for /etc

Inode Number

(2 bytes)

File Names

83

2

1798

1276

...

0

95

188

.

..

init

fsck

…

crash

mkfs

inittab

Each entry : inode number and filename

Directories : How

How do you find files? Read the directory, search for the name you want (checking for wildcards)

How do you list files (ls) Read directory contents, print name field

How do you list file attributes (ls -l) Read directory contents, open inodes, print name + attributes

37

38

Table of Contents


39

Algorithm for Conversion of a Path Name to an Inode

Algorithm namei /* convert path name to inode */Input : path nameOutput : locked inode{

if(path name starts from root) working inode = root inode (algorithm iget);else working inode = current directory inode (algorithm iget);while(there is more path name){

read next path name component from input;verify that working inode is of directory,access permission

OK;if(working inode is of root and component is “..”)

continue; /* loop back to while */read directory (working inode) by repeated use of algorithms

bmap,bread and brelse;…

40

Algorithm for Conversion of a Path Name to an Inode

if(component matches an entry in directory (working inode)){

get inode number for matched component;release working inode (algorithm iput);working inode=inode of matched

component(algorithm iget); } else /* component not in directory

return (no inode);}return (working inode);

}

41

Table of Contents

Inodes Structure of a Regular File Directories Conversion of a Path Name to an Inode Super Block Inode Assignment to a New File Allocation of Disk Blocks Other File Types Summary

42

Super block File System

Super block consists of the size of the file system the number of free blocks in the file system a list of free blocks available on the file system the index of the next free block in the free block list the size of the inode list the number of free inodes in the file system a list of free inodes in the file system the index of the next free inode in the free inode list lock fields for the free block and free inode lists a flag indicating that the super block has been modified

boot block super block inode list data blocks

43

Table of Contents


44

Inode Assignment to a New File

Super block contains an array to store the index numbers of free inodes in the file system

remembered inode 이 inode 보다 작은 번호의 inode 는 superblock free list

에 없음을 보장한다 . reduce free inode search time

45

Algorithm for Assigning New InodesAlgorithm ialloc /* allocate inode */Input : file systemOutput : locked inode{

while(not done){if(super block locked) {

sleep(event super block becomes free); continue;}if(inode list in super block is empty){

lock super block;get remembered inode for free inode search;search disk for free inodes until super block full,

or no more free inodes (bread and brelese);unlock super block;wake up (event super block becomes free);if(no free inodes found on disk) return (no inode);set remembered inode for next free inode search;

}

46

Algorithm for Assigning New Inodes

/* there are inodes in super block inode list */get inode number from super block inode list;get inode (algorithm iget);if(inode not free after all) {

write inode to disk;release inode (algorithm iput);continue; /* while loop */

}/* inode is free */initialize inode;write inode to disk;decrement file system free inode count;return (inode);

} // end of while}

47

Assigning Free Inode from Middle of List

free inodes 83 48 empty

18 19 20 array1

Super Block Free Inode List

index

free inodes 83 empty

18 19 20 array2


index

48

Assigning Free Inode – Super Block List Empty

index

470 empty

array1


index

0

535 free inodes 476 475 471

array2Super Block Free Inode List

048 49 50

remembered inode

49

Algorithm for Freeing Inode

Algorithm ifree /* inode free */Input : file system inode numberOutput : none{

increment file system free inode count;if(super block locked) return;if(inode list full){

if(inode number less than remembered inode for search)set remembered inode for search = input inode

number;} else

store inode number in inode list;return;

}

50

Placing Free Inode Numbers Into the Super Block

535 476 475 471

free inodesremembered inode

Original Super Block List of Free Inodes

index

Free Inode 499

499 476 475 471

free inodesremembered inode index

Free Inode 601

499 476 475 471

free inodesremembered inode index

51

Table of Contents


52

Linked List of Free Disk Block Numbers

109 106 103 100 …………………………..

211 208 205 202 …………………… 112

109

310 307 304 301 …………………… 214

211

409 406 403 400 …………………… 313

310

Super block list

53

Algorithm for Allocating Disk BlockAlgorithm alloc /* file system block allocation */Input : file system numberOutput : buffer for new block{

while(super block locked) sleep (event super block not locked);remove block from super block free list;if(removed last block from free list){

lock super block;read block just taken from free list (algorithm bread);copy block numbers in block into super block;release block buffer (algorithm brelse);unlock super block;wake up processes (event super block not locked);

}…

54

Algorithm for Allocating Disk Block

…get buffer form block removed from super block list (algorithm getblk);zero buffer contents;decrement total count of free blocks;mark super block modified;return buffer;

}

55

Requesting and Freeing Disk Blocks

109 …………………………………………………………

211 208 205 202 …………………………….. 112

109 949 …………………………………………………..

211 208 205 202 ………………………………. 112

super block list

original configuration

109

109

After freeing block number 949

56

211 208 205 202 ……………………………… 112

344 341 338 335 ………………………………. 243

After assigning block number(109)

replenish super block free list

211

109 ………………………………………………………..

211 208 205 202 ………………………………. 112

109

After assigning block number(949)

Requesting and Freeing Disk Blocks

unix file systems acknowledgement : soongsil univ. presentation materials 단국대 최종무...

Documents