0911 juluosdev a_journey_of_filesystem_on_jos

A journey of file system on JOS

Name :waylingDate: 2012/09/11

Who am I?

●JuluOSDev 星系001● virtual file system (2011-12-20 )

● http://www.juluos.org/home/files ● 出沒於TOSSUG & H4●[email protected]

wayling

http://www.juluos.org/home/files

http://www.juluos.org/home/files

mailto:[email protected]

mailto:[email protected]

Introduction JOS

●MIT 6.828 Operating System Engineering●JOS是一個有基本雛形的作業系統，擁有傳統unix-like functiom(fork, exec…) ，但是實作方式是exokernel style(實作的unix-like functiom大部份皆為user level的函式，不實作於kernel level) 。●主要部份:

● Booting● Memory management● User-level environments● Preemptive multitasking● File system and spawn● A shell

Lectures:Frans Kaashoek Robert Morris

JOS File System

●Part1 : File system struct ● In-memory and On-disk struct

●Part 2: Client/Server file system access● Remote procedure call (RPC) base on IPC +

process

JOS File System


●Part 2: Client/Server File System Access● remote procedure call (RPC) base on IPC +

process

The File System

●File system是memory跟disk操作結合而成的，意及我們在存取檔案時，memory跟disk都需要有對應的struct來存放表示。

●static + dynamic組成file system● Static : 格式化硬碟成特有檔案系統格式 ex：

ext2、ext3。● Dynamic ：runtime時作業系統保存對應的資料

結構，藉此操作disk，提供檔案操作給使用者。

JOS file system item(1/2)

●Sectors and Blocks● Sector :Disk 實體磁區 (512 B) 。● Block : file system 存取單位 (4KB) 。

●Superblocks (disk & memory)● 保存block size、 disk size 、以及其他meta-data

用於尋找到root directory 。●Block Bitmap (disk & memory)

● 管理 free disk blocks 。

JOS file system item(2/2)

●File Meta-data ● 保存”檔案”的必要資訊，檔名、大小、以及

block pointer(direct/indirect) 。●Directories versus Regular Files

● Directories主要保存多個file struct 。● Regular Files保存data block 。

JOS file system struct overview

File/Directory struct level

Block struct level

Disk struct (sector) level

User access

File/directorydata blocks

Free Block Bitmap

Spuerblock

BootSector Block 0

Block 1

Block 2

Tota

l dis

k si

ze : N

Blo

cks

B

itmap

siz

e:E

noug

h bl

ock

to h

old N

bits

Block N -1

Block struct level

struct File (256 bytes)

Name : “foo”Size : 54321Direct block pointers:

(10)

Indirect block pointer :

Indirect block (4096 bytes)

(1024)

Block 0

Block 1

Block 2

Block 3

Block 4

Block 5

Block 6

Block 9

Block 10

Block 11

Block 12

…..

File data blocks(4096 bytes each)

File/Directory struct level

JOS file system

●Disk access● user-level file system - IDE disk driver 實做於user

space 。● polling vs interrupt-driven 。

●The Block Cache● 運用虛擬記憶體的特性，當程式存取檔案時轉換為對其

地址空間做存取 (0x10000000 - 0xD000000， 3GB) 。●File operations

● Open/Read/Write ，基本檔案操作。

Format a file system (1/2)

#fsformat fs.img 1024 init newmotd motdstruct File

struct File

struct File struct File

struct File

/

init

newmodemode

init

dir

reg

regreg

reg

data blockdata block

……

=>請參閱source code (fsformat.c)

struct File

Name : “/”Size : 54321Direct block pointers:

11121314

struct File

struct File

Name : “init”Size : 54321

Name : “newmode”Size : 54321

struct File

Name : “init”Size : 54321

212223

35363738

52535455

Format a file system (2/2)

#fs.img (block view)

使用block bitmap紀錄block

分配狀況(不一定連續分配) 。

JOS file system service


●Part 2: Client/Server file system access● remote procedure call (RPC) base on IPC +

process

JOS’s file overview

●Server● server.c

● Server 端主體，處理clinet請求。(JOS定義八種檔案操作請求)● FSREQ_OPEN, 、FSREQ_SET_SIZE 、 FSREQ_READ● FSREQ_WRITE 、 FSREQ_STAT 、 FSREQ_FLUSH● FSREQ_REMOVE 、 FSREQ_SYNC

● fs.c● File system主體，處理block、File操作。

● bc.c● Block cache，類似copy on write 的機制。

●Client● file.c

● Client端主要函式，對server發起IPC操作。● fd.c

● file description (fd)是server跟client間的橋樑。

Client/Server file system access

●JOS使用IPC機制實作client/server架構的file system service。●File system Server (FSServ) :

● 作為user space porcess，提供檔案操作服務，藉由IPC提供其他process所需之請求。

●Client :● 透過FSServ來存取檔案。

●FSServ 特徵:● Process id 1。● Block cache。● 回應client的IPC request。

FSServ Block cache

3GB

0x10000000

0xD000000

FSServ address space

1

2

1.申請實體記憶體page。2.讀取block大小到記憶體中。(4096byte = 512 byte *8)

JOS IPC (Inter-process communication ) (1/2)

●void ipc_send(envid_t to_env, uint32_t val, void *pg, int perm)

→static int sys_ipc_try_send(envid_t envid, uint32_t value, void *srcva, unsigned perm)

對某個process傳送資料(32bit) ，也可指定一個地址(小於UTOP) ，跟對方process共享該page。

=>請參閱source code (ipc.c & syscall.c)

JOS IPC (Inter-process communication ) (2/2)

●int32_t ipc_recv(envid_t *from_env_store, void *pg, int *perm_store)

→static int sys_ipc_recv(void *dstva)

=>請參閱source code (ipc.c & syscall.c)

Blocking自己等待對方把資料傳送完畢。可指定地址(小於UTOP) ，決定要跟傳送者共享page的位置。

Client (fs.c)

// Send an inter-environment request to the file server, and wait for// a reply. static int fsipc(unsigned type, void *dstva){

if (debug){

…….}ipc_send(envs[1].env_id, type, &fsipcbuf, PTE_P | PTE_W | PTE_U);return ipc_recv(NULL, dstva, NULL);

}

封裝IPC成一個fsipc給client使用。

IPC訊息格式定義於 fs.h。

=>請參閱source code (file.c & file.h)

FSServ (serv.c)

●初始OpenFile table。 //用於管理FSServ所開啟的檔案。 ●struct OpenFile opentab[MAXOPEN] = {

{ 0, 0, 1, 0 }};●struct OpenFile { uint32_t o_fileid; struct File *o_file;

int o_mode; struct Fd *o_fd;};

=>請參閱source code (serv.c)

FSServ Client

fd

File

fd

FSREQ_OPEN

ipc_send

ipc_sendipc_recv ipc_recv

1

1

2

3

3

0xD0000000

0xD0000000

0x0ffff000

1.Client傳送一個IPC request 讓FSServ接收，request中夾帶路徑，並申請fd待，FSServ回應fd page，用以共享page。2.FSServ依路徑開啟File struct。3.回應Client,並把開啟的fd address map到clinet的fd上。

共享page


FSServ Client

fd

File

FSREQ_WRITE

ipc_send

ipc_sendipc_recv ipc_recv

1

1

2

3 3

0xD0000000

0x0ffff000

1.Client傳送一個IPC request讓FSServ接收，request中夾帶file id及將寫入的資料buffer。2.將資料buffer寫入檔案的block中。3.回應Client，將file pointer位址返回。


fd

JOS file system RPC

●RPC base on IPC (ex : call file system server read a file)

read( )

lib/fd.c

devfile_read ( )

lib/file.c

fsipc ( )

lib/file.c

ipc_send ( )

file_read ( )

fs/fs.c

serve_read ( )

fs/serv.c

serve ( )

fs/serv.c

ipc_recv( )

RPC mechanism

JOS IPC

Regular env FSServ env

Reference

●[mit 6.828 lab 5]http://pdos.csail.mit.edu/6.

828/2011/labs/lab5/

http://pdos.csail.mit.edu/6.828/2011/labs/lab5/

