TheDesignandImplementationofaLog-StructuredFileSystem

MendelRosenblumandJohnK.Ousterhout

2016-05-30

오기환,이종백,김문학

TechnologyTrend

• Processorspeedimprovingexponentially

• Diskcapacityimprovingexponentially– But,nottransferbandwidthandseektimes

• Memorycapacityimprovingexponentially– Largermemorysizesmeanlargercaches

• Cacheswillcapturemostreadaccesses• Disktrafficwillbedominatedbywrites• Cachescanactaswritebuffersreplacingmanysmallwritesbyfewerbiggerwrites

2016.5.30. 2

Problemswithexistingfilesystems

• Problem1:Fileinformationisspreadaroundthedisk– inodes areseparate fromfiledata– 5diskI/Ooperationsrequiredtocreateanewfile

• directoryinode,directorydata,file inode (twiceforthesakeofdisasterrecovery),filedata

– Results:lessthan5%ofthedisk’spotentialbandwidthisusedforwrites

• Problem2:Metadataupdatesaresynchronous– Requiredforconsistency– Lessefficientthanasynchronouswrites

Log-structuredFileSystem

• Writeallmodificationstodisksequentially inalog-likestructure

– Convertmanysmallrandomwritesintolargesequentialtransfers– Usefilecacheaswritebuffer

• Twoissuesinobtaininggoal– Retrieving Informationfromthelog– Managingfreespace

Log-structuredFileSystem

• TheLOG– Onlystructureondisk

– Containsinodes anddatablocks

– Includesindexinginformationsothatfilescanbereadbackfromthelogrelativelyefficiently

– Mostreadswillaccessdatathatarealreadyinthecache

Filelocationandreading

• StillusesFFS’sinode structure.Butinodes arenotlocatedatfixedpositions.

• Inode mapisusedtolocateafile’slatestversionofinode.Inode mapitselfislocatedindifferentplacesofthedisk,butitslatestversionisloadedintomemoryforfastaccess.

• FilereadingperformanceofLFSissimilartoFFS.

DisklayoutsofLFSandUNIX

FreeSpaceManagement

• Fragmentation fromdeletedandoverwrittenfiles

• Twoapproachestoreducefragmentation– Threading

• Leavelivedatainplaceandthreadthroughthefreeextents• Reducesabilityofsequentialwrites

– Copying• Copylivedataanappendtothefrontofthelog• Leaveslargerfreeextentsforuse• Copyingisveryexpensiveonlong-livedfiles

• SpriteLFSusesacombinationofboththreadingandcopying

Threading&Copying

SegmentCleaningMechanism

• Segmentcleaningprocessinvolves1. Readinganumberofsegmentsintomemory2. Identifyingthelivedata3. Writingthembacktoasmallernumberofcleansegments

• Howareliveblocksidentified?– Eachsegmentmaintainsasegmentsummaryblock toidentifywhatisineachblockandwhichinode thisblockbelongsto

– crosscheckblockswithowninginode’s blockpointers

SegmentCleaningPolicies

• Fourpolicyissues1. Whenshouldthesegmentcleanerexecute?2. Howmanysegmentsshoulditcleanatatime?3. Whichsegmentsshouldbecleaned?4. Howshouldtheliveblocksbegroupedwhentheyarewrittenout?

• Writecost– Usedtocomparecleaningpolicies– Averageamountoftimethediskisbusyperbyteofnewdatawritten,includingallthecleaningoverheads

– 1.0isperfect whilehighermeansfractionsofdiskbandwidtharebeingutilized

SimulationResults

• Considertwofileaccesspatterns– Uniform– Hot-and-cold: (100- x)%oftheaccesses involvex %ofthefiles90%oftheaccesses involve10%ofthefiles

Cost-BenefitPolicy

• GreedyPolicydatashowsthathot-and-coldsegmentsshouldbetreateddifferently– Coldsegmentsshouldbecleanedathighutilization– Hotsegmentsshouldbecleanedatlowutilization

• Cost-Benefitpolicyrateseachsegmentwiththebenefitofcleaningthesegmentandthecostofcleaningthesegment

Cost-BenefitPolicy

CrashRecovery

• Checkpoint– Positioninthelogatwhichallfilesystemstructuresareconsistentandcomplete

– CheckpointRegion• Containstheaddressesofalltheblocksintheinode mapandsegmentusagetable,plusthecurrenttimeandapointertothelastsegmentwritten

• Performedatperiodicintervals• Systemusesthecheckpointregiontoreturnlogtothisstate

• Roll-forward– Usesdataafterthecheckpointtorecoverasmanyfilesaspossible

Micro-BenchmarkResults

SmallFileBenchmark LargeFileBenchmark

CleaningOverheads

• Testedon5differentfilesystemsover4monthperiod

• Waitedseveralmonthstoallowfilesystemtobalance

• Writecostsmallerthansimulated– Thiswasduetoblocksizesusedinsimulation

Conclusion

• Log-structuredfilesystem– WritesmuchlargeramountsofnewdatatodiskperdiskI/O– Usesmostofthedisk’sbandwidth

• Freespacemanagementdonethroughdividingdiskintofixed-sizesegments

• Lowestsegmentcleaningoverheadachievedwithcost-benefitpolicy

FLASHMEMORYSTORAGE(SSD)

5/30/16 19

SolidStateDrive(SSD)

• SSDstypicallyuseNANDFlash

• Lowerlatency

• HigherIOPSandThroughput

• LowerPower

• Morereliable

SSDComponents

• DRAM

• NANDController

• NANDFlash– Pages:Smallestunitthatcanberead/written– Eraseblock:Groupsofpagestobeerased– LimitedP/ECycles

• Firmware– FlashTranslationLayer(FTL)andetc.

21

NANDFlashMemory

• Read/WriteUnit:Page(4KB~32KB)

• Noin-placeupdate:‘Erase-before-write’– EraseUnit:Block(512KB~)– Blockwearsoutafterlimitederases

22

Page0

Block1Page1

Pagem-1

Blockn-1Block0

FlashChip

Read Write Erase25us(2KB)

200us(2KB)

1,500us(128KB)

NANDFlash[1]

Access latencyMedia

[1]:SamsungLarge-blockSLCNAND(K9WAG08U1A)

FlashTranslationLayer(FTL)

• AsoftwarelayerthatallowstheflashmemorytolooklikeaHDD– Addressmapping:LogicaltoPhysical– Garbagecollection,Wear-leveling, etc.

23

FileSystem

Low-levelDeviceDriver

Flashmemory

Read Write

FileSystem

Low-levelDeviceDriver

FTL

Flashmemory

Read Write

Read Write Erase

Mismatch

Page-LevelMappingFTL

24

(A)(B)

Mapping table

1 Initial page address mapping for page (B)

(B)(C)

Mapping table

2 Update data (write new data) in an empty page

(A)X

(B)(C)

Mapping table

3 Page address re-mapping for page (B)

(A)X

(C)

X

1 Collect (copy) valid pages into an empty block

2 Page address re-mapping

(B)(C)(A)

XXX

(B)

XX

(C)

(A)

(B)(C)(A)

XXXX

XX

X

X

(B)(C)(A)

Empty block Empty block

3 Erase invalid blocks (can be re-used now)

WriteAmplificationFactor(WAF)

• Bytesphysicallywritten toNANDversusbytesrequestedfromPC/Server

• FactorsthatcanaffectWAF– SSDComponents

• FlashTranslationLayer(FTL):WearLeveling,GarbageCollection• Over-provisioning

– HostApplication• WriteProfile(Randomvs.Sequential)• FreeUserspace/UseofTRIM

25

WAF=ByteswrittentoNAND

BytesrequestedfromHost

ImpactofWriteProfile

26

1 Initial state by random writes

2 Series of random writes

(B)(F)(C)(E)

(G)(A)

(D)

(A)

(F)X

(E)

(B)X(C)X

(G)X

X

(D)

3 Erase block after copying 2 valid pages

1 Initial State by sequential writes

2 Series of sequential writes

(A)(B)(C)(D)

(F)(G)

(E)

(E)

(B)(C)(D)

XXXX

(F)(G)

X

(A)

(E)

(B)(c)(D)

(F)(G)

X

(A)

3 Erase block without copying valid page

(B)(C)(G)

X

X

(D) (A)

(F)X

(E)

ImpactofOthers

27

※More chances to be invalidated

※ Use TRIM commands

(A)

(F)X

(E)

(B)X(C)X

(G)X

X

(D)

※ Align page boundaries

(E)

(B)(c)(D)

(X)(X)(X)(X)

※ Erase block after copying 3 valid pages

(F)(G)(A)

(F)

(B)(G)(C)

(A)

XX

(E)

XXXX

XX

X

(D)

Data

Data

DifferentGC&Wear-leveling

MoreOver-Provisioning

SSD-awareapplications#1

SSD-awareapplications#2

F2FS:ANewFileSystemforFlashStorage

Changman Leeet,FAST’2015

Introduction

• Randomwritesisbadtoflashstoragedevice– Freespacefragmentation

• Sustainedrandomwriteperformancedegrades• Lifetimereduced

• Sequentialwriteorientedfilesystems– LFS,CoW filesystems

• F2FSContribution– DesignandimplementationofapracticalLFStofullyleverageandoptimizetheusageofNANDflashsolutions

– PerformancecomparisonwithLinuxfilesystems(Ext4,Btrfs,Nilfs2)• Mobilesystemandserversystem

29

KeyDesignConsiderations

• Flash-friendlyon-disklayout

• Cost-effective indexstructure

• Multi-head logging

• Adaptive logging

• Fsync accelerationwithroll-forwardrecovery

30

Flash-friendlyOn-DiskLayout

• Flashawareness– AlltheFSmetadataarelocatedtogetherforlocality– Startaddressofmainareainalignedtothezonesize– Filesystemcleaningisdoneinaunitofsection(FTL’sGCunit)

• CleaningCostReduction– Multi-headloggingforhot/colddataseparation

31

F2FSIndexStructure

• Restrainedupdatepropagation– NodeAddressTranslation(NAT)method

• Multi-headlog

32

SB CP NAT

SegmentInfo.Table(SIT)

SegmentSummaryArea(SSA)

Inode fordirectory

Inode forregularfile

IndirectNode

DirectoryData

FileData

DirectNode

FileData

Multi-headLogging

• Datatemperature classification

• Separation ofmulti-head logsinNANDflash

33

Cleaning

• Cleaningisdone insectionunit– SectiontobealignedwithFTL’sGCunit

• Cleaningprocedure– Victimselection:ThroughreferencingSIT

• Greedyforforegroundcleaning• Cost-benefit forbackgroundcleaning

– Validblockcheck:ThroughSSA– Migration:copy-backvalidblocks– Markvictimsectionas“pre-free”

• Pre-freesectionsarefreedafterthenextcheckpoint

34

AdaptiveLogging

• Toreducecleaningcostathighlyagedconditions,F2FSchangeswritepolicy– Appendlogging(loggingtocleansegments)

• Causessequentialwrites– Threaded logging(loggingtodirtysegments)

• Causesrandomwrites

35

CheckpointandRollback

• Maintainsshadowcopyofcheckpoint,NAT,SITblocks

• Recoversthelatestcheckpoint

• KeepsNAT/SIT journalincheckpointtoavoidNAT,SITwrites

36

CheckpointandRollback

37

Recovery

• Fsync handling– Onfsync,checkpointisnotnecessary– Directnodeblocksarewrittenwithfsync mark

• Roll-forwardrecoveryprocedure– Searchmarkeddirectnodeblocks– UpdateSIT;invalidateolddatablocks– UpdateNAT,SIT– Createcheckpoint

38

Evaluation

• ExperimentalEnvironment

39

Mobilebenchmark

• InF2FS,morethan90%ofwritesaresequential

• F2FSreduceswriteamountperfsync– usingroll-forwardrecovery

40

Serverbenchmark

• PerformancegainismoreonSATASSD– Varmail:2.5xontheSATA,1.8xonthePCIe– Oltp:16%ontheSATA,13%onthePCIe

• F2FSissueslargesizediscard(TRIM)command

41

EffectofMulti-headLogging

• Usingmorelogsgivesbetterhot/coldseparation–2Logs:node,data–4Logs:hotnode,coldnode,hotdata,colddata

42

CleaningCostAnalysis

• F2FSusesadaptiveloggingunderhighutilization–Even in97.5%util.,WAFislessthan1.025

• Withoutadaptivelogging,WAFgoesuptomorethan3

43

AdaptiveLoggingPerformance

• Gracefulperformancedegradationunderhighlyagedvolumeconditions

44

Conclusion

• F2FSfeatures– Flash-friendlyon-disklayout– Costeffective indexstructure(NAT)– Multi-headlogging– Adaptivelogging– Roll-forwardrecovery

• F2FSshowperformancegainoverotherFSs

• F2FSispubliclyavailableinLinuxmainline

45