nanddddd

7/29/2019 NANDDDDD

1/46

NAND FLASH TECHNOLOGY 1

CVR COLLEGE OF ENGINEERING M.TECH. 1ST YEAR-1ST SEMESTER VLSI-SD

1. INTRODUCTION

Flash memory has become a powerful and cost-effective solid-state

storage technology widely used in mobile electronics devices and other consumer applications.

Two major forms of Flash memory, NAND Flash and NOR Flash, have emerged as the

dominant varieties of non-volatile semiconductor memories utilized in portable electronics

devices. NAND Flash, which was designed with a very small cell size to enable a low cost-per-

bit of stored data, has been used primarily as a high-density data storage medium for consumer

devices such as digital still cameras and USB solid-state disk drives.

NOR Flash has typically been used for code storage and direct execution inportable electronics devices, such as cellular phones and PDAs. Recently, however, the

distinction between the two types of Flash memory has become less clear. New cell phone

controllers that support NAND Flash as an alternative to or an addition to NOR Flash have

helped make NAND a viable alternative for a broader array of applications. In addition, data

storage capacity and performance requirements in cell phones have increased significantly with

the growth of feature-rich phones that incorporate camera, music, video, gaming and other

functionality. NAND Flash has become an attractive alternative for the data storage aspects of

todays cell phones because of its higher speed write and erase performance as well as its low

cost-per-bit. As a result, designers of memory subsystems in portable electronics are now using

NAND in some traditional NOR-based applications. For todays full-featured cell phones, many

designers are utilizing memory architectures that combine NOR with NAND for data storage, or

are using NAND as the primary Flash memory in combination with low power DRAM in which

the program code can be shadowed and run. In either case, the different types of memory are

frequently stacked in Multi-Chip Packages (MCP) to create a single component. This overview

will briefly discuss the history of Flash memory development, compare and contrast NAND and

NOR Flash memory, and discuss the ways in which the two technologies are used today.

7/29/2019 NANDDDDD

2/46



2. NAND FLASH MEMORY

2.1 What is NAND flash memory: A form of memory, NAND flash is one of two flash

technologies (the other being NOR). Low power, low cost and extremely durable, NAND offers

high capacity data storage and brings faster erase, write, and read capabilities over NOR

architecture. Flash technology is non-volatile, meaning that it keeps stored information even

when the power is off. It got the name flash because it only took a few seconds to erase the

chipas compared to the main technology at the time, EPROMs, which took about 20 minutes

to erase under an ultraviolet light NAND Flash is a type of memory device called nonvolatile

memory. It is by far the most common nonvolatile memory used for mass storage.the following

table classifies different types of silicon memories:

Type Sub-type Example

Volatile: Retain data only

when power is on. Loss data

when power is off

Static memory: Retain data

indefinitelyas long as power is

on. Consumes none or very

little power to retain data.

SRAM such asCPU cache

Dynamic memory: Retain data

for a small period of time whe

power is on.require.periodoc

refresh to retain data.

Consumes power during

refresh

SDRAM

Nonvolatile: Retains data

regardless power

Programmable memory: Data

can be written into rhe device

many times.

NAND Flash, NOR Flash

one-tmie programmbale

memory:

Mask programmable ROM

Table: Major difference between NAND and other memory:

7/29/2019 NANDDDDD

3/46



From the system designers perspective, the biggest difference is

that NAND Flash is a serial storage device while most other memories are random access

memory. Random access device can be designed easily as the primary storage of a system.

Typical instriction and data fetch the CPU involved an address phase and data phase on the CPU

bus.Random access device ca retrive the required data easily.Subsequently and different address

locations can also be accesses with little penalty. To the contracy, serial storage device requieres

long access time for the initial data subsequent access to any nonconsecutive location also

requires large penalty. As a result, serial storage device such as NAND Flash requires special

NAND Flash controller to access data and is seldom used as the main memory of the system.

2.2 Difference between NAND and NOR:

There are major differences between NAND and NOR highlighted in Table 1. It

shows why NAND memories are ideal for high-capacity storages, while NOR memories are used

for code storage and execution.

NAND NOR

Capacity*1 32Gbit ~1Gbit

Access method Sequential Random

Interface I/O interface Full memory interface

Performance Fast read (serial access cycle)

Fast write

Fast erase (approx.

2ms/block)*2

Fast read (random access)

Slow write

Slow erase (approx.

1s/block)*3

Life Span 100 0001 000 000 10 000100 000

Price Low High

Table: The major difference between NAND and NOR

7/29/2019 NANDDDDD

4/46



The NAND Flash is a new flash configuration that reduces memory cell area

so that a lower bit cost can be achieved. In 1987, Toshiba proposed the NAND Flash, and its

NAND structured cell arranged as eight memory transistors in series. The NAND flash cell

array, fabricated by using conventional self-aligned dual polysilicon gate technology, had only

one memory transistor, one forth of a select transistor and one sixteenth of the contact hole area

per bit. This technology realizes a small cell area without scaling down the device dimensions.

The cell area per bit was half that of a DRAM using the same design rule of 1um (which was

used for the 1M bit DRAM). As a result, Toshiba realized that it was possible for NAND Flash

to be developed earlier than DRAM (for the same density) by one process generation. In

comparison, conventional EEPROM was behind DRAM by one process generation at that time.

The most important item regarding memories is the bit cost. In the case of a

semiconductor memory, the bit cost is dependent on the memory cell area per bit. Since the cell

area of NAND Flash is smaller than that of NOR Flash, NAND Flash always had the potential

from the start to be less expensive than NOR Flash. However, it takes a rather long time for a

NAND Flash to read out the first data byte compared to NOR Flash because of the resistance of

the NAND cell array, although it is much faster than the seek time for a hard disc by several

orders of magnitude. Therefore, the aim of NAND Flash is to replace hard disks.

7/29/2019 NANDDDDD

5/46



Fig: NAND and NOR memory cell

Figure below provides a summary of how NAND and NOR Flash vary for a number of important

design characteristics

7/29/2019 NANDDDDD

6/46



Figure: NAND cell

Figure: NOR cell

7/29/2019 NANDDDDD

7/46



The advantages of NAND Flash are that the erasing and programming times are short. The

programming current is very small into the floating gate because NAND Flash uses Fowler-

Nordheim tunneling for both erasing and programming. Therefore, the power consumption for

programming does not significantly increase even as the number of memory cells being

programmed is increased. As a result, many NAND Flash memory cells can be programmed

simultaneously so that the programming time per byte becomes very short. Conversely, the NOR

Flash can be programmed only by byte or word, and since it uses the hot electron injection

mechanism for programming, it also consumes more power and the programming time per byte

is longer.

The programming time for NOR Flash is typically more than a order of

agnitude greater than that of NAND Flash. The power consumption of NAND Flash or NOR

Flash is about one tenth that of a hard disk drive. Also, the seek time for semiconductor emories

is much faster than that of a hard disk. However, NAND Flash or NOR Flash must be erased

before reprogramming while a hard disk requires no erasure. Therefore, in the case of continuous

programming where the seek time is negligibly small, a hard disk drive can be programmed more

quickly.

For both for NOR Flash and NAND Flash, the endurance (which means the number of

cycles a block or chip can be reprogrammed) is limited. In order to replace the UV-EPROM with

Flash, and endurance of 1000 cycles was sufficient. It is estimated that at least 1,000,000 cycles

are required to replace a hard disk drive. NOR Flash is typically limited to around 100,000

cycles. Since the electron flow-path due to the hot electron injection for programming is different

from the one due to tunneling from the floating gate to the source for erasing, degradation is

enhanced. However, in NAND Flash, both the programming and erasing is achieved by uniform

Fowler- Nordheim tunneling between the floating gate and the substrate. This uniform

programming and uniform erasing technology guarantees a wide cell threshold window even

after 1,000,000 cycles. Therefore, NAND Flash has better characteristics with respect to

program/erase endurance. In some recent scaled NOR Flash memories, their erasing scheme has

7/29/2019 NANDDDDD

8/46



been changed from source side erasing to uniform channel erasing, which is the same as the

NAND Flash.

From a practical standpoint, the biggest difference a designer will notice when

comparing NAND Flash and NOR Flash is the interface. NOR Flash has a fully memory-mapped

random access interface like an EPROM, with dedicated address lines and data lines. Because of

this, it is easy to boot a system using NOR Flash. On the other hand, NAND Flash has no

dedicated address lines. It is controlled using an indirect I/O-like interface and is controlled by

sending commands and addresses through a 8 bit bus to an internal command and address

register. For example, a typical read sequence consists of the following: writing to the command

register the read command, writing to the address register 4 byte of address, waiting for the

device to put the requested data in the output data register, and reading a page of data (typically

528 bytes) from the data register. The NAND Flashs operation is similar to other I/O devices

like the disk drive it was originally intended to replace. But because of its indirect interface, it is

generally not possible to boot from NAND without using a dedicated state machine or

controller. However, the indirect interfaces advantage is that the pinout does not change with

different device densities since the address register is internal. Because NAND Flash is

optimized for solid-state mass storage (low cost, high write speed, high erase speed, high

endurance), it is the memory of choice for memory cards such as the SmartMediaTM, SDTM

card, CompactFlashTM, and MemoryStickTM

7/29/2019 NANDDDDD

9/46



3. NAND FLASH CONTROLLER

The serial structure used in NAND Flash allows very high storage

density but data from the memory block can only be read serially. The disadvantage, as

compared to RAM, is that data cannot be randomly accessed. But once a page of memory is

opened for read, data can be shifted out from the memory quickly. The NAND flash interface

also requires that commands to the NAND flash be sent serially to the device as a command

packet, instead of the parallel address and data signals in typical RAM. These are the major

reason that make interfacing with NAND flash memory much more complicated than interfacing

with typical SRAM or NOR flash devices. A NAND flash controller is design specifically to

handle all required tasks of accessing NAND flash device efficiently.

7/29/2019 NANDDDDD

10/46



3.1 COMMANDS SUPPORTED BY NAND FLASH DEVICES:

There are many commands, some manufacturer specific and supported only by a few devices

while some commands are universal to all NAND flash manufacturers. The most common

commands are

Reset ERASE PROGRAM PROGRAM CONFIRMATION READ DATA READ STATUS READ ID RANDOM READ RANDOM WRITE PAGE CACHE WRITE PAGE CACHE READ INTERNAL DATA MOVE TWO-PLANES READ TWO-PLANE WRITE , and others based on the manufacturers

7/29/2019 NANDDDDD

11/46



4. SLC and MLC

Single-level cell (SLC) and multi-level cell (MLC) Flash memory

are similar in their design. MLC Flash devices cost less and allow for higher storage density.

SLC Flash devices provide faster write performance and greater reliability, even at temperatures

above the operating range of MLC Flash devices. Table 1 provides a summary of the advantages

and disadvantages of SLC Flash and MLC Flash.

SLC MLC

High Density *

Low Cost per Bit *

Endurance *

temperature range *

Low Power Consumption *

Write/Erase Speeds *

Write/Erase Endurance *

Table: Qualities of SLC and MLC

These factors make SLC Flash a good fit in embedded systems, while MLC flash makes it

possible to create affordable mobile devices with large amounts of data storage.

4.1 Flash Memory Explained

7/29/2019 NANDDDDD

12/46



It is important to understand what makes up a Flash cell before explaining the variation between

SLC and MLC Flash. Each cell consists of a single transistor, with an additional floating gate

that can store electrons. Figure 2 shows the architecture of an SLC cell.

Figure : floating gate

A large voltage difference between the drain and the source, VdVs, creates a large

electric field between the drain and the source. The electric field converts the previously

nonconductive poly-Si material to a conductive channel, which allows electrons to flow between

the source to the drain.

The electric field caused by a large gate voltage, Vg, is used to bump electrons up

from the channel onto the floating gate. As an electron travels closer to the drain, it gains more

momentum and thus, more energy. But, this amount of energy is not enough to push an electron

onto the floating gate. Electrons with high momentum near the drain can sometimes bump into Si

(Silicon) atoms. This bump gives the electron enough energy to be pushed onto the floating gate.

The number of electrons on the floating gate affects the threshold voltage of the cell Vt. This

effect is measured to determine the state of the cell.

7/29/2019 NANDDDDD

13/46



4.2 Single-Level Cell (SLC) Flash

As the name suggests, SLC Flash stores one bit value per cell, which basically is a voltage

level. The bit value is interpreted as a 0 or a 1.

Value State

0 Programmed

1 Erased

Table :SLC levels

Since there are only two states, it represents only one bit value. As seen in Table

2, each bit can have a value of programmed or erased.

Figure: voltage reference for SLC

A 0 or 1 is determined by the threshold voltage Vt of the cell. The threshold voltage can be

manipulated by the amount of charge put on the floating gate of the Flash cell. Placing charge on

the floating gate will increase the threshold voltage of the cell. When the threshold voltage is

high enough, around 4.0V, the cell will be read as programmed. No charge, or threshold voltage

< 4.0V, will cause the cell to be sensed as erased.

7/29/2019 NANDDDDD

14/46



SLC Flash is used in commercial and industrial applications that require high performance and

long-term reliability. Some applications include industrial grade Compact Flash cards or Solid

State Drives (SSDs).

4.3 Multi-Level Cell (MLC) Flash

As the name suggests, there are multiple values that an MLC cell can represent. The values can

be interpreted as four distinct states: 00, 01, 10, or 11.

value State

00 Fully programmed

01 Partially programmed

10 Partially erased

11 Fully erased

Table: MLC Levels

These four states yield two bits of information. As seen in table 3, the value of the two bits range

from fully programmed to fully erased.

7/29/2019 NANDDDDD

15/46



Figure :voltage reference for MLC

As seen in figure 2, a Flash cells ability to store charge is why MLC technology works. Since

the delta between each level has decreased, the sensitivity between each level increased. Thus,

more rigidly controlled programming is needed to manipulate a more precise amount of charge

stored on the floating gate. In order for a Flash cell to be considered MLC technology, the cell

must exhibit two characteristics:

1. Precise charge placement

2. Precise charge sensing

Thus, MLC Flash works the same way as SLC Flash. The threshold voltage Vt, is used to

manipulate the state of the Flash. Once again, the amount of charge on the floating gate is what

determines the threshold voltage.

As seen in figure 4, current MLC technology uses two bits, or 4 levels. However, it is possible to

hold more bits. Equation 1 is a generic equation to follow to determine how many states are

needed for the desired bits.

Equation 1 States = 2N

7/29/2019 NANDDDDD

16/46



N is equal to the number of desired bits per cell. For example, for a cell to hold three bits, you

need eight states equal to: 000, 001, 010, 011, 100, 101, 110, 111 . MLC Flash is used in

consumer applications that do not require long term reliability such as consumer grade USB

Flash drives, portable media players, and Compact Flash cards.

4.4 SLC and MLC Compared

Now that the differences between SLC and MLC have been explained, lets compare their

specifications to help further make a distinction between the two grades.

SLC MLC

Density 16Mbit 32Mbit 64Mbit

Read Speed 100ns 120ns 150ns

Block Size 64Kbyte 128Kbyte

Architecture x8 x8 / x16

Endurance 100,000 cycles 10,000 cycles

Operating

Temperature

Industrial Commercial

Table: specification comparison of SLC and MLC

Lets compare each characteristic in table 4. Using the same wafer size, you can double the

density of the MLC Flash by using the charge placement technology. Thus, MLC has greater

densities.

7/29/2019 NANDDDDD

17/46



The read speeds between SLC and MLC are comparable. Reading the level of the Flash cell

compares the threshold voltage using a voltage comparator. Thus, the architecture change does

not affect sensing. In general, the read speeds of Flash are determined by which controller is

used.

The endurance of SLC Flash is 10* more than MLC Flash. The endurance of MLC Flash

decreases due to enhanced degradation of Si. This is a main reason why SLC Flash is considered

industrial grade Flash and MLC Flash is considered consumer grade Flash.

Higher temperatures cause more leakage in the cells. Combined with the increased sensitivity

required to differentiate between the levels, this leakage will cause the sensors to read the wrong

level. As a result, the operating temperature of MLC spans only the commercial range. Leakage

is not significant in SLC Flash and thus, it can operate in an industrial temperature range.

4.5 SPARE COLUMN

NAND flash devices organize 512bytes or 2048 bytes of data into a page. There are also

16 or 64 bytes of extra data called the spare column associated with each page. The spare

columns are fully addressable by the user and is typically used for storing error correction code

(ECC) and other management information to improve data integrity.

7/29/2019 NANDDDDD

18/46



5.MANAGING NAND FLASH

In order to use NAND flash effectively, the NAND flash must be managed by some kind of a

controller (either software or hardware). This is necessary in order to make the NAND flash

appear to the system as ideal block device.

5.1 Bad Block Management

In a brand new device, the standard NAND flash specification allows for the existence of initial

bad blocks. Standard NOR flash devices have extra spare memory blocks that are used to replace

bad blocks, but NAND flash devices have a minimal amount of redundant memory blocks

because it was always expected that an intelligent controller would ignore the bad blocks. Since

it was expected that NAND flash would be used for solid state mass storage, blocks would

eventually wear out; therefore, it was expected that the system be able to handle bad blocks that

would form during use. The standard factory location for the bad block byte is byte 517 (the

518th byte) of a NAND page. If this byte is FFh, the block is good, otherwise, the block is bad

(typically indicated by 00h). This format for marking bad blocks is from SmartMedia card

(NAND flash in a removable card package) and was standardized by the SSFDC Forum (Solid

State Floppy Disk Cardthe former name of SmartMedia). If additional bad blocks form during

use, the block is marked bad. Generally, this is possible even if the block that you are marking is

considered bad. To distinguish between factory marked bad blocks and blocks that go bad during

use, two flag values are defined in the SmartMedia format: 00h (for initial factory marked bad

blocks) and F0h (for blocks that go bad during system use). An alternative approach to the in

block method of keeping track of bad blocks is to maintain a bad block table. However, where

to you store a bad block table since blocks could be bad? For NAND TSOP devices only, the

first block of the NAND flash (block 0) is guaranteed to be good. Thus, block 0 could be used to

hold a bad block table if desired. However, at power up, many systems simply scan the first page

of each block to determine whether they are good or bad and build a bad block table in RAM.

7/29/2019 NANDDDDD

19/46



5.2 Error Correcting Code

The use of an error correcting code is essential in order to maintain the integrity of stored code.

Soft errors (especially during programming) occur at a rate of approximately 10-10 or about 1 bit

per 10 billion bits programmed. Single bit correcting (two bit error detecting) Hamming code

issufficient for NAND flash. Toshiba has developed C sample code for implementing Hamming

code. It is available in a separate document entitled the SmartMediaTM ECC Reference Manual.

5.3 Wear Leveling

If flash memory had infinite write/erase endurance, wear leveling would not be necessary.

However, unlike magnetic media, flash memory eventually wears out and no longer programs

or26 erases in the allotted amount of time. Because the design of typical file systems assumedthe characteristics of magnetic media, certain physical locations may be repeatedly rewritten. For

example, in the DOS FAT file system, the FAT and directory areas must be modified multiple

times each time a file is written or appended. When multiplied by the thousands of files in a

typical file system, the FAT and directory areas of the disk will experience vastly more writes

than any other area of the disk. When flash memory is used to emulate a disk drive, the physical

areas of the flash that contain the FAT and directory would be worn out first, leading to early

failure of the file system stored on the flash. In order to spread out the writes across as much of

the flash as possible, a wear levelling algorithm is implemented by the controller (software or

firmware in a hardware controller) which translates a logical address to different physical

addresses for each write. Generally, this logical to physical lookup table is implemented in RAM

and is initialized at power up by reading each physical block in the NAND flash to determine its

logical block value. Ideally, wear leveling is intrinsic to the file system itself. Several new file

system exist which write new data sequentially rather than overwriting a fixed location. These

file systems use a technique known as journaling. For flash memory, JFFS2 (Journaling Flash

File System 2) and YAFFS (Yet Another Flash File System) exist which automatically spread of

2KB) is updated when writing a 10MB file to a NAND Flash memory with a physical erase out

wear by writing sequentially to free flash space.

7/29/2019 NANDDDDD

20/46



5.4 Lifetime Without Wear Leveling

For systems that have a file allocation table (FAT) based file system, the FAT table is always

stored in the same virtual blocks. Frequent FAT table updates are required during data WRITE

operations, which implies frequent erase cycles on the same physical blocks, hence a reduced

NAND Flash lifetime.

The following example calculates how many times a FAT table (FAT32 and a cluster size unit of

16KB (NAND small page device). To write a file of 10MB, 5KB entries in FAT and 5KB

clusters in the file system are required. This corresponds to 640 physical NAND Flash blocks.

This means that the file can be written at the same location 20 times:

20 5120 = 102400

This is greater than the maximum number of program/erase cycles. The expected NAND Flash

lifetime can be calculated as follows:

>Expected lifetime=size of nand flash*no of erase cycles*FAT overhead/bytes written per day

This means that if the application writes at 3KB/s, the expected lifetime of the NAND blocks is:

Expected lifetime = 10Mbyte 20 0.7 /(3Kbytes) 24 60 60 = 0.55 days

In a NAND Flash, when virtual blocks are mapped to the same physical blocks, the lifetime of

the device is significantly reduced, independently of its size.

5.5 Lifetime with Wear Leveling

Wear leveling extends the lifetime of NAND Flash devices because it ensures that even if an

application writes to the same virtual blocks over and over again, the PROGRAM/ ERASEcycles will be distributed evenly over the NAND Flash memory.

For example, the expected lifetime of a 64MB (512Mb) NAND Flash device can be calculated

as follows:

7/29/2019 NANDDDDD

21/46



Expected lifetime =64Mbyte*100Kcycles *0.7/3KBbyte/s*24*60 *60 = 18,124 days (about 49.7

years)

In this example, 0.7 is the file system overhead.

5.6 Wear Leveling Algorithms

Wear leveling is associated with a block aging table (BAT) to store information about which

blocks have been erased in a selected period of time. There are two kinds of wear leveling that

can be implemented in the FTL:

Dynamic wear leveling

Static wear leveling

Dynamic Wear Leveling

When applying the dynamic wear leveling, new data is programmed to the free blocks (among

blocks used to store user data) that have had the fewest WRITE/ERASE cycles.

Static Wear Leveling

With static wear leveling, the content of blocks storing static data (such as code) is copied to

another block so that the original block can be used for data that is changed more frequently.

Static wear leveling is triggered when the difference between the maximum and the minimum

number of WRITE/ERASE cycles per block reaches a specific threshold. With this particular

technique, the mean age of physical NAND blocks is maintained constant.

7/29/2019 NANDDDDD

22/46



6.APPLICATIONS

Consumer: Smartphones, tablets, readers, GPS systems, MP3 players, digital still cameras, digital video cameras, ultra books, USB memory sticks, SD and uSD cards, set

top boxes, digital voice recorders

Enterprise: Servers, printers, cloud computing

Industrial: Robots, vending machines, security systems, factory automation, automotiveaudio

7/29/2019 NANDDDDD

23/46



Figure: NAND applications growth

7/29/2019 NANDDDDD

24/46



7. CONCLUSON

The NAND Flash memories are suitable for using in applications

where a large amount of data has to be stored in memory, in other words, where it is desired to

use a file system for stored data. Due to specific characteristics of NAND Flash architecture,

there is need to implement the invalid block management system, ECC coding, and eventually

requested file system to the target device. In most cases an embedded memory must contain data

in requested format before assembling in production (e.g. Boot data, pre-formated file system)

with selected invalid block management method and ECC coding. These data has to be

programmed into NAND Flash memory using the device programmer.

7/29/2019 NANDDDDD

25/46



8. REFERENCE

7/29/2019 NANDDDDD

26/46



e weather. When I draw a magnifying glass, show me the map. You might want to useother gestures that you use in everyday life. This system is very customizable.

The technology is mainly based on hand gesture recognition, image capturing, processing,

and manipulation, etc. The map application lets the user navigate a map displayed on a nearby

surface using hand gestures, similar to gestures supported by multi-touch based systems, letting

the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets

the user draw on any surface by tracking the fingertip movements of the users index finger.

1.RELATED TECHNOLOGIESSixthSense technology takes a different approach to computing and tries to make the digital

aspect of our lives more intuitive, interactive and, above all, more natural. We shouldnt have to

think about it separately. Its a lot of complex technology squeezed into a simple portable device.

When we bring in connectivity, we can get instant, relevant visual information projected on any

object we pick up or interact with the technology is mainly based on hand augmented reality,

gesture recognition, computer vision based algorithm etc.

7/29/2019 NANDDDDD

27/46



4.1 Augmented reality

Augmented reality (AR) is a term for a live direct or indirect view of a physical real-

world environment whose elements are augmentedby virtual computer-generated imagery. It is

related to a more general concept called mediated reality in which a view of reality is modified

(possibly even diminished rather than augmented) by a computer. The augmentation is

conventionally in real-time and in semantic context with environmental elements.

Sixth sense technology which uses Augmented Reality concept to super imposes digital

information on the physical world. With the help of advanced AR technology (e.g. adding

computer vision and object recognition) the information about the surrounding real world of the

user becomes interactive and digitally usable. Artificial information about the environment and

the objects in it can be stored and retrieved as an information layer on top of the real world view.

The main hardware components for augmented reality are: display, tracking, input devices,

and computer. Combination of powerful CPU, camera, accelerometers, GPS and solid state

compass are often present in modern Smartphone, which make them prospective platforms.

There are three major display techniques for Augmented Reality:

Head Mounted Displays Handheld Displays Spatial Displays

Head Mounted DisplaysA Head Mounted Display (HMD) places images of both the physical world and

registered virtual graphical objects over the user's view of the world. The HMD's are either

optical see-through or video see-through in nature.

Handheld Displays

Handheld Augment Reality employs a small computing device with a display that fits in a

user's hand. All handheld AR solutions to date have employed video see-through techniques to

7/29/2019 NANDDDDD

28/46



overlay the graphical information to the physical world. Initially handheld AR employed sensors

such as digital compasses and GPS units for its six degree of freedom tracking sensors.

Spatial Displays

Instead of the user wearing or carrying the display such as with head mounted displays or

handheld devices; Spatial Augmented Reality (SAR) makes use of digital projectors to display

graphical information onto physical objects.

Modern mobile augmented reality systems use one or more of the following tracking

technologies: digital cameras and/or other optical sensors, RFID, wireless sensors etc. Each of

these technologies have different levels of accuracy and precision. Most important is the tracking

of the pose and position of the user's head for the augmentation of the user's view.

For users with disabilities of varying kinds, AR has real potential to help people with a

variety of disabilities. Only some of the current and future AR applications make use of a

Smartphone as a mobile computing platform.

4.2 Gesture Recognition

Gesture recognition is a topic in computer science and language technology with the

goal of interpreting human gestures via mathematical algorithms. Gestures can originate from

any bodily motion or state but commonly originate from the face or hand. Current focuses in the

field include emotion recognition from the face and hand gesture recognition. Many approaches

have been made using cameras and computer vision algorithms to interpret sign language.

7/29/2019 NANDDDDD

29/46



Gestures can exist in isolation or involve external objects. Free of any object, we wave,

beckon, fend off, and to a greater or lesser degree (depending on training) make use of more

formal sign languages. With respect to objects, we have a broad range of gestures that are almost

universal, including pointing at objects, touching or moving objects, changing object shape,

activating objects such as controls, or handing objects to others.

Gesture recognition can be seen as a way for computers to begin to understand human

body language, thus building a richer bridge between machines and humans than primitive text

user interfaces or even GUIs (graphical user interfaces), which still limit the majority of input to

keyboard and mouse. Gesture recognition enables humans to interface with the machine (HMI)

and interact naturally without any mechanical devices.

Gestures can be used to communicate with a computer so we will be mostly concerned

with empty handed semiotic gestures. These can further be categorized according to their

functionality.

Symbolic gesturesThese are gestures that, within each culture, have come to a single meaning. An Emblem

such as the OK gesture is one such example, however American Sign Language gestures also

fall into this category.

Deictic gesturesThese are the types of gestures most generally seen in HCI and are the gestures of pointing,

or otherwise directing the listeners attention to specific event or objects in the environment.

Iconic gestures

7/29/2019 NANDDDDD

30/46



As the name suggests, these gestures are used to convey information about the size, shape or

orientation of the object of discourse. They are the gestures made when someone says The plane

flew like this, while moving their hand through the air like the flight path of the aircraft.

Pantomimic gestures:These are the gestures typically used in showing the use of movement of some invisible tool

or object in the speakers hand. When a speaker says I turned the steering wheel hard to the

left, while mimicking the action of turning a wheel with both hands, they are making a

pantomimic gesture.

Using the concept of gesture recognition, it is possible to point a finger at the computer

screen so that the cursor will move accordingly. This could potentially make conventional input

devices such as mouse, keyboards and even touch-screens redundant. Gesture recognition can be

conducted with techniques from computer vision and image processing. The literature includes

ongoing work in the computer vision field on capturing gestures or more general human pose

and movements by cameras connected to a computer.

4.3 Computer vision based algorithm

Computer vision is the science and technology of machines that see. As a scientific

discipline, computer vision is concerned with the theory behind artificial systems that extract

information from images. The image data can take many forms, such as video sequences, views

from multiple cameras, or multi-dimensional data from a medical scanner.

Computer vision, on the other hand, studies and describes the processes implemented in

software and hardware behind artificial vision systems. The software tracks the users gestures

using computer-vision based algorithms. Computer vision is, in some ways, the inverse ofcomputer graphics. While computer graphics produces image data from 3D models, computer

vision often produces 3D models from image data. There is also a trend towards a combination

of the two disciplines, e.g., as explored in augmented reality.

7/29/2019 NANDDDDD

31/46



The fields most closely related to computer vision are image processing, image analysis

and machine vision. Image processing and image analysis tend to focus on 2D images, how to

transform one image to another. His characterization implies that image processing/analysis

neither require assumptions nor produce interpretations about the image content. Computer

vision tends to focus on the 3D scene projected onto one or several images, e.g., how to

reconstruct structure or other information about the 3D scene from one or several images.

Machine vision tends to focus on applications, mainly in manufacturing, e.g., vision based

autonomous robots and systems for vision based inspection or measurement.

The Recognition Algorithms

The computer vision system for tracking and recognizing the hand postures that control

the menus is based on a combination of multi-scale color feature detection, view based

hierarchical hand models and particle filtering. The hand postures or states are represented in

terms of hierarchies of multi-scale color image features at different scales, with qualitative inter-

relations in terms of scale, position and orientation. In each image, detection of multiscale color

features is performed. The hand postures are then simultaneously detected and tracked using

particle filtering, with an extension of layered sampling referred to as hierarchical layered

sampling. To improve the performance of the system, a prior on skin color is included in the

particle filtering.

4.4 Technologies that uses Sixth Sense as Platform

SixthSense technology takes a different approach to computing and tries to make the

digital aspect of our lives more intuitive, interactive and, above all, more natural. When you

7/29/2019 NANDDDDD

32/46



bring in connectivity, you can get instant, relevant visual information projected on any object

you pick up or interact with. So, pick up a box of cereal and your device will project whether it

suits your preferences. Some of the technologies that uses this are Radio Frequency

Identification, gesture gaming, washing machine.

4.4.1 Radio Frequency Identification

SixthSense is a platform for Radio Frequency Identification based enterprise intelligence

that combines Radio Frequency Identification events with information from other enterprise

systems and sensors to automatically make inferences about people, objects, workspaces, and

their interactions.

Radio Frequency Identification is basically an electronic tagging technology that allowsthe detection and tracking of tags and consequently the objects that they are affixed to. This

ability to do remote detection and tracking coupled with the low cost of passive tags has led to

the widespread adoption of RFID in supply chains worldwide.

Pranav Mistry, a researcher at the media lab of the Massachusetts Institute Technology,

has developed a 'sixth sense' device a gadget worn on the wrist that can function as a 'touch

screen' device for many modern applications. The gadget is capable of selecting a product either

by image recognition or radio frequency identification (RFID) tags and project information, like

an Amazon rating.

The idea of SixthSense is to use Radio Frequency Identification technology in

conjunction with a bunch of other enterprise systems such as the calendar system or online

presence that can track user activity. Here, we consider an enterprise setting of the future where

people (or rather their employee badges) and their personal objects such as books, laptops, and

mobile phones are tagged with cheap, passive RFID tags, and there is good coverage of RFID

readers in the workplace.

SixthSense incorporates algorithms that start with a mass of undifferentiated tags and

automatically infer a range of information based on an accumulation of observations. The

technology is able to automatically differentiate between people tags and object tags, learn the

7/29/2019 NANDDDDD

33/46



identities of people, infer the ownership of objects by people, learn the nature of different zones

in a workspace (e.g., private office versus conference room), and perform other such inferences.

By combining information from these diverse sources, SixthSense records all tag-level

events in a raw database. The inference algorithms consume these raw events to infer events at

the level of people, objects, and workspace zones, which are then recorded in a separate

processed database. Applications can either poll these databases by running SQL queries or set

up triggers to be notified of specific events of interest.

SixthSense infers when a user has interacted with an object, for example, when you pick

up your mobile phone. It is a platform in that its programming model makes the inferences made

automatically available to applications via a rich set of APIs. To demonstrate the capabilities of

the platform, the researchers have prototyped a few applications using these APIs, including a

misplaced object alert service, an enhanced calendar service, and rich annotation of video with

physical events.

4.4.2 Sixth Sense Washing Machine

Whirlpool AWOE 8758 White Washing Machine is a remarkable front loader that

incorporates the unparalleled Sixth Sense technology. Whirlpools 2009 range of washing

machines comes integrated with enhanced 6th sense technology that gives more optimisation of

resources and also increased saving in terms of energy, water and time.

Ideal washing machine for thorough washing that requires sixth sense to detect stubborn

stains and adjust wash impact. It is a feature packed washing ally with Sixth Sense Technology

and several customized programs to enhance the washing performance and dexterously assist

you in heavy washing loads.

The New Generation 6th Sense appliances from Whirlpool are helping to protect theenvironment and to reduce your energy bills. Whirlpool 6th Sense appliances are designed to be

intelligent and energy efficient appliances that adapt their performance to better suit your needs.

All Whirlpool appliances with intelligent 6th Sense technology work on three key principles;

Sense, Adaption and Control, to ensure that they achieve optimal performance each and every

time that they are used.

7/29/2019 NANDDDDD

34/46



Whirlpool 6th Sense washing machines can save you up to 50% less water, energy and

time during the cycle. These intelligent machines sense the size of the load and adjust and

control the cycle dependent on the load inside in order to optimise the use of water, energy and

time. Some models also contain a detergent overdosing monitor to make sure that you do not use

too much washing detergent. Tumble dryers use 6th Sense technology to minimise energy and

time wastage by monitoring the humidity inside your laundry and adjusting the drying time

accordingly.

2.APPLICATIONSThe SixthSense prototype implements several applications that demonstrate the usefulness,

7/29/2019 NANDDDDD

35/46



viability and flexibility of the system.

The SixthSense device has a huge number of applications. The following are few of the

applications of Sixth Sense Technology.

Make a call Call up a map Check the time Create multimedia reading experience Drawing application Zooming features Get product information Get book information Get flight updates Feed information on people Take pictures Check the email

5.1 Make a call

7/29/2019 NANDDDDD

36/46



Figure 5.1: Make a call

You can use the Sixth Sense to project a keypad onto your hand, and then use that virtual

keypad to make a call. Calling a number also will not be a great task with the introduction of

Sixth Sense Technology. No mobile device will be required, just type in the number with your

palm acting as the virtual keypad. The keys will come up on the fingers. The fingers of the other

hand will then be used to key in the number and call.

5.2 Call up a map

Figure 5.2: Map

The sixth sense also implements map which lets the user display the map on any physical

7/29/2019 NANDDDDD

37/46



surface and find his destination and he can use his thumbs and index fingers to navigate the map,

for example, to zoom in and out and do other controls.

5.3 Check the time

Figure 5.3: Wrist Watch

Sense all we have to do is draw a circle on our wrist with our index finger to get a virtual

watch that gives us the correct time. The computer tracks the red marker cap or piece of tape,

recognizes the gesture, and instructs the projector to flash the image of a watch onto his wrist.

5.4 Create multimedia reading experiences

7/29/2019 NANDDDDD

38/46



Figure 5.4: Video in Newspaper

The SixthSense system also augments physical objects the user is interacting with by

projecting more information about these objects projected on them. For example, a newspaper

can show live video news or dynamic information can be provided on a regular piece of paper.

Thus a piece of paper turns into a video display.

5.5 Drawing application

Figure 5.5: Drawing

The drawing application lets the user draw on any surface by tracking the fingertip

movements of the users index finger.

7/29/2019 NANDDDDD

39/46



5.6Zooming features

Figure 5.6: Zoom in and Zoom out

The user can zoom in or zoom out using intuitive hand movements.

5.7Get product information

Figure 5.7: Product information

Maes says Sixth Sense uses image recognition or marker technology to recognize

products you pick up, and then feeds you information on those products. For example, if you're

trying to shop "green" and are looking for paper towels with the least amount of bleach in them,

the system will scan the product you pick up off the shelf and give you guidance on whether this

product is a good choice for you.

7/29/2019 NANDDDDD

40/46



5.8 Get book information

Figure 5.8: Book information

Maes says Sixth Sense uses image recognition or marker technology to recognize

products you pick up, then feeds you information on books. The system can project Amazon

ratings on that book, as well as reviews and other relevant information

5.9Take pictures

Figure 5.9: Take Pictures

If we fashion our index fingers and thumbs into a square (the typical "framing" gesture),

the system will snap a photo. After taking the desired number of photos, we can project them

onto a surface, and use gestures to sort through the photos, and organize and resize them.

7/29/2019 NANDDDDD

41/46



5.10Get flight updates

Figure 5.10: Flight updates

The system will recognize your boarding pass and let you know whether your flight is on

time and if the gate has changed.

5.11 Feed information on people

Figure 5.11: Information on people

Sixth Sense also is capable of "a more controversial use. When you go out and meet

someone, projecting relevant information such as what they do, where they work, and also m it

7/29/2019 NANDDDDD

42/46



could display tags about the person floating on their shirt. It could be handy if it displayed their

facebook relationship status so that you knew not to waste your time.

7/29/2019 NANDDDDD

43/46



3.ADVANTAGES6.1 Advantages

SixthSense is a user friendly interface which integrates digital information into thephysical world and its objects, making the entire world your computer.

SixthSense does not change human habits but causes computer and other machines toadapt to human needs.

It uses hand gestures to interact with digital information. Supports multi-touch and multi-user interaction Data access directly from machine in real time It is an open source and cost effective and we can mind map the idea anywhere It is gesture-controlled wearable computing device that feeds our relevant information

and turns any surface into an interactive display.

It is portable and easy to carry as we can wear it in our neck. The device could be used by anyone without even a basic knowledge of a keyboard or

mouse.

There is no need to carry a camera anymore. If we are going for a holiday, then fromnow on wards it will be easy to capture photos by using mere fingers
http://mobilebeyond.net/mobility-portability-and-accessibility-in-a-wireless-age/http://mobilebeyond.net/mobility-portability-and-accessibility-in-a-wireless-age/

7/29/2019 NANDDDDD

44/46



7. FUTURE SCOPE

7.1 Future Enhancements

To get rid of colour markers To incorporate camera and projector inside mobile computing device. Whenever we place pendant- style wearable device on table, it should allow us to use

the table as multi touch user interface.

Applying this technology in various interest like gaming, education systems etc. To have 3D gesture tracking. To make sixth sense work as sixth sense for disabled person.

7/29/2019 NANDDDDD

45/46



8.CONCLUSIONThe key here is that Sixth Sense recognizes the objects around you, displaying

information automatically and letting you access it in any way you want, in the simplest way

possible.

Clearly, this has the potential of becoming the ultimate "transparent" user interface for

accessing information about everything around us. If they can get rid of the colored finger caps

and it ever goes beyond the initial development phase, that is. But as it is now, it may change the

way we interact with the real world and truly give everyone complete awareness of the

environment around us.

7/29/2019 NANDDDDD

46/46


9.REFERENCES1. www.blendernation.com/sixth-sense-technology/ 2. http://boingboing.net/2009/11/12/sixth-sense-technolo.html 3. http://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-

the-world-forever

4. http://theviewspaper.net/sixth-sense-technology-will-revolutionize-the-world/ 5. http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-

whats-stopping-this/
http://www.blendernation.com/sixth-sense-technology/http://www.blendernation.com/sixth-sense-technology/http://boingboing.net/2009/11/12/sixth-sense-technolo.htmlhttp://boingboing.net/2009/11/12/sixth-sense-technolo.htmlhttp://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://theviewspaper.net/sixth-sense-technology-will-revolutionize-the-world/http://theviewspaper.net/sixth-sense-technology-will-revolutionize-the-world/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://lucasrichter.wordpress.com/2009/03/13/pattie-maes-sixth-sense-technology-whats-stopping-this/http://theviewspaper.net/sixth-sense-technology-will-revolutionize-the-world/http://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-foreverhttp://boingboing.net/2009/11/12/sixth-sense-technolo.htmlhttp://www.blendernation.com/sixth-sense-technology/

nanddddd

Documents