クラウド時代の半導体メモリー技術
5 likes•2,504 views
- Hardware such as DRAM and NAND flash are facing scaling challenges as density increases, which could impact performance and cost. New non-volatile memory (NVM) technologies may provide opportunities to address these challenges but require software and system architecture changes to realize their full potential. Key considerations include persistence, performance, and programming models.
![( , 67
ts
l ~∼ p” t
k ]G p v t
k v t
l 77EpA4A7 t
k v t
l v
k z v t ~∼ ~∼
k ” ~∼ x ~∼y
l sA4A7
,](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-6-320.jpg)
![( , 67
l (,
l )
l JSP v
.
Vaa 0 b Wa OW a U] X OW Z aO
W[ bZ S W[ bZ S Va[Z](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-8-320.jpg)
![( , 67
l “ •~∼
k B S 6][ baS C ]XSQa 9OQSP]] ”
k EOQ FQOZS 6][ baW U aSZ
k 8ea S[SZf FV W W U 6][ baW U 5
k GVS OQVW S C
k 9W S5]e H65
k 6GE 6] ] aWb[ G
l
k v x p p v y
k v
k v
l – p p p ~∼–](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-11-320.jpg)
![( , 67
gin a
k
(
k Z]OR a] S
k v v
)
k v
)
NVM)SSD)Challenges)
• So:ware)overheads)in)
kernel)
7)
0
5
10
15
20
25
BaseLatency(us)
PCM
Ring
DMA
Wait
Interrupt
Issue
Copy
Schedule
OS/User
Software is Critical
• Baseline Latencies:
– Hardware: 8.2 us
– Software: 13.4 us
Hardware costs
11[Caulfield,)SC’10])
M6ObZjSZR F6 N
v](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-13-320.jpg)
![( , 67
-‐‑‒ -‐‑‒
An application using NVM.FILE mode may or may not be using memory-mapped file
behavior.
The NVM.FILE mode describes NVM extensions including:
• Discovery and use of atomic write features
• The discovery of granularities (length or alignment characteristics)
4.3.3 NVM.PM.VOLUME mode overview
NVM.PM.VOLUME mode describes the behavior for operating system components (
file systems) accessing persistent memory. NVM.PM.VOLUME mode provides a soft
abstraction for Persistent Memory hardware and profiles functionality for operating sy
components including:
• the list of physical address ranges associated with each PM volume
• the capability to determine whether PM errors have been reported
Figure 5 NVM.PM.VOLUME and NVM.PM.FILE mode examples
Application
PM device PM device PM device. . .
User space
Kernel space
MMU
MappingsPM-aware file system
NVM PM capable driver
Load/
store
Native file
API
PM-aware kernel module
PM device
NVM.PM.VOLUME mode
NVM.PM.FILE mode
4.3.4 NVM.PM.FILE mode overview
NVM.PM.FILE mode describes the behavior for applications accessing persistent me
The commands implementing NVM.PM.FILE mode are similar to those using NVM.F
l AI 5Z]Q []RS
l AI 9WZS []RS
l CS W aS a S[] f I]Zb[S []RS
l CS W aS a S[] f 9WZS []RS
Note that there are other models for connecting a non-PM file system to PM hardware.
4.3 NVM programming modes
4.3.1 NVM.BLOCK mode overview
NVM.BLOCK and NVM.FILE modes are used when NVM devices provide block storage
behavior to software (in other words, emulation of hard disks). The NVM may be exposed as a
single or as multiple NVM volumes. Each NVM volume supporting these modes provides a
range of logically-contiguous blocks. NVM.BLOCK mode is used by operating system
components (for example, file systems) and by applications that are aware of block storage
characteristics and the block addresses of application data.
This specification does not document existing block storage software behavior; the
NVM.BLOCK mode describes NVM extensions including:
• Discovery and use of atomic write and discard features
• The discovery of granularities (length or alignment characteristics)
• Discovery and use of ability for applications or operating system components to mark
blocks as unreadable
Figure 4 NVM.BLOCK and NVM.FILE mode examples
Application
NVM block capable driver
File system
Application
NVM device NVM device
User space
Kernel space
Native file
API
NVM.BLOCK mode
NVM.FILE mode
4.3.2 NVM.FILE mode overview
NVM.FILE mode is used by applications that are not aware of details of block storage
hardware or addresses. Existing applications written using native file I/O behavior should work
FA 4 C ]U O[[W U ]RSZ I]Z](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-14-320.jpg)
![( , 67
l v CS W aS a S[] f C
t
l – 4C p
C fi p 4C
k [OZZ]Q – fSa O ]aVS [OZZ]Q
l v v t
k
k v](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-15-320.jpg)
![( , 67
Lcndl
,
6CH
AIE4 7E4
6CH
AIE4
7E4
6CH
AIE4
4 ES ZOQS RW 5 FVO SR ORR S OQS 6 8 aW SZf AIE4
5Z]Q 9
S[] f 9
QOQVS QOQVSQOQVS
AIE4 B AIE4](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-16-320.jpg)
![( , 67
c mg M N
l AIE4
k Fa] S • AIE4 ~∼
k r r v
-‐‑‒
• Recovery depends on write ordering
CPU
Write-back
Cache
NVM
V D
VD
STORE data[0] = 0xFOOD
STORE data[1] = 0xBEEF
STORE valid = 1
Crash
D
D
CPU
Persistent Memory (PM) Ordering
M FdWTa 4FC BF( N](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-17-320.jpg)
![( , 67
c mg MRN
l
k ”“ t
k aSZ CS W aS a S[] f v
l 6 9 HF BCG
l 6 J5 C6B G
l ” t
.
dering with Existing Hardware
er writes by flushing cachelines via CLFLUSH
CLFLUSH:
talls the CPU pipeline and serializes execution
STORE data[0] = 0xFOOD
STORE data[1] = 0xBEEF
CLFLUSH data[0]
CLFLUSH data[1]
STORE valid = 1
ata[0] ST CLFLUSH
CLFLUSHOPT
• Provides unordered version of CLFLUSH
• Supports efficient cache flushing
data[1]
valid
ST CLFLUSHOPT
ST
data[0] ST CLFLUSHOPT
time
data[1]
valid
ST CLFLUSH
ST
data[0] ST CLFLUSH
time 20
M FdWTa 4FC BF( N
aOZZ
h(](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-18-320.jpg)
![( , 67
X
/
F]b QS0 GEF ( )
l u x p p p
r y
l – ~∼
k y •](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-19-320.jpg)
![( , 67
S T
a
l )
k s
x yo o o o
k p – fi
l • p
l ff ~∼
k Q T Gb P] 5]] a
(
•~∼
p –
v](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-21-320.jpg)
![( , 67
l C OdO S 9WZS Ff aS[
k C 9F M8b ]Ff N F6 9F MF6 N
5C9F MFBFC /N
l C WP O f
k AI SO M4FC BF N 677F MHF8A K N
S[] f S M4FC BF N 4aZO MBBCF 4 N
l 7OaOPO S
k 9B87HF MF : B7 N
l a b[S aOaW]
k AI F64I8A:8E M C7CF (N
((](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-22-320.jpg)
![( , 67
l 4FC BF ( Gba] WOZ
k C ]U O[[W U O R H OUS ]RSZ T] A] I]ZOaWZS
S[] f
k Vaa 0 S SO QV Q dW Q SRb ] O aba] WOZ
l 6EB ( Gba] WOZ
k 7OaOQS aS FW[bZOaW] SaV]R]Z]UWS
k Vaa 0 S] ZS Rb S SRb hPQZ aba] WOZ R [
()](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-23-320.jpg)
![( , 67
C
l W be
k 7E4 –~∼
k
–~∼
k K C SKSQbaW] CZOQS
l 74K 7W SQa 4QQS
k AIE4 v
–~∼
k K C
(
Linux – PDA
Agenda VR3 (2001)](https://image.slidesharecdn.com/20150416-150416055348-conversion-gate01/85/-24-320.jpg)
クラウド時代の半導体メモリー技術
- 1. ( , 67 ts Odʼ’a L P ( , 67
- 2. ( , 67 l s x– ~∼y l s r v v l s v p l k v x y o v k C6 o ~∼ fi ~∼ v v v l k –BFp v (
- 3. ( , 67 -‐‑‒ -‐‑‒ ʼ’mg i L e t ) DRAM(and(Flash(Scaling:(( “The(End(is(Nigh”( 1985( 1990( 1995( 2000( 2005( 2010( 2015( 2020( Density, DRAM( SLC(NAND( M FdWTa 4FC BF( N
- 4. ( , 67 Summary( Register(( Cache( STT&RAM,, NV&DIMM, ns=class(NVM( RRAM,,PCM,, Low(us=class(NVM( NAND,SSD, High(us=class(NVM( HDD( Low(ms=class(Mass(Storage( Capacity( 10( ” v k so ~∼ k s –~∼ M FdWTa 4FC BF( N
- 5. ( , 67 whg Ld l v p – fi t l ” ~∼“ fi k p p p r k l l p ~∼ k p
- 6. ( , 67 ts l ~∼ p” t k ]G p v t k v t l 77EpA4A7 t k v t l v k z v t ~∼ ~∼ k ” ~∼ x ~∼y l sA4A7 ,
- 7. ( , 67 v x CH F8y V X oL oL -‐‑‒ 2015 2030 vv v hmd fl v v fi Lcndl v rr r rr r
- 8. ( , 67 l (, l ) l JSP v . Vaa 0 b Wa OW a U] X OW Z aO W[ bZ S W[ bZ S Va[Z
- 10. ( , 67 : : I The$Machine$could$be$six$%mes$more$powerful$than$an$ equivalent$conven2onal$design,$while$using$just$1.25$ percent$of$the$energy$and$being$around$1/100$the$size. h:p://www.hpl.hp.com/research/systems@research/themachine/
- 11. ( , 67 l “ •~∼ k B S 6][ baS C ]XSQa 9OQSP]] ” k EOQ FQOZS 6][ baW U aSZ k 8ea S[SZf FV W W U 6][ baW U 5 k GVS OQVW S C k 9W S5]e H65 k 6GE 6] ] aWb[ G l k v x p p v y k v k v l – p p p ~∼–
- 12. ( , 67 ( IoTpCPSpHPCp p v • v v v • v v • SoC (eFlash?) • v v • v v v v • v • DRAM
- 13. ( , 67 gin a k ( k Z]OR a] S k v v ) k v ) NVM)SSD)Challenges) • So:ware)overheads)in) kernel) 7) 0 5 10 15 20 25 BaseLatency(us) PCM Ring DMA Wait Interrupt Issue Copy Schedule OS/User Software is Critical • Baseline Latencies: – Hardware: 8.2 us – Software: 13.4 us Hardware costs 11[Caulfield,)SC’10]) M6ObZjSZR F6 N v
- 14. ( , 67 -‐‑‒ -‐‑‒ An application using NVM.FILE mode may or may not be using memory-mapped file behavior. The NVM.FILE mode describes NVM extensions including: • Discovery and use of atomic write features • The discovery of granularities (length or alignment characteristics) 4.3.3 NVM.PM.VOLUME mode overview NVM.PM.VOLUME mode describes the behavior for operating system components ( file systems) accessing persistent memory. NVM.PM.VOLUME mode provides a soft abstraction for Persistent Memory hardware and profiles functionality for operating sy components including: • the list of physical address ranges associated with each PM volume • the capability to determine whether PM errors have been reported Figure 5 NVM.PM.VOLUME and NVM.PM.FILE mode examples Application PM device PM device PM device. . . User space Kernel space MMU MappingsPM-aware file system NVM PM capable driver Load/ store Native file API PM-aware kernel module PM device NVM.PM.VOLUME mode NVM.PM.FILE mode 4.3.4 NVM.PM.FILE mode overview NVM.PM.FILE mode describes the behavior for applications accessing persistent me The commands implementing NVM.PM.FILE mode are similar to those using NVM.F l AI 5Z]Q []RS l AI 9WZS []RS l CS W aS a S[] f I]Zb[S []RS l CS W aS a S[] f 9WZS []RS Note that there are other models for connecting a non-PM file system to PM hardware. 4.3 NVM programming modes 4.3.1 NVM.BLOCK mode overview NVM.BLOCK and NVM.FILE modes are used when NVM devices provide block storage behavior to software (in other words, emulation of hard disks). The NVM may be exposed as a single or as multiple NVM volumes. Each NVM volume supporting these modes provides a range of logically-contiguous blocks. NVM.BLOCK mode is used by operating system components (for example, file systems) and by applications that are aware of block storage characteristics and the block addresses of application data. This specification does not document existing block storage software behavior; the NVM.BLOCK mode describes NVM extensions including: • Discovery and use of atomic write and discard features • The discovery of granularities (length or alignment characteristics) • Discovery and use of ability for applications or operating system components to mark blocks as unreadable Figure 4 NVM.BLOCK and NVM.FILE mode examples Application NVM block capable driver File system Application NVM device NVM device User space Kernel space Native file API NVM.BLOCK mode NVM.FILE mode 4.3.2 NVM.FILE mode overview NVM.FILE mode is used by applications that are not aware of details of block storage hardware or addresses. Existing applications written using native file I/O behavior should work FA 4 C ]U O[[W U ]RSZ I]Z
- 15. ( , 67 l v CS W aS a S[] f C t l – 4C p C fi p 4C k [OZZ]Q – fSa O ]aVS [OZZ]Q l v v t k k v
- 16. ( , 67 Lcndl , 6CH AIE4 7E4 6CH AIE4 7E4 6CH AIE4 4 ES ZOQS RW 5 FVO SR ORR S OQS 6 8 aW SZf AIE4 5Z]Q 9 S[] f 9 QOQVS QOQVSQOQVS AIE4 B AIE4
- 17. ( , 67 c mg M N l AIE4 k Fa] S • AIE4 ~∼ k r r v -‐‑‒ • Recovery depends on write ordering CPU Write-back Cache NVM V D VD STORE data[0] = 0xFOOD STORE data[1] = 0xBEEF STORE valid = 1 Crash D D CPU Persistent Memory (PM) Ordering M FdWTa 4FC BF( N
- 18. ( , 67 c mg MRN l k ”“ t k aSZ CS W aS a S[] f v l 6 9 HF BCG l 6 J5 C6B G l ” t . dering with Existing Hardware er writes by flushing cachelines via CLFLUSH CLFLUSH: talls the CPU pipeline and serializes execution STORE data[0] = 0xFOOD STORE data[1] = 0xBEEF CLFLUSH data[0] CLFLUSH data[1] STORE valid = 1 ata[0] ST CLFLUSH CLFLUSHOPT • Provides unordered version of CLFLUSH • Supports efficient cache flushing data[1] valid ST CLFLUSHOPT ST data[0] ST CLFLUSHOPT time data[1] valid ST CLFLUSH ST data[0] ST CLFLUSH time 20 M FdWTa 4FC BF( N aOZZ h(
- 19. ( , 67 X / F]b QS0 GEF ( ) l u x p p p r y l – ~∼ k y •
- 20. ( , 67 X l AI T WS RZf OQQS OaaS l p k J WaS SOR ~∼– v • l v k BF k v ( MRAMDRAM VM $ VM $ $ ff v VMVMVM
- 21. ( , 67 S T a l ) k s x yo o o o k p – fi l • p l ff ~∼ k Q T Gb P] 5]] a ( •~∼ p – v
- 22. ( , 67 l C OdO S 9WZS Ff aS[ k C 9F M8b ]Ff N F6 9F MF6 N 5C9F MFBFC /N l C WP O f k AI SO M4FC BF N 677F MHF8A K N S[] f S M4FC BF N 4aZO MBBCF 4 N l 7OaOPO S k 9B87HF MF : B7 N l a b[S aOaW] k AI F64I8A:8E M C7CF (N ((
- 23. ( , 67 l 4FC BF ( Gba] WOZ k C ]U O[[W U O R H OUS ]RSZ T] A] I]ZOaWZS S[] f k Vaa 0 S SO QV Q dW Q SRb ] O aba] WOZ l 6EB ( Gba] WOZ k 7OaOQS aS FW[bZOaW] SaV]R]Z]UWS k Vaa 0 S] ZS Rb S SRb hPQZ aba] WOZ R [ ()
- 24. ( , 67 C l W be k 7E4 –~∼ k –~∼ k K C SKSQbaW] CZOQS l 74K 7W SQa 4QQS k AIE4 v –~∼ k K C ( Linux – PDA Agenda VR3 (2001)