SlideShare a Scribd company logo

Linux Kernel - Virtual File System

Adrian Huang
Adrian Huang

The document provides an in-depth examination of the virtual file system (VFS) in the Linux kernel, focusing on its inner workings in relation to the ext4 file system format. It covers key components such as directory entries (dentry), inodes, and the anatomy of system calls like 'mount' and 'ls', explaining object relationships within VFS. Various data structures and their roles in file operations, memory management, and file system interactions are also analyzed.

Software
1 of 33
Downloaded 59 times
1
2
3
4
5
6
7
8
9
10
Most read
11
Most read
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Most read
32
33
Virtual File System (VFS) in Linux Kernel Adrian Huang | Nov, 2021 * Based on kernel 5.11 (x86_64) – QEMU * SMP (4 CPUs) and 8GB memory * Kernel parameter: nokaslr norandmaps * Userspace: ASLR is disabled * Legacy BIOS
Agenda • Kernel components affected by an IO request • Object relationship in VFS • Ext4 file system format • dentry (Directory Entry) and inode (Index Node) • Example: ext4 • Anatomy of `mount` system call • Anatomy of `ls` system call
Kernel components affected by an IO request VFS Page Cache/Buffer Cache (fs/buffer.c) Disk filesystem Disk filesystem Block Device File Mapping Layer Generic Block Layer I/O Scheduler Layer Block Device Driver Disk Disk Disk submit_bio()
open(“/mnt/a.txt”, …) open(“/mnt/b.txt”, …) Process A File Object File Object File Object open file table dentry dentry dentry cache inode inode inode cache inode table Kernel - VFS dentry in data blocks Disk Object relationship in VFS dentry cache list • Linked via hlist_bl_{head,node} (hash table with bit lock) inode cache list • Linked via hlist_{head,node} (hash table) Note
Super Block Group Descriptors Reserved GDT blocks Block bitmap inode bitmap inode table: inode descriptor Data Block Data Block . . . 0 1 Total inode count Total block count Free blocks Free inodes … Block Group 0 . . . Block Group N Boot Sector Partition mode Owner UID, Owner GID i_atime (last access time) i_ctime (last inode change time) i_mtime (last data modification time) size ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent_header eh_magic = 0xF30A eh_entries eh_max eh_depth eh_generation ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent ee_block ee_len ei_start_hi ei_start_lo ext4_extent ee_block ee_len ei_start_hi ei_start_lo . . . . . . root index node leaf node Disk blocks Extent Tree Ext4 file system format Storage size addressed by a block group: 4096 (bytes) * 8 * 4096 (block size) = 128MB
Super Block Group Descriptors Reserved GDT blocks Block bitmap inode bitmap inode table: inode descriptor Data Block Data Block . . . 0 1 Total inode count Total block count Free blocks Free inodes … Block Group 0 . . . Block Group N Boot Sector Partition mode Owner UID, Owner GID i_atime (last access time) i_ctime (last inode change time) i_mtime (last data modification time) size ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent_header eh_magic = 0xF30A eh_entries eh_max eh_depth eh_generation ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent ee_block ee_len ei_start_hi ei_start_lo ext4_extent ee_block ee_len ei_start_hi ei_start_lo . . . . . . root index node leaf node Disk blocks Extent Tree Ext4 file system format
inode# Purpose 0 Doesn't exist; there is no inode 0 1 List of defective blocks 2 Root directory 3 User quota 4 Group quota 5 Boot loader 6 Undelete directory 7 Reserved group descriptors inode. ("resize inode") 8 Journal inode 9 The "exclude" inode 10 Replica inode 11 First non-reserved inode. Usually this is the lost+found directory. See s_first_ino in the superblock. Relationship between inode and directory entry (dentry) inode number link count
inode bitmap: 19 * 4096 = 77824 (0x13000) inode 1-16 inode 17 `dumpe2fs 16MiB.img` Relationship between inode and directory entry (dentry)
inode entry: 35 * 4096 + 128 * (2 – 1) [inode #2] = 0x23080 `dumpe2fs 16MiB.img` inode structure data of inode #2 Relationship between inode and directory entry (dentry)
mode Owner UID, Owner GID i_atime i_ctime i_mtime size ext4_extent_header eh_magic = 0xF30A eh_entries = 1 eh_max = 4 eh_depth = 0 eh_generation = 0 root Disk blocks ext4_extent ee_block = 0 ee_len = 0x0001 ei_start_hi = 0x0000 ei_start_lo = 0x00000004 Relationship between inode and dentry – root (inode #2) ext4_inode (inode #2) leaf node . 2 name inode Directory Entry (dentry) Block #4: A list of struct ext4_dir_entry_2 .. 2 -> 2646185 lost+found 11 a123 12 a789 17 Block #0-3 Block #5 inode structure data of inode #2 Root Directory dentry in data block of inode #2
type = directory i_links_count = 5 Relationship between inode and dentry – ‘b456’ (inode #13) i-node table . 2 name inode Directory Entry (dentry) - / .. 2 -> 2646185 lost+found 11 a123 12 a789 17 2 12 . 12 name inode .. 2 b456 13 hello.c 15 hello-hard-link.c 15 Directory Entry (dentry) - /a123 13 . 13 name inode .. 12 test.c 16 test-hard-link.c 16 test-soft-link.c 14 Directory Entry (dentry) - /a123/b456 14 16 Data block pointers type = directory i_links_count = 3 Data block pointers type = directory i_links_count = 2 Data block pointers type = symbolic link i_links_count = 1 Data block pointers = “test.c” type = file i_links_count = 2 Data block pointers File Data /a123/b456/test.c size = 6 size = 100 inode #
type = directory i_links_count = 5 Relationship between inode and dentry – ‘test.c’ (inode #16) i-node table . 2 name inode Directory Entry (dentry) - / .. 2 -> 2646185 lost+found 11 a123 12 a789 17 2 12 . 12 name inode .. 2 b456 13 hello.c 15 hello-hard-link.c 15 Directory Entry (dentry) - /a123 13 . 13 name inode .. 12 test.c 16 test-hard-link.c 16 test-soft-link.c 14 Directory Entry (dentry) - /a123/b456 14 16 Data block pointers type = directory i_links_count = 3 Data block pointers type = directory i_links_count = 2 Data block pointers type = symbolic link i_links_count = 1 Data block pointers = “test.c” type = file i_links_count = 2 Data block pointers File Data /a123/b456/test.c size = 6 size = 100 inode structure data of inode #16 inode entry: 35 * 4096 + 128 * (16 – 1) [inode #16] = 0x23780 address inode structure of inode #16 Data block pointer represented by struct ext4_extent i_links_count inode #
type = directory i_links_count = 5 Relationship between inode and dentry – ‘test-soft-link.c’ (inode #14) i-node table . 2 name inode Directory Entry (dentry) - / .. 2 -> 2646185 lost+found 11 a123 12 a789 17 2 12 . 12 name inode .. 2 b456 13 hello.c 15 hello-hard-link.c 15 Directory Entry (dentry) - /a123 13 . 13 name inode .. 12 test.c 16 test-hard-link.c 16 test-soft-link.c 14 Directory Entry (dentry) - /a123/b456 14 16 Data block pointers type = directory i_links_count = 3 Data block pointers type = directory i_links_count = 2 Data block pointers type = symbolic link i_links_count = 1 Data block pointers = “test.c” type = file i_links_count = 2 Data block pointers File Data /a123/b456/test.c size = 6 size = 100 inode structure data of inode #14 inode entry: 35 * 4096 + 128 * (14 – 1) [inode #14] = 0x23680 address inode structure of inode #14 i_links_count Data block pointer Hard Link 1. Refer to an inode number 2. Must reside on the same file system 3. Cannot be made to a directory Soft Link 1. Refer to a filename instead of an inode number 2. Can be linked to a file in a different file system 3. Can be made to a directory inode #
mount – mount a new ext4 file system • stat() – get file status from inode o Name resolution → get dentry → get inode • openat() o Name resolution o Update open file table • mount() o Name resolution o Add a mount entry to ‘mount’ hash table o Parse file system
stat("/adrian/img/src.img“, …) user_path_at(): fill a path struct (dentry and mount info) after resolving a pathname vfsmount mnt_root mnt_sb dentry d_name path mnt dentry d_inode path struct • stat() – get file status from inode o Name resolution → get dentry → get inode
stat("/adrian/img/src.img“, …) 1. namei: convert a “name” to an “inode” 2. nameidata: store the current status when walking a path nameidata last name root inode path task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable
stat("/adrian/img/src.img“, …) path_lookupat(): 1. Configure members of nameidata struct based on absolute pathname or relative pathname 2. Name resolution vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode * Data storage (not a data pointer) Note
link_path_walk(): name resolution – round 1 vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” path mnt dentry current->fs->root d_inode qstr name = “adrian/img/src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “adrian/img/src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “img/src.img” hash 1 2 3 4 5 6 Note local variable link_path_walk() • stop until the last dentry is resolved • for-loop in this function qstr name = “adrian” u32 hash u32 len = 6 u64 hash_len union
link_path_walk(): name resolution – round 1 nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” path mnt dentry current->fs->root qstr name = “adrian/img/src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “adrian/img/src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “img/src.img” hash 1 2 3 4 5 6 Note local variable lookup_slow()
vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “img/src.img” u32 hash u32 len = 3 u64 hash_len union variable ‘name’ = “img/src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “src.img” hash 1 2 3 4 5 6 Note link_path_walk(): name resolution – round 2 qstr name = “img” u32 hash u32 len = 3 u64 hash_len union link_path_walk() • stop until the last dentry is resolved • for-loop in this function
vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “src.img” u32 hash u32 len = 3 u64 hash_len union variable ‘name’ = “src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “” 2 Note Function return link_path_walk(): name resolution – round 3 1 hash qstr name = “img” u32 hash u32 len = 3 u64 hash_len union link_path_walk() • stop until the last dentry is resolved • for-loop in this function
vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “src.img” dentry d_name qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union d_inode path mnt dentry variable ‘name’ = “” 3 4 5 6 Function return lookup_last(): name resolution hash 1 2 * Data storage (not a data pointer) Note lookup_last() • get the corresponding dentry based on the filename • “while loop”: Follow the symlink in the final component
vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “src.img” dentry d_name qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union d_inode path mnt dentry variable ‘name’ = “” 3 4 5 6 Function return lookup_last(): name resolution hash 1 2 * Data storage (not a data pointer) Note
mount – mount a new ext4 file system
hlist_head.first hlist_head.first . . mount_hashtable mount mnt_parent struct dentry *mnt_mountpoint struct vfsmount mnt **pprev *next const char *mnt_devname = “none” mount mnt_parent struct dentry *mnt_mountpoint **pprev struct hlist_node mnt_hash *next const char *mnt_devname = “none” struct hlist_node mnt_hash dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_sops s_root s_bdev s_id[32] = “rootfs” s_fs_info s_type = rootfs_fs_type d_parent mnt_root mnt_sb struct vfsmount mnt mount – before mounting a new ext4 file system d_sb
hlist_head.first hlist_head.first . . mount_hashtable mount mnt_parent struct dentry *mnt_mountpoint struct vfsmount mnt **pprev *next const char *mnt_devname = “none” mount mnt_parent struct dentry *mnt_mountpoint **pprev struct hlist_node mnt_hash *next const char *mnt_devname = “none” struct hlist_node mnt_hash dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_sops s_root s_bdev s_id[32] = “rootfs” s_fs_info s_type = rootfs_fs_type d_parent mnt_root mnt_sb struct vfsmount mnt mount – after mounting a new ext4 file system mount mnt_parent struct dentry *mnt_mountpoint **pprev *next const char *mnt_devname = “/dev/loop0” mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union struct vfsmount mnt super_block s_op = ext4_sops s_id[32] = “loop0” s_type = ext4_fs_type d_parent struct hlist_node mnt_hash dentry d_name d_parent qstr name = “mnt” u32 hash u32 len = 3 u64 hash_len union /adrian/mnt d_sb d_sb
hlist_head.first hlist_head.first . . mount_hashtable mount mnt_parent struct dentry *mnt_mountpoint struct vfsmount mnt **pprev *next const char *mnt_devname = “none” mount mnt_parent struct dentry *mnt_mountpoint **pprev struct hlist_node mnt_hash *next const char *mnt_devname = “none” struct hlist_node mnt_hash dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_sops s_root s_bdev s_id[32] = “rootfs” s_fs_info s_type = rootfs_fs_type d_parent mnt_root mnt_sb struct vfsmount mnt mount – after mounting a new ext4 file system mount mnt_parent struct dentry *mnt_mountpoint **pprev *next const char *mnt_devname = “/dev/loop0” mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union struct vfsmount mnt super_block s_op = ext4_sops s_id[32] = “loop0” s_type = ext4_fs_type d_parent struct hlist_node mnt_hash dentry d_name d_parent qstr name = “mnt” u32 hash u32 len = 3 u64 hash_len union /adrian/mnt d_sb d_sb
`ls –color mnt`
`ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …)
`ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …)
`ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …)
path mnt dentry vfsmount mnt_root mnt_sb qstr name = “mnt” u32 hash u32 len = 3 u64 hash_len union dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_ops s_root s_bdev s_id[32] = “rootfs” s_fs_info d_inode dentry d_name d_inode d_parent s_type = rootfs_fs_type path mnt dentry vfsmount mnt_root mnt_sb qstr name = “mnt /” u32 hash u32 len = 3 1 u64 hash_len union dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ext4_sops s_root s_bdev s_id[32] = “loop0” s_fs_info d_inode dentry d_name d_inode d_parent s_type = ext4_fs_type /adrian/mnt data structures in the original file system /adrian/mnt data structures in newly mounted file system traverse_mounts() `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …) The original mount point is changed the newly one when running name resolution
path mnt dentry vfsmount mnt_root mnt_sb qstr name = “mnt /” u32 hash u32 len = 3 1 u64 hash_len union dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ext4_sops s_root s_bdev s_id[32] = “loop0” s_fs_info d_inode dentry d_name d_inode d_parent s_type = ext4_fs_type /adrian/mnt data structures in newly mounted file system flags: path->dentry->d_flags `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …) REF-walk [Two concurrent mechanisms for name resolution] 1. REF-walk: concurrent management with refcounts and spinlock 2. RCU-walk: faster name resolution lookup

More Related Content

What's hot (20)

PDF
Memory Mapping Implementation (mmap) in Linux Kernel
Adrian Huang
 
PPTX
Linux Initialization Process (1)
shimosawa
 
PDF
Memory Management with Page Folios
Adrian Huang
 
PPTX
Linux Initialization Process (2)
shimosawa
 
PDF
Linux kernel debugging
libfetion
 
PPTX
Linux Kernel Booting Process (1) - For NLKB
shimosawa
 
PPTX
U-Boot presentation 2013
Wave Digitech
 
PDF
Lesson 2 Understanding Linux File System
Sadia Bashir
 
PPT
U boot porting guide for SoC
Macpaul Lin
 
PDF
High Performance Storage Devices in the Linux Kernel
Kernel TLV
 
PDF
Arm device tree and linux device drivers
Houcheng Lin
 
PDF
semaphore & mutex.pdf
Adrian Huang
 
PDF
Embedded Android : System Development - Part II (Linux device drivers)
Emertxe Information Technologies Pvt Ltd
 
PPTX
Linux device drivers
Abhishek Sagar
 
PPTX
Linux MMAP & Ioremap introduction
Gene Chang
 
PPTX
Linux Kernel Booting Process (2) - For NLKB
shimosawa
 
PDF
Linux device drivers
Emertxe Information Technologies Pvt Ltd
 
PDF
Kernel_Crash_Dump_Analysis
Buland Singh
 
PPTX
qemu + gdb + sample_code: Run sample code in QEMU OS and observe Linux Kernel...
Adrian Huang
 
PPTX
Linux Kernel MMC Storage driver Overview
RajKumar Rampelli
 
Memory Mapping Implementation (mmap) in Linux Kernel
Adrian Huang
 
Linux Initialization Process (1)
shimosawa
 
Memory Management with Page Folios
Adrian Huang
 
Linux Initialization Process (2)
shimosawa
 
Linux kernel debugging
libfetion
 
Linux Kernel Booting Process (1) - For NLKB
shimosawa
 
U-Boot presentation 2013
Wave Digitech
 
Lesson 2 Understanding Linux File System
Sadia Bashir
 
U boot porting guide for SoC
Macpaul Lin
 
High Performance Storage Devices in the Linux Kernel
Kernel TLV
 
Arm device tree and linux device drivers
Houcheng Lin
 
semaphore & mutex.pdf
Adrian Huang
 
Embedded Android : System Development - Part II (Linux device drivers)
Emertxe Information Technologies Pvt Ltd
 
Linux device drivers
Abhishek Sagar
 
Linux MMAP & Ioremap introduction
Gene Chang
 
Linux Kernel Booting Process (2) - For NLKB
shimosawa
 
Linux device drivers
Emertxe Information Technologies Pvt Ltd
 
Kernel_Crash_Dump_Analysis
Buland Singh
 
qemu + gdb + sample_code: Run sample code in QEMU OS and observe Linux Kernel...
Adrian Huang
 
Linux Kernel MMC Storage driver Overview
RajKumar Rampelli
 

Similar to Linux Kernel - Virtual File System (20)

PPT
Vfs
Waqas !!!!
 
PPTX
UNIT III.pptx
YogapriyaJ1
 
PDF
TLPI Chapter 14 File Systems
Shu-Yu Fu
 
PPT
Linux
keydak11
 
PPTX
Ext filesystem4
Neha Kulkarni
 
PDF
009709863.pdf
KalsoomTahir2
 
ODP
4. linux file systems
Marian Marinov
 
PPT
Unix file systems 2 in unix internal systems
senthilamul
 
PPTX
Examining Linux File Structures
primeteacher32
 
PDF
Ch11 file system implementation
Welly Dian Astika
 
PDF
Part 03 File System Implementation in Linux
Tushar B Kute
 
PDF
Course 102: Lecture 27: FileSystems in Linux (Part 2)
Ahmed El-Arabawy
 
PPTX
Ubuntu File System
Bhagyesh Patel
 
PPTX
Files and directories in Linux 6
Meenakshi Paul
 
PDF
Linux fundamental - Chap 10 fs
Kenny (netman)
 
PPTX
file system overview in oerating system .
pavanm6360
 
PDF
Operating Systems - Implementing File Systems
Mukesh Chinta
 
PDF
1 04 rao
dimitar9
 
PDF
File systems for Embedded Linux
Emertxe Information Technologies Pvt Ltd
 
PPTX
Raspberry pi Part 5
Techvilla
 
Vfs
Waqas !!!!
 
UNIT III.pptx
YogapriyaJ1
 
TLPI Chapter 14 File Systems
Shu-Yu Fu
 
Linux
keydak11
 
Ext filesystem4
Neha Kulkarni
 
009709863.pdf
KalsoomTahir2
 
4. linux file systems
Marian Marinov
 
Unix file systems 2 in unix internal systems
senthilamul
 
Examining Linux File Structures
primeteacher32
 
Ch11 file system implementation
Welly Dian Astika
 
Part 03 File System Implementation in Linux
Tushar B Kute
 
Course 102: Lecture 27: FileSystems in Linux (Part 2)
Ahmed El-Arabawy
 
Ubuntu File System
Bhagyesh Patel
 
Files and directories in Linux 6
Meenakshi Paul
 
Linux fundamental - Chap 10 fs
Kenny (netman)
 
file system overview in oerating system .
pavanm6360
 
Operating Systems - Implementing File Systems
Mukesh Chinta
 
1 04 rao
dimitar9
 
File systems for Embedded Linux
Emertxe Information Technologies Pvt Ltd
 
Raspberry pi Part 5
Techvilla
 
Ad

More from Adrian Huang (6)

PDF
Linux Synchronization Mechanism: RCU (Read Copy Update)
Adrian Huang
 
PDF
spinlock.pdf
Adrian Huang
 
PDF
malloc & vmalloc in Linux
Adrian Huang
 
PDF
Physical Memory Management.pdf
Adrian Huang
 
PDF
Physical Memory Models.pdf
Adrian Huang
 
PDF
Anatomy of the loadable kernel module (lkm)
Adrian Huang
 
Linux Synchronization Mechanism: RCU (Read Copy Update)
Adrian Huang
 
spinlock.pdf
Adrian Huang
 
malloc & vmalloc in Linux
Adrian Huang
 
Physical Memory Management.pdf
Adrian Huang
 
Physical Memory Models.pdf
Adrian Huang
 
Anatomy of the loadable kernel module (lkm)
Adrian Huang
 
Ad

Recently uploaded (20)

PPTX
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
PDF
Automate Cybersecurity Tasks with Python
VICTOR MAESTRE RAMIREZ
 
PDF
SciPy 2025 - Packaging a Scientific Python Project
Henry Schreiner
 
PDF
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
PPTX
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
PPTX
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
PDF
AI + DevOps = Smart Automation with devseccops.ai.pdf
Devseccops.ai
 
PPTX
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
PDF
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
PPTX
Agentic Automation Journey Session 1/5: Context Grounding and Autopilot for E...
klpathrudu
 
PDF
Odoo CRM vs Zoho CRM: Honest Comparison 2025
Odiware Technologies Private Limited
 
PDF
Empower Your Tech Vision- Why Businesses Prefer to Hire Remote Developers fro...
logixshapers59
 
PPTX
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
PDF
Technical-Careers-Roadmap-in-Software-Market.pdf
Hussein Ali
 
PPTX
Milwaukee Marketo User Group - Summer Road Trip: Mapping and Personalizing Yo...
bbedford2
 
PPTX
Customise Your Correlation Table in IBM SPSS Statistics.pptx
Version 1 Analytics
 
PDF
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
PPTX
ChiSquare Procedure in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
Driver Easy Pro 6.1.1 Crack Licensce key 2025 FREE
utfefguu
 
PPTX
Foundations of Marketo Engage - Powering Campaigns with Marketo Personalization
bbedford2
 
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
Automate Cybersecurity Tasks with Python
VICTOR MAESTRE RAMIREZ
 
SciPy 2025 - Packaging a Scientific Python Project
Henry Schreiner
 
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
AI + DevOps = Smart Automation with devseccops.ai.pdf
Devseccops.ai
 
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
Agentic Automation Journey Session 1/5: Context Grounding and Autopilot for E...
klpathrudu
 
Odoo CRM vs Zoho CRM: Honest Comparison 2025
Odiware Technologies Private Limited
 
Empower Your Tech Vision- Why Businesses Prefer to Hire Remote Developers fro...
logixshapers59
 
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
Technical-Careers-Roadmap-in-Software-Market.pdf
Hussein Ali
 
Milwaukee Marketo User Group - Summer Road Trip: Mapping and Personalizing Yo...
bbedford2
 
Customise Your Correlation Table in IBM SPSS Statistics.pptx
Version 1 Analytics
 
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
ChiSquare Procedure in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
Driver Easy Pro 6.1.1 Crack Licensce key 2025 FREE
utfefguu
 
Foundations of Marketo Engage - Powering Campaigns with Marketo Personalization
bbedford2
 

Linux Kernel - Virtual File System

  • 1. Virtual File System (VFS) in Linux Kernel Adrian Huang | Nov, 2021 * Based on kernel 5.11 (x86_64) – QEMU * SMP (4 CPUs) and 8GB memory * Kernel parameter: nokaslr norandmaps * Userspace: ASLR is disabled * Legacy BIOS
  • 2. Agenda • Kernel components affected by an IO request • Object relationship in VFS • Ext4 file system format • dentry (Directory Entry) and inode (Index Node) • Example: ext4 • Anatomy of `mount` system call • Anatomy of `ls` system call
  • 3. Kernel components affected by an IO request VFS Page Cache/Buffer Cache (fs/buffer.c) Disk filesystem Disk filesystem Block Device File Mapping Layer Generic Block Layer I/O Scheduler Layer Block Device Driver Disk Disk Disk submit_bio()
  • 4. open(“/mnt/a.txt”, …) open(“/mnt/b.txt”, …) Process A File Object File Object File Object open file table dentry dentry dentry cache inode inode inode cache inode table Kernel - VFS dentry in data blocks Disk Object relationship in VFS dentry cache list • Linked via hlist_bl_{head,node} (hash table with bit lock) inode cache list • Linked via hlist_{head,node} (hash table) Note
  • 5. Super Block Group Descriptors Reserved GDT blocks Block bitmap inode bitmap inode table: inode descriptor Data Block Data Block . . . 0 1 Total inode count Total block count Free blocks Free inodes … Block Group 0 . . . Block Group N Boot Sector Partition mode Owner UID, Owner GID i_atime (last access time) i_ctime (last inode change time) i_mtime (last data modification time) size ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent_header eh_magic = 0xF30A eh_entries eh_max eh_depth eh_generation ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent ee_block ee_len ei_start_hi ei_start_lo ext4_extent ee_block ee_len ei_start_hi ei_start_lo . . . . . . root index node leaf node Disk blocks Extent Tree Ext4 file system format Storage size addressed by a block group: 4096 (bytes) * 8 * 4096 (block size) = 128MB
  • 6. Super Block Group Descriptors Reserved GDT blocks Block bitmap inode bitmap inode table: inode descriptor Data Block Data Block . . . 0 1 Total inode count Total block count Free blocks Free inodes … Block Group 0 . . . Block Group N Boot Sector Partition mode Owner UID, Owner GID i_atime (last access time) i_ctime (last inode change time) i_mtime (last data modification time) size ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent_header eh_magic = 0xF30A eh_entries eh_max eh_depth eh_generation ext4_extent_idx ei_block ei_leaf_lo ei_leaf_hi ei_unused ext4_extent ee_block ee_len ei_start_hi ei_start_lo ext4_extent ee_block ee_len ei_start_hi ei_start_lo . . . . . . root index node leaf node Disk blocks Extent Tree Ext4 file system format
  • 7. inode# Purpose 0 Doesn't exist; there is no inode 0 1 List of defective blocks 2 Root directory 3 User quota 4 Group quota 5 Boot loader 6 Undelete directory 7 Reserved group descriptors inode. ("resize inode") 8 Journal inode 9 The "exclude" inode 10 Replica inode 11 First non-reserved inode. Usually this is the lost+found directory. See s_first_ino in the superblock. Relationship between inode and directory entry (dentry) inode number link count
  • 8. inode bitmap: 19 * 4096 = 77824 (0x13000) inode 1-16 inode 17 `dumpe2fs 16MiB.img` Relationship between inode and directory entry (dentry)
  • 9. inode entry: 35 * 4096 + 128 * (2 – 1) [inode #2] = 0x23080 `dumpe2fs 16MiB.img` inode structure data of inode #2 Relationship between inode and directory entry (dentry)
  • 10. mode Owner UID, Owner GID i_atime i_ctime i_mtime size ext4_extent_header eh_magic = 0xF30A eh_entries = 1 eh_max = 4 eh_depth = 0 eh_generation = 0 root Disk blocks ext4_extent ee_block = 0 ee_len = 0x0001 ei_start_hi = 0x0000 ei_start_lo = 0x00000004 Relationship between inode and dentry – root (inode #2) ext4_inode (inode #2) leaf node . 2 name inode Directory Entry (dentry) Block #4: A list of struct ext4_dir_entry_2 .. 2 -> 2646185 lost+found 11 a123 12 a789 17 Block #0-3 Block #5 inode structure data of inode #2 Root Directory dentry in data block of inode #2
  • 11. type = directory i_links_count = 5 Relationship between inode and dentry – ‘b456’ (inode #13) i-node table . 2 name inode Directory Entry (dentry) - / .. 2 -> 2646185 lost+found 11 a123 12 a789 17 2 12 . 12 name inode .. 2 b456 13 hello.c 15 hello-hard-link.c 15 Directory Entry (dentry) - /a123 13 . 13 name inode .. 12 test.c 16 test-hard-link.c 16 test-soft-link.c 14 Directory Entry (dentry) - /a123/b456 14 16 Data block pointers type = directory i_links_count = 3 Data block pointers type = directory i_links_count = 2 Data block pointers type = symbolic link i_links_count = 1 Data block pointers = “test.c” type = file i_links_count = 2 Data block pointers File Data /a123/b456/test.c size = 6 size = 100 inode #
  • 12. type = directory i_links_count = 5 Relationship between inode and dentry – ‘test.c’ (inode #16) i-node table . 2 name inode Directory Entry (dentry) - / .. 2 -> 2646185 lost+found 11 a123 12 a789 17 2 12 . 12 name inode .. 2 b456 13 hello.c 15 hello-hard-link.c 15 Directory Entry (dentry) - /a123 13 . 13 name inode .. 12 test.c 16 test-hard-link.c 16 test-soft-link.c 14 Directory Entry (dentry) - /a123/b456 14 16 Data block pointers type = directory i_links_count = 3 Data block pointers type = directory i_links_count = 2 Data block pointers type = symbolic link i_links_count = 1 Data block pointers = “test.c” type = file i_links_count = 2 Data block pointers File Data /a123/b456/test.c size = 6 size = 100 inode structure data of inode #16 inode entry: 35 * 4096 + 128 * (16 – 1) [inode #16] = 0x23780 address inode structure of inode #16 Data block pointer represented by struct ext4_extent i_links_count inode #
  • 13. type = directory i_links_count = 5 Relationship between inode and dentry – ‘test-soft-link.c’ (inode #14) i-node table . 2 name inode Directory Entry (dentry) - / .. 2 -> 2646185 lost+found 11 a123 12 a789 17 2 12 . 12 name inode .. 2 b456 13 hello.c 15 hello-hard-link.c 15 Directory Entry (dentry) - /a123 13 . 13 name inode .. 12 test.c 16 test-hard-link.c 16 test-soft-link.c 14 Directory Entry (dentry) - /a123/b456 14 16 Data block pointers type = directory i_links_count = 3 Data block pointers type = directory i_links_count = 2 Data block pointers type = symbolic link i_links_count = 1 Data block pointers = “test.c” type = file i_links_count = 2 Data block pointers File Data /a123/b456/test.c size = 6 size = 100 inode structure data of inode #14 inode entry: 35 * 4096 + 128 * (14 – 1) [inode #14] = 0x23680 address inode structure of inode #14 i_links_count Data block pointer Hard Link 1. Refer to an inode number 2. Must reside on the same file system 3. Cannot be made to a directory Soft Link 1. Refer to a filename instead of an inode number 2. Can be linked to a file in a different file system 3. Can be made to a directory inode #
  • 14. mount – mount a new ext4 file system • stat() – get file status from inode o Name resolution → get dentry → get inode • openat() o Name resolution o Update open file table • mount() o Name resolution o Add a mount entry to ‘mount’ hash table o Parse file system
  • 15. stat("/adrian/img/src.img“, …) user_path_at(): fill a path struct (dentry and mount info) after resolving a pathname vfsmount mnt_root mnt_sb dentry d_name path mnt dentry d_inode path struct • stat() – get file status from inode o Name resolution → get dentry → get inode
  • 16. stat("/adrian/img/src.img“, …) 1. namei: convert a “name” to an “inode” 2. nameidata: store the current status when walking a path nameidata last name root inode path task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable
  • 17. stat("/adrian/img/src.img“, …) path_lookupat(): 1. Configure members of nameidata struct based on absolute pathname or relative pathname 2. Name resolution vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode * Data storage (not a data pointer) Note
  • 18. link_path_walk(): name resolution – round 1 vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” path mnt dentry current->fs->root d_inode qstr name = “adrian/img/src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “adrian/img/src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “img/src.img” hash 1 2 3 4 5 6 Note local variable link_path_walk() • stop until the last dentry is resolved • for-loop in this function qstr name = “adrian” u32 hash u32 len = 6 u64 hash_len union
  • 19. link_path_walk(): name resolution – round 1 nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” path mnt dentry current->fs->root qstr name = “adrian/img/src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “adrian/img/src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “img/src.img” hash 1 2 3 4 5 6 Note local variable lookup_slow()
  • 20. vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “img/src.img” u32 hash u32 len = 3 u64 hash_len union variable ‘name’ = “img/src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “src.img” hash 1 2 3 4 5 6 Note link_path_walk(): name resolution – round 2 qstr name = “img” u32 hash u32 len = 3 u64 hash_len union link_path_walk() • stop until the last dentry is resolved • for-loop in this function
  • 21. vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) * Data storage (not a data pointer) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “src.img” u32 hash u32 len = 3 u64 hash_len union variable ‘name’ = “src.img” dentry d_name d_inode path mnt dentry variable ‘name’ = “” 2 Note Function return link_path_walk(): name resolution – round 3 1 hash qstr name = “img” u32 hash u32 len = 3 u64 hash_len union link_path_walk() • stop until the last dentry is resolved • for-loop in this function
  • 22. vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “src.img” dentry d_name qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union d_inode path mnt dentry variable ‘name’ = “” 3 4 5 6 Function return lookup_last(): name resolution hash 1 2 * Data storage (not a data pointer) Note lookup_last() • get the corresponding dentry based on the filename • “while loop”: Follow the symlink in the final component
  • 23. vfsmount mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len u64 hash_len union nameidata last (S*) name root (S*) inode path (S*) task_struct mm nameidata filename name uptr (userland pointer) refcnt aname char iname[] = “/adrian/img/src.img" char __user *name = “/adrian/img/src.img” local variable path mnt dentry current->fs->root d_inode qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union variable ‘name’ = “src.img” dentry d_name qstr name = “src.img” u32 hash u32 len = 6 u64 hash_len union d_inode path mnt dentry variable ‘name’ = “” 3 4 5 6 Function return lookup_last(): name resolution hash 1 2 * Data storage (not a data pointer) Note
  • 24. mount – mount a new ext4 file system
  • 25. hlist_head.first hlist_head.first . . mount_hashtable mount mnt_parent struct dentry *mnt_mountpoint struct vfsmount mnt **pprev *next const char *mnt_devname = “none” mount mnt_parent struct dentry *mnt_mountpoint **pprev struct hlist_node mnt_hash *next const char *mnt_devname = “none” struct hlist_node mnt_hash dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_sops s_root s_bdev s_id[32] = “rootfs” s_fs_info s_type = rootfs_fs_type d_parent mnt_root mnt_sb struct vfsmount mnt mount – before mounting a new ext4 file system d_sb
  • 26. hlist_head.first hlist_head.first . . mount_hashtable mount mnt_parent struct dentry *mnt_mountpoint struct vfsmount mnt **pprev *next const char *mnt_devname = “none” mount mnt_parent struct dentry *mnt_mountpoint **pprev struct hlist_node mnt_hash *next const char *mnt_devname = “none” struct hlist_node mnt_hash dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_sops s_root s_bdev s_id[32] = “rootfs” s_fs_info s_type = rootfs_fs_type d_parent mnt_root mnt_sb struct vfsmount mnt mount – after mounting a new ext4 file system mount mnt_parent struct dentry *mnt_mountpoint **pprev *next const char *mnt_devname = “/dev/loop0” mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union struct vfsmount mnt super_block s_op = ext4_sops s_id[32] = “loop0” s_type = ext4_fs_type d_parent struct hlist_node mnt_hash dentry d_name d_parent qstr name = “mnt” u32 hash u32 len = 3 u64 hash_len union /adrian/mnt d_sb d_sb
  • 27. hlist_head.first hlist_head.first . . mount_hashtable mount mnt_parent struct dentry *mnt_mountpoint struct vfsmount mnt **pprev *next const char *mnt_devname = “none” mount mnt_parent struct dentry *mnt_mountpoint **pprev struct hlist_node mnt_hash *next const char *mnt_devname = “none” struct hlist_node mnt_hash dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_sops s_root s_bdev s_id[32] = “rootfs” s_fs_info s_type = rootfs_fs_type d_parent mnt_root mnt_sb struct vfsmount mnt mount – after mounting a new ext4 file system mount mnt_parent struct dentry *mnt_mountpoint **pprev *next const char *mnt_devname = “/dev/loop0” mnt_root mnt_sb dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union struct vfsmount mnt super_block s_op = ext4_sops s_id[32] = “loop0” s_type = ext4_fs_type d_parent struct hlist_node mnt_hash dentry d_name d_parent qstr name = “mnt” u32 hash u32 len = 3 u64 hash_len union /adrian/mnt d_sb d_sb
  • 29. `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …)
  • 30. `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …)
  • 31. `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …)
  • 32. path mnt dentry vfsmount mnt_root mnt_sb qstr name = “mnt” u32 hash u32 len = 3 u64 hash_len union dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ramfs_ops s_root s_bdev s_id[32] = “rootfs” s_fs_info d_inode dentry d_name d_inode d_parent s_type = rootfs_fs_type path mnt dentry vfsmount mnt_root mnt_sb qstr name = “mnt /” u32 hash u32 len = 3 1 u64 hash_len union dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ext4_sops s_root s_bdev s_id[32] = “loop0” s_fs_info d_inode dentry d_name d_inode d_parent s_type = ext4_fs_type /adrian/mnt data structures in the original file system /adrian/mnt data structures in newly mounted file system traverse_mounts() `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …) The original mount point is changed the newly one when running name resolution
  • 33. path mnt dentry vfsmount mnt_root mnt_sb qstr name = “mnt /” u32 hash u32 len = 3 1 u64 hash_len union dentry d_name qstr name = “/” u32 hash u32 len = 1 u64 hash_len union super_block s_blocksize s_maxbytes s_op = ext4_sops s_root s_bdev s_id[32] = “loop0” s_fs_info d_inode dentry d_name d_inode d_parent s_type = ext4_fs_type /adrian/mnt data structures in newly mounted file system flags: path->dentry->d_flags `ls –color mnt`: openat(AT_FDCWD, "/adrian/mnt“, …) REF-walk [Two concurrent mechanisms for name resolution] 1. REF-walk: concurrent management with refcounts and spinlock 2. RCU-walk: faster name resolution lookup