Driver development – memory management
Download as PPT, PDF2 likes1,931 views
1. The document discusses various methods for managing memory in the Linux kernel, including physical memory, virtual memory, page tables, and different allocators like kmalloc, vmalloc, and SLAB for allocating memory to processes and the kernel. 2. It explains concepts like physical vs virtual addresses, page tables that map virtual to physical memory, and the Memory Management Unit (MMU) that handles virtual address translation. 3. Different allocators like kmalloc, vmalloc and SLAB are used depending on the size and properties of the memory needed, with kmalloc and SLAB handling physically contiguous memory and vmalloc only requiring virtual contiguity.
Driver development – memory management
- 1. Driver development – memory management • Physical memory and virtual memory • Virtual memory organization • Physical and virtual memory mapping • Accessing physical memory • Allocators in kernel memory • Kmalloc allocator and APIs • Vmalloc allocator and APIs
- 2. MMU CPU Kernel space User space 0xFFFFFFFF 0x00000000 0xFFFFFFFF 0x00000000 0x00000000 0xFFFFFFFF 0xC0000000 Physical and virtual address 0xC0000000 Physical address space Virtual address space Process 1 Process 2 All processes have their own virtual address space , and run as if they had access to the whole address space Memory Management Unit Kernel space User space
- 3. Physical address • Physical memory is storage hardware that records data with low latency and small granularity. • Physical memory addresses are numbers sent across a memory bus to identify the specific memory cell within a piece of storage hardware associated with a given read or write operation. • Examples of storage hardware providing physical memory are DIMMs (DRAM), SD memory cards (flash), video cards (frame buffers and texture memory), and so on. • Only the kernel uses physical memory addresses directly. • User space programs exclusively use virtual addresses.
- 4. Virtual address • Virtual memory provides a software-controlled set of memory addresses, allowing each process to have its own unique view of a computer's memory. • Virtual addresses only make sense within a given context, such as a specific process. The same virtual address can simultaneously mean different things in different contexts. • Virtual addresses are the size of a CPU register. On 32 bit systems each process has 4 gigabytes of virtual address space all to itself, which is often more memory than the system actually has. • Virtual addresses are interpreted by a processor's Memory Management Unit (mmu), using data structures called page tables which map virtual address ranges to associated content. • Virtual memory is used to implement allocation, swapping, file mapping, copy on write shared memory, defragmentation, and more.
- 5. Memory management Unit (MMU) • The memory management unit is the part of the CPU that interprets virtual addresses. • Attempts to read, write, or execute memory at virtual addresses are either translated to corresponding physical addresses, or else generate an interrupt (page fault) to allow software to respond to the attempted access. • This gives each process its own virtual memory address range, which is limited only by address space (4 gigabytes on most 32-bit system), while physical memory is limited by the amount of available storage hardware. • Physical memory addresses are unique in the system, virtual memory addresses are unique per-process.
- 7. Page tables • Page tables are data structures which contains a process's list of memory mappings and track associated resources. • Each process has its own set of page tables, and the kernel also has a few page table entries for things like disk cache. • 32-bit Linux systems use three-level tree structures to record page tables. The levels are the Page Upper Directory (PUD), Page Middle Directory (PMD), and Page Table Entry (PTE). • 64-bit Linux can use 4-level page tables.
- 8. CPU cache • The CPU cache is a very small amount of very fast memory built into a processor, containing temporary copies of data to reduce processing latency. • The L1 cache is a tiny amount of memory (generally between 1k and 64k) wired directly into the processor that can be accessed in a single clock cycle. • The L2 cache is a larger amount of memory (up to several megabytes) adjacent to the processor, which can be accessed in a small number of clock cycles. • Access to un-cached memory (across the memory bus) can take dozens, hundreds, or even thousands of clock cycles.
- 9. Translation look–aside buffer (TLB) • The TLB is a small fixed-size array of recently used pages, which the CPU checks on each memory access. • It lists a few of the virtual address ranges to which physical pages are currently assigned. • The TLB is a cache for the MMU. • Accesses to virtual addresses listed in the TLB go directly through to the associated physical memory • Accesses to virtual addresses not listed in the TLB (a "TLB miss") trigger a page table lookup, which is performed either by hardware, or by the page fault handler, depending on processor type.
- 10. Kernel memory - pages • The kernel treats physical pages as the basic unit of memory management. • Although the processor’s smallest addressable unit is a byte or a word, the memory management unit typically deals in pages. • In terms of virtual memory, pages are the smallest unit that matters. • Most 32-bit architectures have 4KB pages, whereas most 64-bit architectures have 8KB pages. • This implies that on a machine with 4KB pages and 1GB of memory, physical memory is divided into 262,144 distinct pages. • The kernel memory manager also handles smaller memory (less than page size) allocation using the slabs/SLUB allocator. • Kernel allocated pages cannot be swapped. They always remain in memory.
- 11. Memory Zones • Not all memory is equally addressable • Different types of memory have to be used for different things • Linux uses different zones to handle this – ZONE DMA: Some older I/O devices can only address memory up to 16M – ZONE NORMAL: Regular memory up to 896M – ZONE HIGHMEM: Memory above 896M
- 12. Virtual memory organization: 1GB/3GB • 1GB reserved for kernel-space • Contains kernel code and core data structures identical in all address spaces • Most memory can be a direct mapping of physical memory at a fixed offset • Complete 3GB exclusive mapping available for each user-space process • Process code and data (program, stack, …) • Memory-mapped files, not necessarily mapped to physical memory User Space Processes N Kernel Space 0xFFFFFFFF 0x00000000 0xC0000000
- 13. Page allocators in the kernel Some kernel Code Kmalloc() allocator Vmalloc ()allocator Non-physical Contiguous memory SLAB allocator Allows to create caches, each cache storing objects of the same size. Page Allocator Allows to allocate contiguous areas of physical pages (4K, 8K, 16K , etc.)
- 14. Page allocators • Suitable for data larger than page size for e.g. 4K s • The kernel represents every physical page on the system with the ‘struct page’ data structure, defined in linux/mm_types.h • The kernel use this data structure to keep track of all pages in the system, because the kernel needs to know whether the page is free (i.e. page is not allocated) • The allocated area is virtually contiguous but also physically contiguous. It is allocated in the identity-mapped part of the kernel memory space. • This means that large areas may not be available or hard to retrieve due to physical memory fragmentation.
- 15. Getting pages • The kernel provides one low-level mechanism for requesting memory, along with several interfaces to access it. • All these interfaces allocate memory with page-size granularity and are declared in linux/gfp.h. • The core function is struct page* alloc_pages(gfp_t gfp_mask, unsigned int order); • This allocates 2^order (i.e. 1<<order) contiguous physical pages • On success, returns a pointer to the first page’s page structure • On error, returns NULL
- 16. Contd… • To get logical address from the page pointer void *page_address(struct page *page); • This returns a pointer to the logical address where the given physical page resides. • If you don’t need the actual struct page, you can call unsigned long __get_free_pages(gfp_t gfp_mask, unsigned int order); • This function works the same as alloc_pages(), except that it directly returns the logical address of the first requested page. • To allocate single page struct page * alloc_page(gfp_t gfp_mask); unsigned long __get_free_page(gfp_t gfp_mask);
- 17. Freeing pages • A family of functions enables you to free allocated pages when you no longer need them: void __free_pages(struct page *page, unsigned int order) void free_pages(unsigned long addr, unsigned int order) void free_page(unsigned long addr) • You must be careful to free only pages you allocate. • Passing the wrong struct page or address, or the incorrect order, can result in corruption.
- 18. Page allocator flags • GFP_KERNEL • Standard kernel memory allocation. The allocation may block in order to find enough available memory. Fine for most needs, except in interrupt handler context. • GFP_ATOMIC • RAM allocated from code which is not allowed to block (interrupt handlers or critical sections). Never blocks, allows to access emergency pools, but can fail if no free memory is readily available. • GFP_DMA • Allocates memory in an area of the physical memory usable for DMA transfers. • Others are defined in include/linux/gfp.h • (GFP: __get_free_pages).
- 19. SLAB allocator • There are certain kinds of data structures that are frequently allocated and freed • Instead of constantly asking the kernel memory allocator for such pieces, they’re allocated in groups and freed to per-type linked lists. • To allocate such an object, check the linked list; only if it’s empty is the generic memory allocator called. • The object size can be smaller or greater than the page size • To free such an item, just put it back on the list. • If a set of free objects constitute an entire page, it can be reclaimed if necessary
- 20. Contd… • The SLAB allocator takes care of growing or reducing the size of the cache as needed, depending on the number of allocated objects. It uses the page allocator to allocate and free pages. • SLAB caches are used for data structures that are present in many instances in the kernel: directory entries, file objects, network packet descriptors, process descriptors, etc. • See /proc/slabinfo • They are rarely used for individual drivers. • See include/linux/slab.h for the API
- 21. Kmalloc allocator • The kmalloc() function is a simple interface for obtaining kernel memory in byte-sized chunks. If you need whole pages, the previously discussed interfaces might be a better choice. • The kmalloc allocator is the general purpose memory allocator in the Linux kernel, for objects from 8 bytes to 128 KB • The allocated area is guaranteed to be physically contiguous • The allocated area size is rounded up to the next power of two size • The kmalloc() function’s operation is similar to that of user-space’s familiar malloc() routine, with the exception of the additional flags parameter. • It uses the same flags as the page allocator (gfp_t and gfp_mask) with the same semantics. • It should be used as the primary allocator unless there is a strong reason to use another one.
- 22. Kmalloc API • #include <linux/slab.h> void *kmalloc(size_t size, int flags); • Allocate size bytes, and return a pointer to the area (virtual address) • size: number of bytes to allocate • flags: same flags as the page allocator void *kzalloc(size_t size, gfp_t flags); • Allocates a zero-initialized buffer void kfree (const void *ptr); • Free an allocated area
- 23. Vmalloc • The vmalloc() function works in a similar fashion to kmalloc(), except it allocates memory that is only virtually contiguous and not necessarily physically contiguous. • This is how a user-space allocation function works. • The pages returned by malloc() are contiguous within the virtual address space of the processor, but there is no guarantee that they are actually contiguous in physical RAM. • The kmalloc() function guarantees that the pages are physically contiguous (and virtually contiguous). • The vmalloc() function ensures only that the pages are contiguous within the virtual address space. • It does this by allocating potentially non-contiguous chunks of physical memory and “fixing up” the page tables to map the memory into a contiguous chunk of the logical address space.
- 24. Contd… • Mostly hardware devices require physically contiguous memory allocations. • Any regions of memory that hardware devices work with must exist as a physically contiguous block and not merely a virtually contiguous one. • Blocks of memory used only by software— for example, process-related buffers—are fine using memory that is only virtually contiguous. • In your programming, you never know the difference. • All memory appears to the kernel as logically contiguous.
- 25. Vmalloc API • #include <linux/vmalloc.h> void *vmalloc(unsigned long size); • On success, returns pointer to virtually contiguous memory • On error, returns NULL • Void vfree(const void *ptr) • Frees the block of memory beginning at ‘ptr’ that was previously allocated with vmalloc.
- 26. Picking an allocation method • If you need contiguous physical pages, use one of the low-level page allocators or kmalloc(). • The two most common flags given to these functions are GFP_ATOMIC and GFP_KERNEL. • Specify the GFP_ATOMIC flag to perform a high priority allocation that will not sleep. This is a requirement of interrupt handlers and other pieces of code that cannot sleep. • Code that can sleep, such as process context code , should use GFP_KERNEL. This flag specifies an allocation that can sleep, if needed, to obtain the requested memory. • If you do not need physically contiguous pages—only virtually contiguous —use vmalloc()