Linux-Kernel-Internals-Part-1

[As part of my graduate studies and GATE preparation, I got to know basic concepts of Operating System but I always this in the back of my mind wish I could get a sneak-peek of linux OS specially kernel.
Recent discovery of security flaws in Intel processor design has forced major operating system vendors in this space to provide critical patches. So, here I am fulfilling my wish!  🙂 ]

What is Operating System?

In simple terms, Operating system is resource manager for your computer. In other words, it’s a software which enables interaction(use/administration/control) between your computer hardware and other softwares. OS includes kernel, device-drivers, boot-loader, user interfaces, basic file and system utilities.

What is kernel?

Kernel is the software that provides basic services for all other part of system, manages hardware and distributes system resources. User-interfaces are outermost part of OS and kernel is the innermost.

What are typical components of linux kernel?

1. Interrupt handlers: to service interrupt requests
2. Scheduler: to share processor time
3. Memory management system: to manage process address spaces
4. Other System Services: to enable networking and inter-process communication.

Relationship between applications, kernel and hardware: How does applications get work done?

The following diagram should demonstrate relation between applications, kernel and hardware.

1. System Memory in linux is divided into kernel space (where kernel runs(executes) and provides it’s services) and user space (set of memory locations in which user processes (i.e., everything other than the kernel) run).  One of the roles of the kernel is to manage individual user processes within this space and to prevent them from interfering with each other[1].
2. Kernel typically resides in an elevated system state as compared to normal user application. User applications only see a subset of machine’s available resources. Applications communicate with kernel via system calls.
3. When an application executes a system call, we say “kernel is executing on behalf of application”.
   Application is said to be “executing system-call in kernel space” and kernel is running in “process context.”
4. Kernel has full access to hardware. When a hardware needs to communicate with the system, it sends an interrupt to the processor which in turn interrupts the kernel.
5. Each interrupt has an assigned number which executes a specific interrupt handler to process and respond to the interrupt. Interrupt handlers don’t run in process context, instead they run in “interrupt context” that is not associated with any process.

Each processor in linux is doing exactly one of the 3 following things at any given time:

1. In User Space: Executing User Code in a Process
2. In Kernel Space: In process context executing on behalf a specific process.
3. In Kernel Space: In interrupt context, not associated with any process, handling an interrupt.

Getting linux kernel source code both 2.6 and 4.14

I recommend you get source code of both version 2.6 and 4.14(latest) and extract them to separate folders, so that as we go along, we may switch as required.

Download URL for Linux Kernel 2.6.34: https://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.34.tar.gz
Download URL for Linux Kernel 4.14: https://www.kernel.org/pub/linux/kernel/v4.x/linux-4.14.tar.gz

Please download both of the file and extract them by running tar -xvf command:

Folder structure(No need to remember them):

1. Documentation: Kernel Source Documentation
2. arch : Architecture specific source
3. block: Block I/O layer
4. crypto: Crypto API
5. drivers: Device Drivers
6. firmware: Device firmware needed to use certain drivers
7. fs: VFS and individual file systems
8. include: Kernel Headers
9. init: Kernel Boot and Intialization
10. ipc: Inter Process Communication
11. kernel: Core Subsystems
12. lib: Helper Routines
13. mm: Memory Management Subsystem ad the VM
14. net: Networking Subsystem
15. samples: sample demonstrative code
16. scripts: scripts used to build linux kernel
17. security: Linux Security Module
18. sound: Sound subsystem
19. tools:Tools helpful for developing linux
20. usr: Early user-space code(called initramfs)
21. virt: Virtualization Infrastructure.

What to expect next from me? A series of videos complemented with blog posts about linux kernel internals. Specifically I want to cover the following topics:

1. Process Management, Scheduling and System Calls.
2. Synchronization and timing, Interrupts
3. Memory Management
4. File System
5. Any other **appealing** topic.

I am referring this book: Linux Kernel Development by Robert Love for getting to know more. Although it is based on Linux kernel version 2.6.34(quite old), most of the concepts still apply to today’s kernel 4.14. I will try to point out differences as I explore further.

References:

[1] Kernel Space Definition