Beekeeping Linux: Evolution of Kernel Technologies For Instrumentation

As developers and engineers build anything, it's hard to simulate how it's going to function in actual environment. That's why to have a proactive way to understand and improve software we add ability to monitor in real time how it's functioning. Every since Linux has started becoming mainstream, there has always been developments towards how to provide these system level insights in most accessible way. This talk will go through the journey of how the technology of Linux Instrumentation has evolved over the years to adjust to varied kind and scale of workloads.

Topics to be covered:

0. Understanding Instrumentation and the need for it.

1. unix /dev/mem to directly accessing information from memory

2. procfs

3. ptrace -> strace

4. perf/auditd/ring buffers ( ways to relay information more efficiently )

5. kprobes and tracepoints

6. Kernel Modules

   - SystemTap

   - Ways to leverage kprobes and tracepoints

   - Downsides of Kernel Modules

8. ftrace

9. eBPF

   - Is it the silver bullet?

   - When and at what scale does eBPF makes sense?

- Kernel Runtime Security Instrumentation (KRSI) : Now referred to as BPF LSM

---

I presented at IndiaFOSS 2.0 (Ref https://www.youtube.com/watch?v=yEFO31iv56g) and talked about eBPF for security. Security is combination of both Observability and Enforcement. This talk goes deeper into Observability and explores how eBPF as a technology came into being more from a Instrumentation Perspective and eventually leading to Security Use Cases as well.