ptrace and forensic analysis

The ptrace() command is the system call that is most commonly used for memory analysis of a userland. In fact, if you are designing forensics software that runs in userland, the only way it can access other processes memory is through the ptrace system call, or by reading the proc filesystem (unless, of course, the program has some type of explicit shared memory IPC setup).

Note

One may attach to a process and then open/lseek/read/write /proc/<pid>/mem as an alternative to ptrace read/write semantics.

In 2011, I was awarded a contract by the DARPA CFT (Cyber Fast Track) program to design something called Linux VMA Monitor. The purpose of this software is to detect a wide range of known and unknown process memory infections, such as rootkits and memory-resident viruses.

It essentially performs automated intelligent memory forensic analysis on every single process address space using special heuristics that understands ELF execution. It can spot anomalies or parasites, such as hijacked functions and generic code infections. The software can either analyze live memory and work as a host intrusion detection system, or take snapshots of the process memory and perform an analysis on them. This software can also detect and disinfect ELF binaries that are infected with viruses on disk.

The ptrace system call is used heavily in the software and demonstrates a lot of interesting code around the ELF binary and ELF runtime infections. I have not released the source code as I intend to provide a more production-ready version prior to the release. Throughout this text, we will cover almost all the infection types that Linux VMA Monitor can detect/disinfect, and we will discuss and demonstrate the heuristics used to identify these infections.

For well over a decade, hackers have been hiding complex malware within process memory to remain stealthy. This may be a combination of shared library injection and GOT poisoning, or any other set of techniques. The chances of a system administrator finding these are very slim, especially since there is not a lot of software publicly available for detecting many of these attacks.

I have released several tools, including but not limited to AVU and ECFS, both of which can be found on GitHub and my website at http://bitlackeys.org/. Whatever other software is in existence for such things is highly specialized and privately used, or it simply may not exist at all. Meanwhile, a good forensics analyst can use a debugger or write custom software to detect such malware, and it is important to know what you are looking for and why. Since this chapter is all about ptrace, I wanted to emphasize how it is interrelated with forensic analysis. And it is, and especially for those who are interested in designing specialized software for the purpose of identifying threats in memory.

Towards the end of the chapter, we will see how to write a program to detect function trampolines in running software.

What to look for in the memory

An ELF executable is nearly the same in the memory as it is on the disk, with the exception of changes to the data segment variables, global offset table, function pointers, and uninitialized variables (the .bss section).

This means that many of the virus or rootkit techniques that are used in ELF binaries can also be applied to processes (runtime code), and therefore they are better for an attacker to remain hidden. We will cover all of these common infection vectors in depth throughout the book, but here is a list of some techniques that have been used to implement infectious code:

Using a combination of ELF format parsing, /proc/<pid>/maps, and ptrace, one can create a set of heuristics to detect every one of the preceding techniques, and create a counter method to disinfect the process from the so-called parasite code. We will delve into all of these techniques throughout the book, primarily in Chapter 4, ELF Virus Technology – Linux/Unix Viruses and Chapter 6, ELF Binary Forensics in Linux.