Loop Detection - Chapter 1 & 2

Loop Detection

Peter Silberman

peter.silberman@gmail.com

 

Chapter 1

Foreword

Abstract: During the course of this paper the reader will gain new knowledge about previous and new research on the subject of loop detection. The topic of loop detection will be applied to the field of binary analysis and a case study will given to illustrate its uses. All of the implementations provided in this document have been written in C/C++ using Interactive Disassembler (IDA) plug-ins.

Thanks: The author would like to thank Pedram Amini, thief, Halvar Flake, skape, trew, Johnny Cache and everyone else at nologin who help with ideas, and kept those creative juices flowing.

Chapter 2

Introduction

The goal of this paper is to educate the reader both about why loop detection is important and how it can be used. When a security researcher thinks of insecure coding practices, things like calls to strcpy and sprintf are some of the first things to come to mind. These function calls are considered low hanging fruit. Some security researchers think of integer overflows or off-by-one copy errors as types of vulnerabilities. However, not many people consider, or think to consider, the mis-usage of loops as a security problem. With that said, loops have been around since the beginning of time (e.g. first coding languages). The need for a language to iterate over data to analyze each object or character has always been there. Still, not everyone thinks to look at a loop for security problems.

What if a loop doesn’t terminate correctly? Depending on the operation the loop is performing, it’s possible that it could corrupt surrounding memory regions if not properly managed. If the loop frees memory that no longer exists or is not memory, a double-free bug could’ve been found. These are all things that could, and do, happen in a loop.

As the low hanging fruit is eliminated in software by security researchers and companies doing decent to moderate QA testing, the security researchers have to look elsewhere to find vulnerabilities in software. One area that has only been touched on briefly in the public relm, is how loops operate when translated to binaries (BugScan is an example of a company that has implemented ”buffer iteration” detection but hasn’t talked publically about it. http://www.logiclibrary.com). The reader may ask: why would one want to look at loops? Well, a lot of companies implement their own custom string routines, like strcpy and strcat, which tend to be just as dangerous as the standard string routines.

These functions tend to go un-analyzed because there is no quick way to say that they are copying a buffer. Due to this reason, loop detection can help the security research identify areas of interest. During the course of this article the reader will learn of the different ways to detect loops using graph analysis, how to implement loop detection, see a new loop detection IDA plug-in, and a case study that will tie it all together