Mac OS X PPC Shellcode Tricks - Chapter 1 & 2

Mac OS X PPC Shellcode Tricks

H D Moore

hdm@metasploit.com

 

Chapter 1

Foreword

Abstract: Developing shellcode for Mac OS X is not particularly difficult, but there are a number of tips and techniques that can make the process easier and more effective. The independent data and instruction caches of the PowerPC processor can cause a variety of problems with exploit and shellcode development.

The common practice of patching opcodes at run-time is much more involved when the instruction cache is in incoherent mode. NULL-free shellcode can be improved by taking advantage of index registers and the reserved bits found in many opcodes, saving space otherwise taken by standard NULL evasion techniques. The Mac OS X operating system introduces a few challenges to unsuspecting developers; system calls change their return address based on whether they succeed and oddities in the Darwin kernel can prevent standard execve() shellcode from working properly with a threaded process. The virtual memory layout on Mac OS X can be abused to overcome instruction cache obstacles and develop even smaller shellcode.

Thanks: The author would like to thank B-r00t, Dino Dai Zovi, LSD, Palante, Optyx, and the entire Uninformed Journal staff.

Chapter 2

Introduction

With the introduction of Mac OS X, Apple has been viewed with mixed feelings by the security community. On one hand, the BSD core offers the familiar Unix security model that security veterans already understand. On the other, the amount of proprietary extensions, network-enabled software, and growing mass of advisories is giving some a cause for concern. Exploiting buffer overflows, format strings, and other memory-corruption vulnerabilities on Mac OS X is a bit different from what most exploit developers are familiar with. The incoherent instruction cache, combined with the RISC fixed-length instruction set, raises the bar for exploit and payload developers.

On September 12th of 2003, B-r00t published a paper titled ”Smashing the Mac for Fun and Profit”. B-root’s paper covered the basics of Mac OS X shellcode development and built on the PowerPC work by LSD, Palante, and Ghandi.

This paper is an attempt to extend, rather than replace, the material already available on writing shellcode for the Mac OS X operating system. The first section covers the fundamentals of the PowerPC architecture and what you need to know to start writing shellcode. The second section focuses on avoiding NULL bytes and other characters through careful use of the PowerPC instruction set.

The third section investigates some of the unique behavior of the Mac OS X platform and introduces some useful techniques.