Skip to content

Hacking on Manticore

Jump to bottom Edit New page
Mark Mossberg edited this page Mar 23, 2018 · 21 revisions

Developer Installation

For a dev install that includes dependencies for tests, run:

git clone https://github.com/trailofbits/manticore.git && cd manticore
pip install --no-binary keystone-engine -e .[dev]

Note: That pip command tries to install to the system install directory, which might fail with "Permission Denied". We recommend alternatively using a virtualenv or a user install (pip install --no-binary keystone-engine --user -e .[dev]; installs everything into ~/.local).

The -e flag to pip will install the package in "editable" mode. This means you don't have to reinstall the package after making changes to the source.

You can run the tests with the commands below. The nosetests utility is installed automatically by that pip command line and can be used to conveniently run tests.

cd manticore
# all tests
nosetests
# just one file
nosetests tests/test_armv7cpu.py
# just one test class
nosetests tests/test_armv7cpu.py:Armv7CpuInstructions
# just one test
nosetests tests/test_armv7cpu.py:Armv7CpuInstructions.test_mov_imm_min

Alternatively, you can invoke tests via python -m nose.

Moreover, you can invoke multiprocess test invocation via the --processes flag. Note, however, that several tests (e.g., tests/test_memdumps.py) require longer execution times, thus you need to specify the appropriate timeout period via the --process-timeout flag. E.g.,

nosetests --processes=8 --process-timeout=120 tests/test_binaries.py

Text Editors

We use a range of text editors, including PyCharm and Vim. @mossberg likes PyCharm because it catches a lot of silly bugs for him ;)

Event System

Manticore implements a synchronous publish/subscribe system for executing code when certain analysis events happen.

If you are implementing an event handler there are certain things you need to know about locking: https://github.com/trailofbits/manticore/pull/433#issuecomment-320056828

Existing events that get produced

The following classes produce the following events:

  • manticore.core.cpu.Cpu
    • will_write_register, arguments: register, value
    • did_write_register, arguments: register, value
    • will_read_register, arguments: register
    • did_read_register, arguments: register, value
    • will_write_memory, arguments: where, expression, size
    • did_write_memory, arguments: where, expression, size
    • will_read_memory, arguments: where, size
    • did_read_memory, arguments: where, value, size
    • will_decode_instruction, arguments: pc
    • will_execute_instruction, arguments: instruction
    • will_emulate_instruction, arguments: instruction
    • did_emulate_instruction, arguments: instruction
    • did_execute_instruction, arguments: instruction
  • manticore.core.Executor
    • did_add_state, arguments: state_id, state
    • will_generate_testcase, arguments: state, prefix, message
    • will_fork_state, arguments: state, expression, solutions, policy
    • forking_state, arguments: state, expression, new_value, policy
    • will_load_state, arguments: current_state, current_state_id
    • will_terminate_state, arguments: current_state, current_state_id, message or exception
    • will_finish_run
    • All events produced by currently-executing manticore.core.State
  • manticore.core.State
    • state_generate_inputs, arguments: name, message
    • All events produced by manticore.platforms.Platform
  • manticore.platforms.Platform
    • All events produced by manticore.core.cpus.Cpu

Adding a Syscall

as of a0717aa661c0b04d5f73879b265da4da05756630

To implement a Linux syscall:

  • Look up the name of your syscall in manticore/platforms/linux_syscalls.py to get the correct name of your syscall for the corresponding syscall number.
  • In manticore/platforms/linux.py, add a method to the SLinux (Symbolic Linux) class for your syscall. Name your method precisely the name above. The arguments to this method should be
    • 1: self (standard Python self variable)
    • 2: cpu (manticore.core.abstractcpu.Cpu object representing current cpu state)
    • 3+: arguments to the syscall
  • Implement the logic of the syscall in this method, using the Cpu APIs as needed
  • The method should return the value returned by the syscall

Note on syscalls

When adding syscalls, make sure you're implementing the semantics defined by the system call, and not the similarly-named libc implementation. This mostly entails how you communicate error, for example, returning a negative errno value instead of returning -1 and setting a global. Check the linux source for the uses of SYSCALL_DEFINEn (where n is number of arguments. For more information, see adding-syscalls.rst) macros.

Adding an Instruction

as of c78ea5c9109191654d26c7bfd2bedd662dafcdc5

To implement a cpu instruction:

  • Open the file according to the architecture for this instruction
    • x86 is in manticore/core/cpu/x86.py
    • armv7 is in manticore/core/cpu/arm.py
  • Add a method to the Cpu class in either of those files that subclasses Cpu
    • Decorate it with the @instruction decorator
    • The arguments to the method should be
      • 1: self
      • 2+ one argument for every operand in instruction.operands as decoded by Capstone. The types of these arguments are manticore.core.abstractcpu.Operand which is a light wrapper over a Capstone operand object (e.g. ArmOp)) and notably support convenience .read and .write methods.
  • Implement the instruction's effects

Using PDB

You can use pdb to debug your Manticore hooks. However, you must run Manticore in single-process mode (i.e. call Manticore.run(procs=1), or don't specify it at all) for this to work. Doing so when executing a multi-worker analysis run will cause pdb.set_trace() and other interactive commands to fail.

Source Tree

manticore/
├── binary # code related to binary formats. ignore this
│   ├── grr
│   │   ├── __init__.py
│   │   └── snapshot.py
│   ├── __init__.py
│   └── pe
│       ├── __init__.py
│       └── minidump.py
├── core
│   ├── cpu # code implementing symbolic emulators
│   │   ├── abstractcpu.py
│   │   ├── arm.py
│   │   ├── bitwise.py
│   │   ├── cpufactory.py
│   │   ├── __init__.py
│   │   ├── register.py
│   │   └── x86.py
│   ├── executor.py # main symbolic execution file
│   ├── __init__.py
│   ├── mappings.py
│   ├── memory.py
│   ├── parser
│   │   ├── __init__.py
│   │   └── parser.py
│   ├── smtlib # code related to handling symbolic data
│   │   ├── constraints.py #
│   │   ├── expression.py # defines symbolic data types
│   │   ├── __init__.py
│   │   ├── operators.py # library of operators for transparently handling concrete or symbolic data
│   │   ├── solver.py # code for interacting with the SMT solver
│   │   └── visitors.py # code for transforming expression trees, including serializing to SMTLIB
│   └── state.py # defines type for program state
├── __init__.py
├── __main__.py
├── manticore.py # high level API object
├── platforms # operating system models implemented here
│   ├── cgcrandom.py
│   ├── decree.py
│   ├── __init__.py
│   ├── libc.py
│   ├── linux.py
│   ├── windows.py
│   └── windows_syscalls.py
└── utils
    ├── emulate.py # code integrating unicorn for emulation of unimplemented instructions
    ├── event.py
    ├── helpers.py
    ├── __init__.py
    ├── iterpickle.py
    └── nointerrupt.py

Building the html docs

  • Do a Manticore dev install (see README), or just install Sphinx
  • cd docs
  • make html
Add a custom footer
Clone this wiki locally