Reverse Engineering Basics

The goal

Reverse engineering in CTFs means reading a compiled binary to understand its behaviour — usually to satisfy a hidden check that prints the flag, or to extract the flag from data embedded in the binary.

Know what you have first

Before opening a disassembler, identify the file:

file challenge          # ELF, PE, Mach-O, script, archive…
strings challenge       # printable strings — flags often hide here
xxd challenge | head    # magic bytes

Common formats you’ll see:

Type	Signature	Notes
ELF (Linux)	7f 45 4c 46	x86-64 most common
PE (Windows)	4d 5a (MZ)	may need Wine
Python bytecode	.pyc	decompile with pycdc
Java class	ca fe ba be	decompile with cfr

Static vs dynamic analysis

Static — read the binary without running it. Safe, works on malware, no interaction needed. Tools: Ghidra, Binary Ninja, IDA.

Dynamic — run the binary under a debugger and watch execution. Lets you see decrypted data, evaluated branches, and actual flag checks. Tools: GDB, x64dbg, strace.

The real workflow alternates between both: static analysis tells you where to look, dynamic analysis shows you what actually happens.

First crackme workflow

1. file + strings — does the flag appear in plaintext? (Sometimes yes.)
2. Open in Ghidra. Find main. Read the control flow.
3. Find the comparison: strcmp, memcmp, a loop comparing byte-by-byte.
4. Trace backwards from the comparison to understand how the expected value is computed.
5. If the logic is simple, derive the answer statically. If obfuscated, use a debugger to break at the comparison and read the expected bytes from memory.

Reading disassembly

You do not need to understand every instruction. Focus on:

• cmp / test — the condition being checked
• je / jne / jz / jnz — which branch do you need to take?
• call — what function is called, and what does it return?
• mov rdi, rsi — argument passing (System V ABI: rdi, rsi, rdx …)

Ghidra’s decompiler view renders pseudocode on the right — start there and drop into disassembly only when the pseudocode is misleading.