Penetration Testing · 9 min read

Binary reverse engineering: when software needs a closer look.

Learn when to investigate compiled software, how static and runtime analysis complement each other, and what evidence should guide a decision to fix, restrict, or replace a component.

THE BRIEF

What you’ll take away

  • Start with a specific behavior question that examining the compiled software can help answer.
  • Tie conclusions to the exact files, configuration, observed behavior, and remaining uncertainty.
  • Translate the evidence into an owned decision to fix, restrict, request more evidence, or defer use.
Explore the guide
In this guide

Recognize the question hidden inside the software

A business can depend on software it cannot fully explain. A desktop tool still works, a plugin remains essential, or an acquired product arrives without reliable source code. The difficult question is whether a particular behavior creates an unacceptable exposure. Binary reverse engineering examines compiled software to help answer that question. Its value is the connection between implementation evidence and a decision someone needs to make.

Consider a fictional engineering firm that uses a legacy desktop component to import documents supplied by project partners. The component has limited documentation, and the current source code is unavailable to the firm. Before expanding its use, the IT owner wants to understand how it handles incoming files and whether importing a document initiates unexpected network activity. This example illustrates a review process; it does not describe an SPS customer or a discovered vulnerability.

Those are narrower, more useful questions than asking whether the entire product is secure. Identify the incoming material, the operation it triggers, and the consequence that matters. For this firm, the concern is a partner-supplied file reaching a workstation that can also access project storage. Leadership needs evidence to decide whether to keep the workflow, restrict it, request a vendor change, or move away from the component.

Choose the assessment that can answer your question

Reverse engineering is useful when the answer depends on behavior inside an executable or library and available documentation does not resolve it. It may clarify how selected inputs reach a sensitive operation, where an important check occurs, or what changed between two supplied builds. A broad vendor questionnaire cannot establish those details. Equally, a close examination of one library cannot establish how well the vendor operates its entire development and support program.

For the fictional firm, start by separating three decisions. Procurement needs to understand the vendor’s maintenance and support commitments. IT needs to understand how the product connects to the environment. The technical reviewer needs to examine the import component’s selected behavior. A vendor assessment addresses the wider product and integration picture; binary analysis investigates the implementation questions that remain. These assessments can complement each other without treating either report as complete coverage of the other.

Check feasibility before committing to a deep investigation. The analyst needs to identify the file format, processor architecture, dependencies, and available test setup. Missing libraries, inaccessible services, or protections that obstruct analysis can limit the answer. If the real concern is a device’s wireless interface or physical access, broader device testing may be needed. If current source becomes available, reviewing it alongside the shipped binary may make the investigation more efficient.

Explore Vendor & Third-Party Software Assessment

Establish exactly which software the evidence describes

A product name and a screenshot of its version number are weak foundations for a technical conclusion. Record the actual executable and library files, their versions, cryptographic hashes, origin, and collection date. Include the operating environment and configuration used during observation. Keep the reviewed originals protected and conduct analysis on controlled working copies. This gives the team a way to connect a finding to the software it actually reviewed.

NIST’s Secure Software Development Framework describes providing release-integrity information, with protected published hashes and code signing among its examples. [1] For a review, distinguish a locally recorded fingerprint from verification against a trusted release reference. The first helps identify the reviewed bytes. The second can help check whether those bytes match the expected release. Neither establishes that the program behaves securely or is suitable for your intended workflow.

The fictional firm should confirm which import library the desktop application actually loads. An installer package, an archived support attachment, and the file currently deployed may differ. Record supporting configuration and sample-document characteristics without copying unnecessary project content. If the vendor supplies a replacement during the engagement, preserve the distinction between the original and replacement builds. Otherwise, an apparently conflicting result may simply describe a different file or setting rather than a changed interpretation.

Use static analysis and runtime observation together

Static analysis examines the file without executing the target program. Disassembly exposes machine instructions; decompilation presents an interpretation in a more readable, source-like form. Tools such as Ghidra provide these capabilities and ways to follow relationships through compiled code. [2] In the import example, the analyst can investigate selected paths from document handling to file access or network operations, then identify which interpretations need additional checking.

A decompiled view is not a recovered copy of the original source. Names, types, and control flow can require interpretation. Ghidra’s advanced training documents analysis assumptions and cases where incorrect function information or incomplete flow recovery affects the result. [3] A convincing-looking function is therefore evidence to examine, not a conclusion to accept automatically. The report should explain which important interpretations were checked against the underlying instructions or other observations.

Runtime observation examines what happens when the approved component executes under specified conditions. Ghidra’s debugger training treats static analysis as part of debugging and cautions against running unfamiliar or untrusted code without precautions. [4] For the fictional firm, an agreed isolated setup can help investigate which files the component touches and whether it attempts a connection during selected imports. Record the inputs, permissions, configuration, and observations that make the result meaningful.

The two methods answer different parts of the question. A network-related function in the binary does not prove that a normal import reaches it. No connection during one observed run does not prove that another input or setting cannot trigger one. If the component cannot run because a dependency is missing, say so. Static evidence may still support useful conclusions, while runtime-dependent questions remain unresolved. The buyer should see that boundary clearly.

01 / COMBINE METHODS WITH CARE

Two ways to examine the same software.

Authorized file + a focused question
STATIC INSPECTIONRead the compiled structure.

Inspect selected functions and data handling without execution.

LIMIT

Packing and missing context can constrain interpretation.

CONTROLLED RUNTIMEObserve agreed behavior.

Use an authorized, isolated setup where execution is feasible.

LIMIT

Only exercised paths and test conditions are observed.

Check the evidence together.

Cross-check interpretations and report unresolved gaps.

Methods support a bounded analysis question. Neither promises complete coverage or recovery of the exact original source code.

Protect the investigation as well as the business

Confirm authorization for the supplied software and the proposed analysis before handing over files. Agree where artifacts and derived material may be stored, who may access them, and whether anything may leave the approved environment. NIST SP 800-115 provides a foundation for assessment plans, authorization, rules of engagement, and handling assessment data. [5] Apply those principles to the actual investigation rather than assuming that possession of a file settles every permission question.

For the fictional firm, use representative synthetic documents wherever they can answer the question. Keep production credentials and ordinary project shares outside the test environment. Agree how outbound attempts will be contained and observed, including behavior that depends on an unavailable service. A blocked connection is an observation under that restriction; it does not reveal everything that would have happened if the remote service had responded.

Include analysis tools and assistance in the handling agreement. Decompiled material, strings, logs, and screenshots may contain sensitive information even when the original documents are excluded. Decide whether external AI services are permitted before using them on client material. An automated explanation can guide further investigation, but the responsible analyst must check its claims against the binary or observed behavior. It should not turn an uncertain interpretation into a confirmed finding.

Related reading: a better penetration test starts with a better scope

Require an evidence trail that another engineer can follow

An actionable finding connects the exact artifact to the relevant location or function, the conditions needed to reach the behavior, and the observed or supported consequence. Ask for enough detail that the software owner can investigate and a reviewer can understand the reasoning. The technical attachment may contain addresses, annotated functions, and controlled reproduction information. The leadership summary should explain why the behavior matters to the approved business use.

Keep observed behavior, supported interpretation, and unresolved hypotheses distinct. An analyst may observe a process making an outbound attempt, infer which function initiated it, and still lack evidence about the intended remote exchange. Those are different statements. Similarly, a crash is evidence of a failure under specific conditions; it does not alone establish arbitrary code execution or the full business impact. Confidence should follow the evidence actually obtained.

In the fictional example, an unexpected destination would prompt several follow-up questions: which process initiated the attempt, what triggered it, what information was observed leaving, and whether configuration or documentation explains it. Do not label every unfamiliar connection malicious. Conversely, a plausible explanation from a vendor needs to match the reviewed build and conditions. Record unexamined document types, unavailable dependencies, and paths that could not be resolved so silence in the report is not mistaken for a favorable result.

02 / KEEP THE CONCLUSION TRACEABLE

An observation needs a path to evidence.

Illustrative review of a fictional legacy document-import component.

  1. File, version, fingerprint

    Identify the exact artifact reviewed.

  2. Located observation

    Point to a function, address, or captured behavior.

  3. Validation + context

    Record conditions, reproducibility, and impact.

  4. Supported conclusion

    State the evidence, confidence, and coverage limits.

Evidence missing? Keep the hypothesis open.

State what remains unknown and what would resolve it.

A tool flag or plausible theory does not confirm a vulnerability. Report observed behavior, supported inference, and unresolved hypotheses separately.

Turn the report into a specific operating decision

A useful conclusion tells the owner what to do with the evidence. A vendor fix may be appropriate when the review establishes an unsafe behavior in a maintained component. Restricting use may be appropriate while a correction is evaluated. More evidence may be needed when the relevant path remains uncertain. Deferring the rollout or considering replacement may be the sound choice when an important question cannot be resolved within the available support and operating constraints.

For the fictional firm, a temporary restriction might limit who can import documents, separate the import workflow from broader project access, or constrain unnecessary outbound connectivity. Each option needs evaluation against the actual exposure and business workflow. Isolation does not repair a defect, and a network restriction does not address every local processing risk. Name the owner who will implement the measure, verify that it works as intended, and decide when it must be reviewed.

Record the decision conditions alongside the technical conclusion. Which build and configuration are accepted? Which users or document sources are permitted? Which unresolved questions remain, and what event reopens the decision? Avoid compressing those answers into a universal “safe” label. A bounded review can support a conditional operating choice while still leaving unexamined code and future changes outside its conclusions. Leadership should approve the remaining exposure with those limits visible.

03 / USE FINDINGS TO MAKE A DECISION

Match the next step to the evidence.

SCOPED REVIEWFindings + coverage limitations
Fix

Work with the vendor; retest the updated build.

Restrict

Limit features or isolate the component.

Request evidence

Resolve the unknowns that affect the decision.

Defer

Delay adoption or consider replacement.

Name an owner and a review trigger.

Revisit the decision after a new build, changed use, or changed exposure.

A scoped review supports a defined business decision. It is not a certification or a guarantee for unexamined behavior.

Evaluate the replacement build against the original question

When a vendor provides an update, collect the new artifact identity and a description of the intended change. Revisit the original observation under the agreed conditions and examine the relevant changed behavior. A different hash confirms that the bytes differ; it does not explain the change or establish that the reported issue is resolved. A release note is useful context, while verification needs evidence tied to the supplied build.

For the import component, preserve representative test inputs and the environment details needed to compare behavior. Confirm that legitimate imports still work within the agreed verification scope. If the vendor changes the parser, adds a service dependency, or introduces a new network route, assess whether that change needs additional investigation. Retesting an original finding should not silently become an assurance statement about every feature in the new release.

Keep the result understandable: the original issue was verified fixed under the tested conditions, remains partly addressed, remains open, or could not be retested. Give unresolved items an owner and next decision point. A technical fix also does not establish whether earlier misuse occurred. If there is evidence of an actual incident, address that through a separately authorized response process with the appropriate evidence and expertise.

Related reading: turn findings into verified fixes

Buy a useful answer and a usable handoff

Before commissioning the work, describe the decision that is waiting, supply the approved files, and identify the engineer who can explain the surrounding workflow. Ask what the provider can examine with the available access, how uncertain interpretations will be reported, and what evidence you will retain. An annotated behavior map is valuable when it helps your team find and understand the relevant code; volume of decompiled output is not a measure of assurance.

SPS Binary Reverse Engineering investigates authorized executables, libraries, and supplied firmware images through selected behavior analysis. The engagement confirms supported formats and handling conditions, starts with static analysis, and uses agreed isolated execution where feasible. Deliverables include a reviewed file inventory, an annotated function and behavior map, evidence-backed findings, documented uncertainty, and remediation guidance. A technical walkthrough and scoped retest of remediated findings support the handoff when an updated binary and test setup are supplied.

For the fictional firm, the useful end product is a defensible decision about its import workflow and a clear explanation for the software owner. Exact original source recovery, exhaustive coverage, and a guaranteed vulnerability discovery are not the promise. The work should reduce a specific uncertainty enough to guide the next responsible action.

Explore Binary Reverse Engineering

See what useful reporting looks like. Our illustrative sample pentest report shows scope, evidence, risk, remediation, and verification using a fictional environment.

YOUR NEXT STEP

Have a software behavior you need to understand?

Tell us which executable, library, or supplied firmware component is involved, what source or documentation is available, and which business decision depends on the answer. We will help define a focused, authorized investigation.

Discuss a binary review

References & further reading

  1. NIST SP 800-218: Secure Software Development Framework, Version 1.1 — release integrity (PS.2)
  2. NSA: Ghidra Software Reverse Engineering Framework
  3. Ghidra: Improving Disassembly and Decompilation
  4. Ghidra Debugger: Getting Started
  5. NIST SP 800-115: Technical Guide to Information Security Testing and Assessment

Share this guide on LinkedIn