Skip to content

API Hooking

API hooking is a process of intercepting and altering the behavior of API calls. This technique is commonly used by many Endpoint Detection and Response (EDR) or antivirus vendors to monitor processes or code execution in real-time for malicious activity.

The process of API hooking

API hooking occurs during the startup of a program when certain libraries/DLLs are loaded as modules into the address space of the corresponding user program.

API hooking

Step 1: When the program calls MessageBoxA(), it jumps to the function address.

Step 2: Insert a jump instruction (jmp) in MessageBoxA() to redirect it to our hook function.

Step 3: After executing the hook, it jumps to the trampoline function, which contains a copy of the original first few bytes of MessageBoxA(). This allows the original function's logic to continue.

Step 4: Once MessageBoxA() finishes executing, it returns to the user code to continue execution.

Microsoft Detours

Microsoft Detours is a software package for monitoring and intercepting API calls on Windows. It provides a general method for implementing x86 and x64 Windows API hooking, allowing for monitoring, tampering, or any other actions you wish to perform using API hooking. For more details, please refer to Detours.

Example:

The following code demonstrates how to use the Detours library to hook a function on the Windows platform and how to unhook it.

#include <windows.h>
#include <detours.h>
#include <iostream>

typedef BOOL(WINAPI* FuncMessageBoxA)(HWND, LPCSTR, LPCSTR, UINT);
FuncMessageBoxA pMessageBoxA = MessageBoxA;

BOOL WINAPI HookedMessageBoxA(HWND hWnd, LPCSTR lpText, LPCSTR lpCaption, UINT uType)
{
    std::cout << "Intercepted MessageBoxA called!" << std::endl;
    std::cout << "Text: " << lpText << std::endl;
    std::cout << "Caption: " << lpCaption << std::endl;
    BOOL result = pMessageBoxA(hWnd, "Hooked Function", lpCaption, uType);
    return result;
}
int main()
{
    DetourTransactionBegin();
    DetourUpdateThread(GetCurrentThread());
    DetourAttach(&(PVOID&)pMessageBoxA, HookedMessageBoxA);
    DetourTransactionCommit();
    // Hooked
    MessageBoxA(NULL, "Original MessageBox!", "Hooked MessageBoxA", MB_OK);
    getchar();

    /*
    DetourTransactionBegin();
    DetourUpdateThread(GetCurrentThread());
    DetourDetach(&(PVOID&)pMessageBoxA, HookedMessageBoxA);
    DetourTransactionCommit();
    MessageBoxA(NULL, "Really Original Messagebox!", "Original MessageBoxA", MB_OK);
    */

    return 0;
}

Hooking: Ex1 Cancel hooking: Ex2

EDR hook list

Antivirus software and Endpoint Detection and Response (EDR) platforms can also use behavior-based analysis to identify suspicious API activities. For a list of commonly used EDR hooks, you can refer to this curated EDR hook list.

EDR hook list

Tools

Resources