Strange Behavior - Encoders

[huntergregal]

First - love pwntools and use it all the time for CTFs. Thanks for all the hardwork!

Multiple issues when working with shellcode encoders:

1. Many encoders flat out fail to alter shellcode bytes with no raised exception or notification to the user. At the very least should return "None" or something. Silent failures make debugging in a ctf hard. It seems like from the code that encoder failures should log an error and raise it (https://github.com/Gallopsled/pwntools/blob/dev/pwnlib/encoders/encoder.py#L105). Most of the time this doesn't happen which hints at error detection failure in the encoder code. Have not dug into it yet. Notice in attached poc.py below many encoders fail silently. (hopefully my checking is proper....)
2. On the off chance the encoder detects that it failed to encode the shellcode it will raise an exception. The exception itself is a bit ugly and contains color encoded characters - but the real issue is that it does not properly set a message attribute which results in an exception being raised when calling repr on the initial exception. See "Weird behavior 2" in attached poc.py ---- EDIT: addressed in add self.message and change sys.exc_type to sys.exec_info() in Pwnlib… #1876
3. doing encode(sc, avoid=b'\x01') on certain amd64 shellcode randomly raises an exception. See "Weird behavior 1" in my PoC/test code below.

poc.py

#!/usr/bin/env python3
from pwn import *
import pkg_resources, platform
import string
# -- system info
import pkg_resources, platform, subprocess

# -- debugging
#context.log_level = 'debug'
#from IPython import embed;
#repl = lambda : embed(colors='linux')

info(' === system info ===')
ver = pkg_resources.get_distribution('pwntools').version
info(f'python version: {platform.python_version()}')
info(f'pwntools version: {ver}')
uname = platform.os.uname()
info(f'System Release: {uname.release}')
info(f'System Version: {uname.version}')
lsb_release = subprocess.check_output("""sh -c '. /etc/os-release; echo "$PRETTY_NAME"'""",
                    shell=True,universal_newlines=True).strip()
info(f'LSB Release: {lsb_release}')

def hasWhitespace(sc):
    blacklist = string.whitespace.encode()
    if len([c for c in sc if c in blacklist]) > 0:
        return True
    else:
        return False

def isPrintable(sc):
    whitelist = string.printable.encode()
    if len([c for c in sc if c not in whitelist]) > 0:
        return False
    else:
        return True
    
def check(res):
    if res:
        success('      -> PASS')
    else:
        warn('      -> FAIL')

def test(sc):
    info('  -> testing encoder.alphanumeric:')
    try:
        enc = encoders.encoder.alphanumeric(sc)
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(enc.isalpha())

    info('  -> testing encoder.alphanumeric (force=True):')
    try:
        enc = encoders.encoder.alphanumeric(sc, force=True)
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(enc.isalpha())

    info('  -> testing encoder.null:')
    try:
        enc = encoders.encoder.null(sc)
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(b'\0' not in enc)

    info('  -> testing encoder.null (force=True):')
    try:
        enc = encoders.encoder.null(sc, force=True)
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(b'\0' not in enc)

    info('  -> testing encoder.line:')
    try:
        enc = encoders.encoder.line(sc)
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(isPrintable(sc) and not hasWhitespace(enc))

    info('  -> testing encoder.line (force=True):')
    try:
        enc = encoders.encoder.line(sc, force=True)
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(isPrintable(sc) and not hasWhitespace(enc))

    info('  -> testing encode(sc, avoid=b"\\x01"):')
    try:
        enc = encode(sc, avoid=b'\x01')
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(b'\x01' not in enc)

    info('  -> testing encode(sc, avoid=b"\\x00"):')
    try:
        enc = encode(sc, avoid=b'\x00')
    except Exception as e:
        warn(f'Failed to run encoder!!! - {e}')
    debug(hexdump(enc))
    check(b'\0' not in enc)
    

info('\n=== Testing amd64 ===')
context.arch = 'amd64'
context.bits = 64
info('shellcode asm("cat /flag") + asm(memcpy 0xdeadbeef, 0x0a00babe, 64) + asm(sh()):')
try:
    sc = asm(shellcraft.amd64.cat('/flag'))
    sc += asm(shellcraft.amd64.memcpy(0xdeadbeef, 0x0a00babe, 64))
    sc += asm(shellcraft.amd64.sh())
    debug(hexdump(sc))
    test(sc)
except Exception as e:
    warn(r'Failed to build shellcode!!! - {e}')

info('\n=== Testing arm ===')
context.arch = 'arm'
context.bits = 32
info('shellcode asm("cat /flag") + asm(memcpy 0xdeadbeef, 0x0a00babe, 64) + asm(sh()):')
try:
    sc = asm(shellcraft.arm.cat('/flag'))
    sc += asm(shellcraft.arm.memcpy(0xdeadbeef, 0x0a00babe, 64))
    sc += asm(shellcraft.arm.sh())
    debug(hexdump(sc))
    test(sc)
except Exception as e:
    warn(f'Failed to build shellcode!!! - {e}')

info('\n=== Testing aarch64 ===')
context.arch = 'aarch64'
context.bits = 64
info('shellcode asm("cat /flag") + asm(memcpy 0xdeadbeef, 0x0a00babe, 64) + asm(sh()):')
try:
    sc = asm(shellcraft.aarch64.cat('/flag'))
    sc += asm(shellcraft.aarch64.memcpy(0xdeadbeef, 0x0a00babe, 64))
    sc += asm(shellcraft.aarch64.sh())
    debug(hexdump(sc))
    test(sc)
except Exception as e:
    warn(f'Failed to build shellcode!!! - {e}')

# Demonstrate another weird issue
info('\n=== Wierd behavior 1 - inconsistent exception===')
context.arch = 'amd64'
context.bits = 64
sc = asm(shellcraft.amd64.cat('/flag'))
sc += asm(shellcraft.amd64.memcpy(0xdeadbeef, 0x0a00babe, 64))
sc += asm(shellcraft.amd64.sh())
info('amd64 shellcode: encode(sc, avoid=b"\\x01)"')
info('Encoder will randomly raise exception -- run 10 times')
for i in range(0,10):
    try:
        encode(sc, avoid=b'\x01')
        success(f'  -> #{i} PASS')
    except Exception as e:
        warn(f'  -> #{i} FAIL W/ {e}')



info('\n=== Wierd behavior 2 - raised encoder exceptions have no "message" attribute ===')
context.arch = 'aarch64'
context.bits = 64
sc = asm(shellcraft.aarch64.cat('/flag'))
sc += asm(shellcraft.aarch64.memcpy(0xdeadbeef, 0x0a00babe, 64))
sc += asm(shellcraft.aarch64.sh())
try:
    enc = encoders.encoder.alphanumeric(sc, force=True)
except Exception as e:
    try:
        error_string = repr(e)
    except Exception as e:
        warn(f'caught another exception while handling encoder exception:\n"{e}"')

poc.py Output

[*]  === system info ===
[*] python version: 3.8.5
[*] pwntools version: 4.5.0
[*] System Release: 5.8.0-50-generic
[*] System Version: #56~20.04.1-Ubuntu SMP Mon Apr 12 21:46:35 UTC 2021
[*] LSB Release: Ubuntu 20.04.2 LTS
[*] 
    === Testing amd64 ===
[*] shellcode asm("cat /flag") + asm(memcpy 0xdeadbeef, 0x0a00babe, 64) + asm(sh()):
[*]   -> testing encoder.alphanumeric:
[!]       -> FAIL
[*]   -> testing encoder.alphanumeric (force=True):
[!]       -> FAIL
[*]   -> testing encoder.null:
[+]       -> PASS
[*]   -> testing encoder.null (force=True):
[!]       -> FAIL
[*]   -> testing encoder.line:
[!]       -> FAIL
[*]   -> testing encoder.line (force=True):
[!]       -> FAIL
[*]   -> testing encode(sc, avoid=b"\x01"):
[!] Encoder <pwnlib.encoders.amd64.delta.amd64DeltaEncoder object at 0x7fd818ec4640> did not succeed
[!] Failed to run encoder!!! - sequence item 0: expected str instance, int found
[+]       -> PASS
[*]   -> testing encode(sc, avoid=b"\x00"):
[+]       -> PASS
[*] 
    === Testing arm ===
[*] shellcode asm("cat /flag") + asm(memcpy 0xdeadbeef, 0x0a00babe, 64) + asm(sh()):
[*]   -> testing encoder.alphanumeric:
[!]       -> FAIL
[*]   -> testing encoder.alphanumeric (force=True):
[!]       -> FAIL
[*]   -> testing encoder.null:
[!]       -> FAIL
[*]   -> testing encoder.null (force=True):
[+]       -> PASS
[*]   -> testing encoder.line:
[!]       -> FAIL
[*]   -> testing encoder.line (force=True):
[!]       -> FAIL
[*]   -> testing encode(sc, avoid=b"\x01"):
[+]       -> PASS
[*]   -> testing encode(sc, avoid=b"\x00"):
[+]       -> PASS
[*] 
    === Testing aarch64 ===
[*] shellcode asm("cat /flag") + asm(memcpy 0xdeadbeef, 0x0a00babe, 64) + asm(sh()):
[*]   -> testing encoder.alphanumeric:
[!]       -> FAIL
[*]   -> testing encoder.alphanumeric (force=True):
[ERROR] No encoders for aarch64 which can avoid '[^A-Za-z0-9]' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!] Failed to run encoder!!! - No encoders for aarch64 which can avoid '[^A-Za-z0-9]' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!]       -> FAIL
[*]   -> testing encoder.null:
[!]       -> FAIL
[*]   -> testing encoder.null (force=True):
[ERROR] No encoders for aarch64 which can avoid '\\x00' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!] Failed to run encoder!!! - No encoders for aarch64 which can avoid '\\x00' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!]       -> FAIL
[*]   -> testing encoder.line:
[!]       -> FAIL
[*]   -> testing encoder.line (force=True):
[ERROR] No encoders for aarch64 which can avoid '\\s' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!] Failed to run encoder!!! - No encoders for aarch64 which can avoid '\\s' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!]       -> FAIL
[*]   -> testing encode(sc, avoid=b"\x01"):
[!] Failed to run encoder!!! - sequence item 0: expected str instance, int found
[!]       -> FAIL
[*]   -> testing encode(sc, avoid=b"\x00"):
[!] Failed to run encoder!!! - sequence item 0: expected str instance, int found
[!]       -> FAIL
[*] 
    === Wierd behavior 1 - inconsistent exception===
[*] amd64 shellcode: encode(sc, avoid=b"\x01)"
[*] Encoder will randomly raise exception -- run 10 times
[!]   -> #0 FAIL W/ sequence item 0: expected str instance, int found
[!]   -> #1 FAIL W/ sequence item 0: expected str instance, int found
[!]   -> #2 FAIL W/ sequence item 0: expected str instance, int found
[!]   -> #3 FAIL W/ sequence item 0: expected str instance, int found
[!]   -> #4 FAIL W/ sequence item 0: expected str instance, int found
[+]   -> #5 PASS
[!]   -> #6 FAIL W/ sequence item 0: expected str instance, int found
[+]   -> #7 PASS
[!]   -> #8 FAIL W/ sequence item 0: expected str instance, int found
[!]   -> #9 FAIL W/ sequence item 0: expected str instance, int found
[*] 
    === Wierd behavior 2 - raised encoder exceptions have no "message" attribute ===
[ERROR] No encoders for aarch64 which can avoid '[^A-Za-z0-9]' for
    00000000  ee c5 8c d2  8e 2d ac f2  ee 0c c0 f2  ee 0f 1f f8  │····│·-··│····│····│
    00000010  80 f3 9f d2  e0 ff bf f2  e0 ff df f2  e0 ff ff f2  │····│····│····│····│
    00000020  e1 03 00 91  e2 03 1f aa  e3 03 1f aa  08 07 80 d2  │····│····│····│····│
    00000030  01 00 00 d4  e1 03 00 aa  20 00 80 d2  e2 03 1f aa  │····│····│ ···│····│
    00000040  e3 ff 9f d2  e3 ff af f2  e8 08 80 d2  01 00 00 d4  │····│····│····│····│
    00000050  e0 dd 97 d2  a0 d5 bb f2  c1 57 97 d2  01 40 a1 f2  │····│····│·W··│·@··│
    00000060  02 08 80 d2  23 14 40 38  03 14 00 38  42 04 00 f1  │····│#·@8│···8│B···│
    00000070  aa ff ff 54  ee 45 8c d2  2e cd ad f2  ee e5 c5 f2  │···T│·E··│.···│····│
    00000080  ee 65 ee f2  0f 0d 80 d2  ee 3f bf a9  e0 03 00 91  │·e··│····│·?··│····│
    00000090  e1 03 1f aa  e2 03 1f aa  a8 1b 80 d2  01 00 00 d4  │····│····│····│····│
    000000a0
[!] caught another exception while handling encoder exception:
    "'PwnlibException' object has no attribute 'message'"

[Arusekk]

Your test case is too big; encoders for every arch are separate beings, please split.

[huntergregal]

As for the issue where encode fails with:
Failed to run encoder!!! - sequence item 0: expected str instance, int found

i think the culprit may be when avoid=str vs avoid=bytes.

It seems the Encode class does not handle when avoid is bytes very well. Tweaking https://github.com/Gallopsled/pwntools/blob/dev/pwnlib/encoders/encoder.py#L100 like so seemed to the issue:

100c100
<         avoid_errmsg = ''.join(avoid)
---
>         avoid_errmsg = ''.join(repr(avoid))

But I'm hesitant to submit a PR with this as maybe the right solution is to enforce avoid == str ??

EDIT: upon looking at the encoders most of them actually seem to expect avoid to be bytes. The arm alphanumeric encoder seems to ignore the avoid arg completely though. I suppose the avoid here is implied.

I'll go ahead and submit a PR for this change as it appears avoid should in fact be bytes and not a string. It doesn't appear to be an issue if a string is used instead - however. this patch will support both in the error message

[Arusekk]

Encoders have never been fully ported to python 3 (#529 #1583 #1767 #1761), and were probably broken in the meantime. If you have working small tests reproducing the issues, preferably something as simple as (for polymorphic encoders, setting a random seed just before the test looks reasonable, too)

>>> encoders.aarch64.encode(asm(shellcraft.dupsh()))  # not even correct, but you get the idea
b'AAASDGHASDBHNPCIiohurnHSDOqnwekPPPPPn'

please add a WIP PR adding the tests in documentation of the tested encoders. Like tests for aarch64 encoder named foobar into files under pwnlib/shellcraft/encoders/aarch64/foobar*

EDIT: if you want to cast text strings to bytes anywhere in encoders, use packing._need_bytes, as this will inform the user of their terrible mistake

[huntergregal]

Your test case is too big; encoders for every arch are separate beings, please split.

oh for sure. was mostly to highlight some of the inconsistencies and less to be used as an actual test case. At a later date I'll try to build some small and sane tests for the individual encoders under docs.