IOActive Security Advisory: Incorrect input validation in PyString_FromStringAndSize() leads to multiple buffer overflows

Title: Incorrect input validation in PyString_FromStringAndSize()
leads to multiple buffer overflows
Date Discoverd: ??-April-2008
Date Reported: 08-April-2008
Date Patched: 09-April-2008
Date Disclosed: 11-April-2008
Criticality: High

Affected Products

Python 2.5.2, earlier and unstable version are likely to be vulnerable

Synopsis

The Python core API provides multiple functions for the allocation
of string objects, specifically providing an API call that allows
for either the allocation or reallocation of a PyStringObject.
This function, PyString_FromStringAndSize() takes two parameters:
a pointer and a signed integer. If the pointer is non-NULL then
the memory pointed to it is reallocated to the size specified
by the second parameter. If the pointer is NULL then the number
of bytes specified by the integer are allocated and returned.

During the course of its operations this second parameter is not
validated to contain a positive value. This in turn is summed
with the size of a PyStringObject and passed as a length to an
allocation function, potentially missallocating memory.

The result of this was multiple buffer overflows in various
components such as the previously disclosed zlib bug, the SSL
module, et cetera. Furthermore, a Python developer, Alexander
Belopolsky noted that the functions PyBytes_FromStringAndSize()
and PyUnicode_FromStringAndSize() contained the same
characteristics.

Techical Details

Python-2.5.2/Objects/stringobject.c:

52 PyObject *
53 PyString_FromStringAndSize(const char *str, Py_ssize_t size)
54 {
55 register PyStringObject *op;
56 assert(size >= 0);
57 if (size == 0 && (op = nullstring) != NULL) {
[…]
63 }
64 if (size == 1 && str != NULL &&
65 (op = characters[str & UCHAR_MAX]) != NULL)
66 {
[…]
72 }
73
74 / Inline PyObject_NewVar */
75 op = (PyStringObject *)PyObject_MALLOC(sizeof(PyStringObject) +
size);

The type Py_ssize_t is defined to be one of a number of types dependant
on platform, however it regardless of platform it will be signed. There
is an assert() at line 56 that attempts to verify the sanity of the
second parameter however in non-debug builds the assert() is omitted.
Then at line 75 the size parameter and the size of a string object
are summed together and passed as a parameter to PyObject_MALLOC().

Reproduction / Proof-of-Concept

When the length variable contains a value of -24 then the allocator is
told to reserve 0 bytes of memory, however the allocator modifies
the request and will allocate one byte of memory. For values ranging
between -2 and -23 a small amount of memory will be allocated due
to being summed with the size of a PyStringObject. Because of this
being an API call, exploitation beyond that is dependant on the
caller and current environment.

Remediation

This bug was patched in CVS, patching all three object types.
Further details can be found at http://bugs.python.org/issue2587
and http://svn.python.org/view?rev=62271&view=rev and
http://svn.python.org/view?rev=62272&view=rev