[Python-3000-checkins] r66038 - in python/branches/py3k: Doc/c-api/bytes.rst Doc/c-api/object.rst Include/bytesobject.h Include/object.h Lib/test/test_bytes.py Misc/NEWS Objects/bytesobject.c Objects/object.c

Discussion:

benjamin.peterson

2008-08-26 16:46:48 UTC

Permalink

Author: benjamin.peterson
Date: Tue Aug 26 18:46:47 2008
New Revision: 66038

Log:
make bytes(o) respect __bytes__ #2415

This adds two new C-API functions: PyObject_Bytes and PyBytes_FromObject.

Reviewer: Barry

Modified:
python/branches/py3k/Doc/c-api/bytes.rst
python/branches/py3k/Doc/c-api/object.rst
python/branches/py3k/Include/bytesobject.h
python/branches/py3k/Include/object.h
python/branches/py3k/Lib/test/test_bytes.py
python/branches/py3k/Misc/NEWS
python/branches/py3k/Objects/bytesobject.c
python/branches/py3k/Objects/object.c

Modified: python/branches/py3k/Doc/c-api/bytes.rst
==============================================================================
--- python/branches/py3k/Doc/c-api/bytes.rst (original)
+++ python/branches/py3k/Doc/c-api/bytes.rst Tue Aug 26 18:46:47 2008
@@ -118,6 +118,12 @@
arguments.

+.. cfunction:: PyObject* PyBytes_FromObject(PyObject *o)
+
+ Return the bytes representation of object *o* that implements the buffer
+ protocol.
+
+
.. cfunction:: Py_ssize_t PyBytes_Size(PyObject *o)

Return the length of the bytes in bytes object *o*.

Modified: python/branches/py3k/Doc/c-api/object.rst
==============================================================================
--- python/branches/py3k/Doc/c-api/object.rst (original)
+++ python/branches/py3k/Doc/c-api/object.rst Tue Aug 26 18:46:47 2008
@@ -139,6 +139,14 @@
Python expression ``str(o)``. Called by the :func:`str` built-in function
and, therefore, by the :func:`print` function.

+.. cfunction:: PyObject* PyObject_Bytes(PyObject *o)
+
+ .. index:: builtin: bytes
+
+ Compute a bytes representation of object *o*. *NULL* is returned on failure
+ and a bytes object on success. This is equivalent to the Python expression
+ ``bytes(o)``.
+

.. cfunction:: int PyObject_IsInstance(PyObject *inst, PyObject *cls)

Modified: python/branches/py3k/Include/bytesobject.h
==============================================================================
--- python/branches/py3k/Include/bytesobject.h (original)
+++ python/branches/py3k/Include/bytesobject.h Tue Aug 26 18:46:47 2008
@@ -48,6 +48,7 @@

PyAPI_FUNC(PyObject *) PyBytes_FromStringAndSize(const char *, Py_ssize_t);
PyAPI_FUNC(PyObject *) PyBytes_FromString(const char *);
+PyAPI_FUNC(PyObject *) PyBytes_FromObject(PyObject *);
PyAPI_FUNC(PyObject *) PyBytes_FromFormatV(const char*, va_list)
Py_GCC_ATTRIBUTE((format(printf, 1, 0)));
PyAPI_FUNC(PyObject *) PyBytes_FromFormat(const char*, ...)

Modified: python/branches/py3k/Include/object.h
==============================================================================
--- python/branches/py3k/Include/object.h (original)
+++ python/branches/py3k/Include/object.h Tue Aug 26 18:46:47 2008
@@ -423,6 +423,7 @@
PyAPI_FUNC(PyObject *) PyObject_Repr(PyObject *);
PyAPI_FUNC(PyObject *) PyObject_Str(PyObject *);
PyAPI_FUNC(PyObject *) PyObject_ASCII(PyObject *);
+PyAPI_FUNC(PyObject *) PyObject_Bytes(PyObject *);
PyAPI_FUNC(int) PyObject_Compare(PyObject *, PyObject *);
PyAPI_FUNC(PyObject *) PyObject_RichCompare(PyObject *, PyObject *, int);
PyAPI_FUNC(int) PyObject_RichCompareBool(PyObject *, PyObject *, int);

Modified: python/branches/py3k/Lib/test/test_bytes.py
==============================================================================
--- python/branches/py3k/Lib/test/test_bytes.py (original)
+++ python/branches/py3k/Lib/test/test_bytes.py Tue Aug 26 18:46:47 2008
@@ -458,6 +458,18 @@
with open(fd, "rb", buffering=0) as f:
self.assertRaises(TypeError, f.readinto, b"")

+ def test_custom(self):
+ class A:
+ def __bytes__(self):
+ return b'abc'
+ self.assertEqual(bytes(A()), b'abc')
+ class A: pass
+ self.assertRaises(TypeError, bytes, A())
+ class A:
+ def __bytes__(self):
+ return None
+ self.assertRaises(TypeError, bytes, A())
+

class ByteArrayTest(BaseBytesTest):
type2test = bytearray

Modified: python/branches/py3k/Misc/NEWS
==============================================================================
--- python/branches/py3k/Misc/NEWS (original)
+++ python/branches/py3k/Misc/NEWS Tue Aug 26 18:46:47 2008
@@ -27,6 +27,13 @@

- Issue #3650: Fixed a reference leak in bytes.split('x').

+- bytes(o) now tries to use o.__bytes__() before using fallbacks.
+
+C API
+-----
+
+- PyObject_Bytes and PyBytes_FromObject were added.
+
Library
-------

Modified: python/branches/py3k/Objects/bytesobject.c
==============================================================================
--- python/branches/py3k/Objects/bytesobject.c (original)
+++ python/branches/py3k/Objects/bytesobject.c Tue Aug 26 18:46:47 2008
@@ -2882,11 +2882,10 @@
static PyObject *
string_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
- PyObject *x = NULL, *it;
+ PyObject *x = NULL;
const char *encoding = NULL;
const char *errors = NULL;
PyObject *new = NULL;
- Py_ssize_t i, size;
static char *kwlist[] = {"source", "encoding", "errors", 0};

if (type != &PyBytes_Type)
@@ -2924,6 +2923,14 @@
"encoding or errors without a string argument");
return NULL;
}
+ return PyObject_Bytes(x);
+}
+
+PyObject *
+PyBytes_FromObject(PyObject *x)
+{
+ PyObject *new, *it;
+ Py_ssize_t i, size;

/* Is it an int? */
size = PyNumber_AsSsize_t(x, PyExc_ValueError);

Modified: python/branches/py3k/Objects/object.c
==============================================================================
--- python/branches/py3k/Objects/object.c (original)
+++ python/branches/py3k/Objects/object.c Tue Aug 26 18:46:47 2008
@@ -453,6 +453,45 @@
return res;
}

+PyObject *
+PyObject_Bytes(PyObject *v)
+{
+ PyObject *bytesmeth, *result, *func;
+ static PyObject *bytesstring = NULL;
+
+ if (bytesstring == NULL) {
+ bytesstring = PyUnicode_InternFromString("__bytes__");
+ if (bytesstring == NULL)
+ return NULL;
+ }
+
+ if (v == NULL)
+ return PyBytes_FromString("<NULL>");
+
+ if (PyBytes_CheckExact(v)) {
+ Py_INCREF(v);
+ return v;
+ }
+
+ /* Doesn't create a reference */
+ func = _PyType_Lookup(Py_TYPE(v), bytesstring);
+ if (func != NULL) {
+ result = PyObject_CallFunctionObjArgs(func, v, NULL);
+ if (result == NULL)
+ return NULL;
+ if (!PyBytes_Check(result)) {
+ PyErr_Format(PyExc_TypeError,
+ "__bytes__ returned non-bytes (type %.200s)",
+ Py_TYPE(result)->tp_name);
+ Py_DECREF(result);
+ return NULL;
+ }
+ return result;
+ }
+ PyErr_Clear();
+ return PyBytes_FromObject(v);
+}
+
/* The new comparison philosophy is: we completely separate three-way
comparison from rich comparison. That is, PyObject_Compare() and
PyObject_Cmp() *just* use the tp_compare slot. And PyObject_RichCompare()

Neal Norwitz

2008-08-26 23:53:52 UTC

Permalink

On Tue, Aug 26, 2008 at 9:46 AM, benjamin.peterson

Post by benjamin.peterson
Modified: python/branches/py3k/Objects/bytesobject.c
==============================================================================
--- python/branches/py3k/Objects/bytesobject.c (original)
+++ python/branches/py3k/Objects/bytesobject.c Tue Aug 26 18:46:47 2008
@@ -2924,6 +2923,14 @@
"encoding or errors without a string argument");
return NULL;
}
+ return PyObject_Bytes(x);
+}
+
+PyObject *
+PyBytes_FromObject(PyObject *x)
+{
+ PyObject *new, *it;
+ Py_ssize_t i, size;
/* Is it an int? */
size = PyNumber_AsSsize_t(x, PyExc_ValueError);

What happens if x (the object passed in) is NULL?

n

Benjamin Peterson

2008-08-27 00:02:21 UTC

Permalink

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 9:46 AM, benjamin.peterson

What happens if x (the object passed in) is NULL?

It segfaults, but this is ok because PyObject_Bytes checks for NULL.

Post by Neal Norwitz
n
_______________________________________________
Python-3000-checkins mailing list
Python-3000-checkins at python.org
http://mail.python.org/mailman/listinfo/python-3000-checkins

--
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

Neal Norwitz

2008-08-27 00:04:36 UTC

Permalink

On Tue, Aug 26, 2008 at 5:02 PM, Benjamin Peterson

Post by Benjamin Peterson

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 9:46 AM, benjamin.peterson

What happens if x (the object passed in) is NULL?

It segfaults, but this is ok because PyObject_Bytes checks for NULL.

Since this is now a public API, anyone can pass NULL here. In general
we check for NULLs from most public APIs (I think). Though IIRC we
are pretty inconsistent.

n

Benjamin Peterson

2008-08-27 00:13:03 UTC

Permalink

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 5:02 PM, Benjamin Peterson

Post by Benjamin Peterson

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 9:46 AM, benjamin.peterson

What happens if x (the object passed in) is NULL?

It segfaults, but this is ok because PyObject_Bytes checks for NULL.

Since this is now a public API, anyone can pass NULL here. In general
we check for NULLs from most public APIs (I think). Though IIRC we
are pretty inconsistent.

I wasn't under that impression. For example, a quick glance shows that
none of the PyList API functions check for NULL.

Anyway, what should PyBytes_FromObject return for NULL anyway?
Probably just another NULL.

Post by Neal Norwitz
n

--
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

Neal Norwitz

2008-08-27 00:25:12 UTC

Permalink

On Tue, Aug 26, 2008 at 5:13 PM, Benjamin Peterson

Post by Benjamin Peterson

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 5:02 PM, Benjamin Peterson

Post by Benjamin Peterson

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 9:46 AM, benjamin.peterson

What happens if x (the object passed in) is NULL?

It segfaults, but this is ok because PyObject_Bytes checks for NULL.

Since this is now a public API, anyone can pass NULL here. In general
we check for NULLs from most public APIs (I think). Though IIRC we
are pretty inconsistent.

I wasn't under that impression. For example, a quick glance shows that
none of the PyList API functions check for NULL.

Like I said, we're inconsistent. :-) We discussed this on python-dev
a while ago, but there wasn't really any resolution IIRC.

I generally prefer to be safe which is why I prefer to have checks
unless there's a good reason not to.

Post by Benjamin Peterson
Anyway, what should PyBytes_FromObject return for NULL anyway?
Probably just another NULL.

Objects/dictobject.c has good examples. Something like:

if (x == NULL) {
PyErr_BadInternalCall();
return NULL;
}

n

Benjamin Peterson

2008-08-27 00:28:53 UTC

Permalink

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 5:13 PM, Benjamin Peterson

Post by Benjamin Peterson

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 5:02 PM, Benjamin Peterson

Post by Benjamin Peterson

Post by Neal Norwitz
On Tue, Aug 26, 2008 at 9:46 AM, benjamin.peterson

What happens if x (the object passed in) is NULL?

It segfaults, but this is ok because PyObject_Bytes checks for NULL.

Since this is now a public API, anyone can pass NULL here. In general
we check for NULLs from most public APIs (I think). Though IIRC we
are pretty inconsistent.

I wasn't under that impression. For example, a quick glance shows that
none of the PyList API functions check for NULL.

Like I said, we're inconsistent. :-) We discussed this on python-dev
a while ago, but there wasn't really any resolution IIRC.
I generally prefer to be safe which is why I prefer to have checks
unless there's a good reason not to.

Post by Benjamin Peterson
Anyway, what should PyBytes_FromObject return for NULL anyway?
Probably just another NULL.

if (x == NULL) {
PyErr_BadInternalCall();
return NULL;
}

Well, I suppose it can't hurt. :)

Post by Neal Norwitz
n

--
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."