Linux/Unix > Python Essential Reference, Second Edition > 3. Types and Objects > Performance and Memory Considerations | See All Titles

< BACK | Make Note | Bookmark | CONTINUE >

155117184014003188065099048180054212144238241179195140058238110137104029061152217152035180

### Performance and Memory Considerations

All Python objects minimally include an integer reference count, a descriptor defining the type, and the representation of the actual data. Table 3.12 shows the approximate memory requirements of various built-in objects based on the C implementation of Python 2.0 running on a 32-bit machine. The precise values may vary slightly according to the implementation of the interpreter and machine architecture. (For instance, the memory requirements may double on a 64-bit machine.) Although you may never need to think about memory utilization, Python is used in a variety of high-performance and memory-critical applications ranging from supercomputing to mobile computing. The memory footprint of the built-in types is presented here to help programmers make informed design decisions in memory-critical settings.

##### Table 3.12. Memory Size of Built-in Datatypes

Type | Size
--- | ---
Integer | 12 bytes
Long integer | 12 bytes + (nbits/16 + 1)*2 bytes
Floats | 16 bytes
Complex | 24 bytes
List | 16 bytes + 4 bytes for each item
Tuple | 16 bytes + 4 bytes for each item
String | 20 bytes + 1 byte per character
Unicode string | 24 bytes + 2 bytes per character
Dictionary | 24 bytes + 12*2n bytes, n = log2(nitems)+1
Class instance | 16 bytes plus a dictionary object
Xrange object | 24 bytes

Because strings are used so frequently, the interpreter uses a number of optimizations. First, a string `s` can be interned using the built-in function `intern(``s` `)`. This function looks in an internal hash table to see whether the string value already exists. If so, a reference to that string—instead of a copy of the string data—is stored in the string object. If not, the data in `s` is added to the hash table. Interned strings live until the interpreter exits; if you’re concerned about memory, you shouldn’t intern infrequently used strings. Also, to increase the performance of dictionary lookups, strings cache their last computed hash-table value.

Dictionaries are implemented using a hash table with open indexing. The number of entries allocated to a dictionary is equal to twice the smallest power of 2 that’s greater than the number of objects stored in the dictionary. When a dictionary expands, its size doubles. On average, about half of the entries allocated to a dictionary are unused.

The execution of a Python program is primarily a sequence of function calls involving the special methods described in the earlier section “Special Methods.” Next to choosing the most efficient algorithm, performance improvements can be made by understanding Python’s object model and trying to minimize the number of special method calls that occur during execution. This is especially true for name lookups on modules and classes. For example, consider the following code:

    import math
    d = 0.0
    for i in xrange(1000000):
         d = d + math.sqrt(i)

In this case, each iteration of the loop involves two name lookups. First, the `math` module is located in the global namespace; then it’s searched for a function object named `sqrt`. Now consider the following modification:

    from math import sqrt
    d = 0.0
    for i in xrange(1000000):
         d = d + sqrt(i)

In this case, one name lookup is eliminated from the inner loop. In fact, when running on the author’s speedy 200 MHz PC, this simple change makes the code run more than twice as fast as before.

Unnecessary method calls can also be eliminated by making careful use of temporary values and avoiding unnecessary lookups in sequences and dictionaries. For example, consider the two classes shown in Listing 3.2:

##### Listing 3.2 Unnecessary Method Calls

    class Point:
        def _ _init_ _(self,x,y,z):
              self.x = x
              self.y = y
              self.z = z

    class Poly:
        def _ _init_ _(self):
              self.pts = [ ]
        def addpoint(self,pt):
              self.pts.append(pt)
        def perimeter(self):
              d = 0.0
              self.pts.append(self.pts[0])     # Temporarily close the polygon
              for i in xrange(len(self.pts)-1):
                   d2 = (self.pts[i+1].x - self.pts[i].x)**2 + \
                        (self.pts[i+1].y - self.pts[i].y)**2 + \
                        (self.pts[i+1].z - self.pts[i].z)**2
                   d = d + math.sqrt(d2)
              self.pts.pop()                   # Restore original list of points
              return d

In the `perimeter()` method, each occurrence of `self.pts[i]` involves two special-method lookups—one involving a dictionary and another involving a sequence. You can reduce the number of lookups by rewriting the method as shown in Listing 3.3:

##### Listing 3.3 Improved Version of Listing 3.2

    class Poly:
          ...
          def perimeter(self):
              d = 0.0
              pts = self.pts
              pts.append(pts[0])
              for i in xrange(len(pts)-1):
                    p1 = pts[i+1]
                    p2 = pts[i]
                    d2 = (p1.x - p2.x)**2 + \
                         (p1.y - p2.y)**2 + \
                         (p1.z - p2.z)**2
                    d = d + math.sqrt(d2)
              pts.pop()
              return d

Although the performance gains made by such modifications are often modest (15–20%), an understanding of the underlying object model and the manner in which special methods are invoked can result in faster programs. Of course, if performance is extremely critical, you often can export functionality to a Python extension module written in C.

| Last updated on 3/28/2002 Python Essential Reference, Second Edition, © 2002 New Riders Publishing

< BACK | Make Note | Bookmark | CONTINUE >

© 2002, O'Reilly & Associates, Inc.