Linux/Unix > Python Essential Reference, Second Edition > A. The Python Library > Threads | See All Titles

< BACK | Make Note | Bookmark | CONTINUE >

155117184014003188065099048180054212144238241179195140058238111161105080005019029193022105

### Threads

This section describes modules that can be used to develop multithreaded applications. First, a little terminology and background.

#### Thread Basics

A running program is called a process. Associated with each process is a system state including memory, lists of open files, a program counter that keeps track of the instruction being executed, and a call stack used to hold the local variables of functions. Normally, a process executes statements in a single sequence of control flow. This sequence is sometimes called a thread (or main thread).

When a program creates new processes by using the `os.system()`, `os.fork()`, `os.spawnv()`, and similar system calls, these processes run as independent programs, each with its own set of system resources and main thread of execution. However, it’s also possible for a program to create additional threads of execution that exist inside the calling process and share data and system resources with the original thread of execution. Threads are particularly useful when an application wants to perform tasks concurrently without spawning child processes, or when subtasks need to read and write shared data.

A multithreaded program executes by dividing its processing time between all active threads. For example, a program with 10 active threads of execution would allocate approximately 1/10 of its CPU time to each thread and cycle between threads in rapid succession.

Since threads share the same data, an extreme degree of caution is required whenever shared data structures are updated by one of the threads. In particular, attempts to update a data structure by multiple threads at approximately the same time can lead to a corrupted and inconsistent program state (a problem formally known as a race condition). To fix these problems, threaded programs need to lock critical sections of code by using mutual-exclusion locks and other similar synchronization primitives.

More information regarding the theory and implementation of threads and locks can be found in most operating system textbooks.

#### Python Threads

Python supports threads on Windows, Solaris, and systems that support the POSIX threads library (`pthreads`). However, threads are often disabled by default, so it may be necessary to rebuild the interpreter with thread support before using any of the modules in this section. (Beginning with Python 2.1, thread support is enabled by default.)

The scheduling of threads and thread switching is tightly controlled by a global interpreter lock that allows only a single thread of execution to be running in the interpreter at once. Furthermore, thread switching can only occur between the execution of individual bytecodes in the interpreter. The frequency with which the interpreter checks for thread switching is set by the `sys.setcheckinterval()` function. By default, the interpreter checks for thread switching after every 10 bytecode instructions.

When working with extension modules, the interpreter may invoke functions written in C. Unless specifically written to interact with a threaded Python interpreter, these functions block the execution of all other threads until they complete execution. Thus, a long-running calculation in an extension module may limit the effectiveness of using threads. However, most of the I/O functions in the standard library have been written to work in a threaded environment.

Finally, programmers need to be aware that threads can interact strangely with signals and interrupts. For instance, the `KeyboardInterrupt` exception can be received by an arbitrary thread, while signals used in conjunction with the `signal` module are only received by the main thread.

#### `thread`

The `thread` module provides low-level functions for working with threads. This module is available only on UNIX and Windows.

    allocate_lock()

Creates a new lock object of type `LockType`. Locks are initially unlocked.

    exit()

Raises the `SystemExit` exception. Forces a thread to exit.

    get_ident()

Returns the integer “thread identifier” of the current thread.

    start_new_thread(func, args [, kwargs])

Executes the function `func` in a new thread. `func` is called using `apply(``func` `,` `args` `,` `kwargs` `)`. On success, control is immediately returned to the caller. When the function `func` returns, the thread exits silently. If the function terminates with an unhandled exception, a stack trace is printed and the thread exits (other threads continue to run, however).

A lock object `lck` returned by `allocate_lock()` has the following methods:

    lck.acquire([waitflag])

Acquires the lock, waiting until the lock is released by another thread if necessary. If `waitflag` is omitted, the function returns `None` when the lock is acquired. If `waitflag` is set to `0`, the lock is acquired only if it can be acquired immediately without waiting. If `waitflag` is nonzero, the method blocks until the lock is released. When `waitflag` is supplied, the function returns `1` if the lock was acquired successfully, `0` if not.

    lck.release()

Releases the lock.

    lck.locked()

Returns the lock status: `1` if locked, `0` if not.

##### Example

The following example shows a simple thread that prints the current time every five seconds:

    import thread
    import time
    def print_time(delay):
        while 1:
            time.sleep(delay)
            print time.ctime(time.time())

    # Start the new thread
    thread.start_new_thread(print_time,(5,))
    # Now go do something else while the thread runs
    while 1:
          pass

##### Exception

    error

Exception raised on thread-specific errors.

##### Notes

- Calling `sys.exit()` or raising the `SystemExit` exception is equivalent to calling `thread.exit()`.

- The `acquire()` method on a lock cannot be interrupted.

- When the main thread exits, whether the other threads survive depends on the system. On most systems, they’re killed immediately without executing any cleanup. Furthermore, the cleanup actions of the main thread are somewhat limited. In particular, standard I/O files are not flushed, nor are object destructors invoked.

? See Also threading (221).

#### `threading`

The `threading` module provides high-level thread support with a `Thread` class and classes for various synchronization primitives. It’s built using the lower-level `thread` module.

The following utility functions are available:

    activeCount()

Returns the number of currently active `Thread` objects.

    currentThread()

Returns the `Thread` object corresponding to the caller’s thread of control.

    enumerate()

Returns a list of all currently active `Thread` objects.

##### Thread Objects

The `Thread` class is used to represent a separate thread of control. A new thread can be created as follows:

    Thread(group=None, target=None, name=None, args=(), kwargs={})

Creates a new `Thread` instance. `group` is `None` and is reserved for future extensions. `target` is a callable object invoked by the `run()` method when the thread starts. By default, it’s `None`, meaning that nothing is called. `name` is the thread name. By default, a unique name of the form `"Thread-``N` `"` is created. `args` is a tuple of arguments that are passed to the `target` function. `kwargs` is a dictionary of keyword arguments that are passed to `target` .

A `Thread` object `t` supports the following methods:

    t.start()

Starts the thread by invoking the `run()` method in a separate thread of control. This method can be invoked only once.

    t.run()

This method is called when the thread starts. By default, it calls the target function passed in the constructor. This method can also be redefined in subclasses of `Thread`.

    t.join([timeout])

Waits until the thread terminates or a timeout occurs. `timeout` is a floating-point number specifying a timeout in seconds. A thread cannot join itself, and it’s an error to join a thread before it has been started.

    t.getName()

Returns the thread name.

    t.setName(name)

Sets the thread name.

    t.isAlive()

Returns `1` if the thread is alive, `0` otherwise. A thread is alive from the moment the `start()` method returns until its `run()` method terminates.

    t.isDaemon()

Returns the thread’s daemon flag.

    t.setDaemon(daemonic)

Sets the thread’s daemon flag to the Boolean value `daemonic` . This must be called before `start()` is called. The initial value is inherited from the creating thread. The entire Python program exits when no active non-daemon threads are left.

A thread can be flagged as a “daemon thread” using the `setDaemon()` method. If only daemon threads remain, a program will exit.All programs have a main thread that represents the initial thread of control. It’s not a daemon thread.

In some cases, dummy thread objects are created. These are threads of control started outside the `threading` module such as from a C extension module. Dummy threads are always considered alive, active, and daemonic, and cannot be joined. Furthermore, they’re never deleted, so it’s impossible to detect the termination of such threads.

As an alternative to explicitly creating a `Thread` object, the `Thread` class can also be subclassed. If this approach is used, the `run()` method can be overridden to perform the activity of the thread. The constructor can also be overridden, but it’s very important to invoke the base class constructor `Thread._ _init_ _()` in this case. It’s an error to override any other methods of the `Thread` class.

##### Lock Objects

A primitive lock (or mutual exclusion lock) is a synchronization primitive that’s in either a “locked” or “unlocked” state. Two methods, `acquire()` and `release()`, are used to change the state of the lock. If the state is locked, attempts to acquire the lock are blocked until the lock is released. If more than one thread is waiting to acquire the lock, only one is allowed to proceed when the lock is released. The order in which waiting threads proceed is undefined.

A new `Lock` instance is created using the following constructor:

    Lock()

Creates a new lock object, initially unlocked.

A `Lock` object `lck` supports the following methods:

    lck.acquire([blocking = 1])

Acquires the lock, blocking until the lock is released if necessary. If `blocking` is supplied and set to zero, the function returns immediately with a value of `0` if the lock could not be acquired, or `1` if locking was successful.

    lck.release()

Releases a lock. It’s an error to call this method when the lock is in an unlocked state.

##### RLock

A reentrant lock is a synchronization primitive that’s similar to a `Lock`, but that can be acquired multiple times by the same thread. This allows the thread owning the lock to perform nested `acquire()` and `release()` operations. In this case, only the outermost `release()` operation resets the lock to its unlocked state.

A new `RLock` object is created using the following constructor:

    RLock()

Creates a new reentrant lock object.

An `RLock` object `rlck` supports the following methods:

    rlck.acquire([blocking = 1])

Acquires the lock, blocking until the lock is released if necessary. If no thread owns the lock, it’s locked and the recursion level is set to `1`. If this thread already owns the lock, the recursion level of the lock is increased by one and the function returns immediately.

    rlck.release()

Releases a lock by decrementing its recursion level. If the recursion level is zero after the decrement, the lock is reset to the unlocked state. Otherwise, the lock remains locked. This function should only be called by the thread that currently owns the lock.

##### ConditionVariables

A condition variable is a synchronization primitive, built on top of another lock, that’s used when a thread is interested in a particular change of state or event to occur. A typical use is a producer-consumer problem where one thread is producing data to be consumed by another thread. A new `Condition` instance is created using the following constructor:

    Condition([lock])

Creates a new condition variable. `lock` is an optional `Lock` or `RLock` instance. If not supplied, a new `RLock` instance is created for use with the condition variable.

A condition variable `cv` supports the following methods:

    cv.acquire(*args)

Acquires the underlying lock. This method calls the corresponding `acquire(*``args` `)` method on the underlying lock and returns its return value.

    cv.release()

Releases the underlying lock. This method calls the corresponding `release()` method on the underlying lock.

    cv.wait([timeout])

Waits until notified or until a timeout occurs. This method is called after the calling thread has already acquired the lock. When called, the underlying lock is released, and the thread goes to sleep until it’s awakened by a `notify()` or `notifyAll()` call performed on the condition variable by another thread. Once awakened, the thread reacquires the lock and the method returns. `timeout` is a floating-point number in seconds. If this time expires, the thread is awakened, the lock reacquired, and control returned.

    cv.notify([n])

Wakes up one or more threads waiting on this condition variable. This method is called only after the calling thread has acquired the lock, and does nothing if no threads are waiting. `n` specifies the number of threads to awaken and defaults to `1`. Awakened threads don’t return from the `wait()` call until they can reacquire the lock.

    cv.notifyAll()

Wakes up all threads waiting on this condition.

##### Examples

The following examples show a producer-consumer problem using condition variables:

    # Consume one item
    def consumer():
       cv.acquire()
       while not item_is_available():
             cv.wait()     # Wait for item
       cv.release()
       consume_item()      # Consume the item

    # Produce one item
    def produce():
       produce_item()      # Produce the item
       cv.acquire()
       make_item_available()
       cv.notify()         # Notify the consumer
       cv.release()

##### Semaphore

A semaphore is a synchronization primitive based on a counter that’s decremented by each `acquire()` call and incremented by each `release()` call. If the counter ever reaches zero, the `acquire()` method blocks until some other thread calls `release()`.

    Semaphore([value])

Creates a new semaphore. `value` is the initial value for the counter. If omitted, the counter is set to a value of `1`.

A `Semaphore` instance `s` supports the following methods:

    s.acquire([blocking])

Acquires the semaphore. If the internal counter is larger than zero on entry, decrements it by one and returns immediately. If it’s zero, blocks until another thread calls `release()`. The `blocking` argument has the same behavior as described for `Lock` and `RLock` objects.

    s.release()

Releases a semaphore by incrementing the internal counter by one. If the counter is zero and another thread is waiting, that thread is awakened. If multiple threads are waiting, only one will be returned from its `acquire()` call. The order in which threads are released is not deterministic.

##### Events

Events are used to communicate between threads. One thread signals an “event” and one or more other threads wait for it. An `Event` instance manages an internal flag that can be set to true with the `set()` method and reset to false with the `clear()` method. The `wait()` method blocks until the flag is true.

    Event()

Creates a new `Event` instance with the internal flag set to false.

An `Event` instance `e` supports the following methods:

    e.isSet()

Returns true only if the internal flag is true.

    e.set()

Sets the internal flag to true. All threads waiting for it to become true are awakened.

    e.clear()

Resets the internal flag to false.

    e.wait([timeout])

Blocks until the internal flag is true. If the internal flag is true on entry, returns immediately. Otherwise, blocks until another thread calls `set()` to set the flag to true, or until the optional timeout occurs. `timeout` is a floating-point number specifying a timeout period in seconds.

##### Example

The following example illustrates the use of the threading module by fetching a collection of URLs in separate threads. In this example, threads are defined by subclassing the `Thread` class.

    import threading
    import urllib
    class FetchUrlThread(threading.Thread):
         def __init__(self, url,filename):
                threading.Thread.__init__(self)
                self.url = url
                self.filename = filename
         def run(self):
                print self.getName(), 'Fetching ', self.url
                urllib.urlretrieve(self.url,self.filename)
                print self.getName(), 'Saved in ', self.filename
    urls = [ ('http://www.python.org','/tmp/index.html'),
              ('ftp://ftp.python.org/pub/python/2.1/Python-2.1.tgz', '/tmp/py21.tgz'),
              ('http://www.petaflop.org/p.png','/tmp/p.png'),
              ('http://www.pud.com','/tmp/pud.html')
           ]

    # Go fetch a bunch of URLs in separate threads
    for url,file in urls:
        t = FetchUrlThread(url,file)
        t.start()

? See Also thread (219), Queue (226).

#### `Queue`

The `Queue` module implements a multi-producer, multi-consumer FIFO queue that can be used to safely exchange information between multiple threads of execution. It’s available only if thread support has been enabled.

The `Queue` module defines the following class:

    Queue(maxsize)

Creates a new queue in which `maxsize` is the maximum number of items that can be placed in the queue. If `maxsize` is less than or equal to zero, the queue size is infinite.

A `Queue` object `q` has the following methods:

    q.qsize()

Returns the approximate size of the queue. Because other threads may be updating the queue, this number is not entirely reliable.

    q.empty()

Returns `1` if the queue is empty, `0` otherwise.

    q.full()

Returns `1` if the queue is full, `0` otherwise.

    q.put(item [, block])

Puts `item` into the queue. If optional argument `block` is `1` (the default), the caller blocks until a free slot is available. Otherwise (`block` is `0`), the `Full` exception is raised if the queue is full.

    q.put_nowait(item)

Equivalent to `q` `.put(``item` `, 0)`.

    q.get([block])

Removes and returns an item from the queue. If optional argument `block` is `1` (the default), the caller blocks until an item is available. Otherwise (`block` is `0`), the `Empty` exception is raised if the queue is empty.

    q.get_nowait()

Equivalent to `q` `.get(0)`.

##### Exceptions

    Empty

Exception raised when nonblocking `get()` or `get_nowait()` is called on a `Queue` object that’s empty or locked.

    Full

Exception raised when nonblocking `put()` or `put_nowait()` is called on a `Queue` object that’s full or locked.

? See Also thread (219), threading (221).

| Last updated on 3/28/2002 Python Essential Reference, Second Edition, © 2002 New Riders Publishing

< BACK | Make Note | Bookmark | CONTINUE >

© 2002, O'Reilly & Associates, Inc.