Linux/Unix > Python Essential Reference, Second Edition > A. The Python Library > String Handling | See All Titles

< BACK | Make Note | Bookmark | CONTINUE >

155117184014003188065099048180054212144238241179195140058238111161105083198031187007140042

### String Handling

The modules in this section are used for string processing.

#### `codecs`

The `codecs` module provides an interface for accessing different string encoding and decoding functions (codecs) as well as a collection of base classes that can be used to define new codecs. The following functions are available:

    register(search_function)

Registers a new codec search function. This function should take a single argument in the form of an encoding string (for example, `'utf-8'`) and return a tuple of functions `(``encoder` `,` `decoder` `,` `streamreader` `,` `streamwriter` `)`.

    lookup(encoding)

Looks up a codec in the codec registry. `encoding` is a string such as `'utf-8'`. Returns a tuple of functions `(``encoder` `,` `decoder` `,` `streamreader` `,` `streamwriter` `)`. Internally, this function keeps a cache of previously used encodings. If a match is not found in the cache, all the registered search functions are invoked until a match is found. If no match is found, `LookupError` is raised.

    open(filename, mode[, encoding[, errors[, buffering]]])

Opens `filename` in the given `mode` and provides transparent data encoding/ decoding according to the encoding specified in `encoding` . `errors` is one of `'strict'`, `'ignore'`, or `'replace'`. The default is `'strict'`. `buffering` has the same meaning as for the built-in `open()` function.

    EncodedFile(file, inputenc[, outputenc [, errors]])

A class that provides an encoding wrapper around a file object `file` . Data written to the file is first interpreted according to the input encoding `inputenc` and then written to the file using the output encoding `outputenc` . Data read from the file is decoded according to `inputenc` . If `outputenc` is omitted, it defaults to `inputenc` . `errors` has the same meaning as for `open()` and defaults to `'strict'`.

To define new codecs, the `codecs` module defines a base class `Codec` that is subclassed when defining encoders and decoders. The interface to a `Codec` object `c` is as follows:

    c.encode(self, input [, errors])

Encodes input and returns a tuple `(``output` `,` `length` `)` where `length` is the length of the data in `input` that was consumed in the encoding. `errors` is one of `'strict'`, `'ignore'`, or `'replace'` and defaults to `'strict'`.

    c.decode(self, input [, errors])

Decodes input and returns a tuple (`output` , `length` ) where `length` is the length of the data that was consumed in the decoding. `errors` defaults to `'strict'`.

Neither the `encode()` nor `decode()` method should maintain internal state. In addition, both methods must be able to operate with zero-length input, producing a zero-length output object of the proper type.

In addition, the `codecs` module provides base classes for four different types of I/O interfaces. All of these classes are subclasses of `Codec`.

    StreamWriter(stream [, errors])

Provides a wrapper around `stream` for producing an encoded output stream. An instance `w` of `StreamWriter` provides the same methods as `stream` . In addition, the following methods are defined:

    w.write(object)

Writes an encoded version of `object` to `w` .

    w.writelines(list)

Writes a concatenated list of strings to `w` .

    w.reset()

Flushes the output buffers and resets the internal encoding state.

    StreamReader(stream [, errors])

Provides a wrapper around `stream` for reading an encoded input stream. An instance `r` of `StreamReader` provides the same methods as `stream` in addition to the following methods:

    r.read([size])

Reads decoded data from `r` . `size` is the approximate number of bytes to read. The decoder may adjust this value slightly to accommodate the underlying encoding. If `size` is omitted, all data is read and decoded.

    r.readline([size])

Reads a single line of input using the underlying stream’s `readline()` method and returns as decoded data. `size` is simply passed to the underlying `readline()` method.

    r.readlines([size])

Reads all lines and returns as a list of decoded lines.

    r.reset()

Resets the codec buffers. This is usually used to recover from decoding errors.

    StreamReaderWriter(stream, reader, writer [, errors])

Provides a wrapper around a stream that provides both encoding and decoding. `stream` is any file object. `reader` must be a factory function or class implementing the `StreamReader` interface. `writer` must be a factory function or class implementing the `StreamWriter` interface. A `StreamWriter` instance provides the combined interface of `StreamReader` and `StreamWriter`.

    StreamRecoder(stream, encode, decode, reader, writer [, errors])

Provides a wrapper around `stream` that allows for conversion between two different encodings (for example, UTF-8 to and from UTF-16). `stream` may be any filelike object. The `encode` and `decode` arguments define the encoding and decoding functions that are returned or accepted by the `read()` and `write()` methods, respectively; that is, data returned by `read()` is encoded according to `encode` and data given to `write()` is decoded according to `decode` . `reader` and `writer` are the `StreamReader` and `StreamWriter` classes used to read and write the actual contents of the data stream. A `StreamRecoder` object provides the combined interface of `StreamReader` and `StreamWriter`.

`codecs` also defines the following byte-order marker constants that can be used to help interpret platform-specific files:

Constant | Description
--- | ---
`BOM` | Native byte-order marker for the machine
`BOM_BE` | Big-endian byte-order marker
`BOM_LE` | Little-endian byte-order marker
`BOM32_BE` | 32-bit big-endian marker
`BOM32_LE` | 32-bit little-endian marker
`BOM64_BE` | 64-bit big-endian marker
`BOM64_LE` | 64-bit little-endian marker

##### Example

The following example illustrates the implementation of a new encoding using simple exclusive-or (XOR) based encryption. This only works for 8-bit strings, but it could be extended to support Unicode:

    # xor.py: Simple encryption using XOR
    import codecs

    # Encoding/decoding function (works both ways)
    def xor_encode(input, errors = 'strict', key=0xff):
        output = "".join([chr(ord(c) ^ key) for c in input])
        return (output,len(input))

    # XOR Codec class
    class Codec(codecs.Codec):
        key = 0xff
        def encode(self,input, errors='strict'):
            return xor_encode(input,errors,self.key)
        def decode(self,input, errors='strict'):
            return xor_encode(input,errors,self.key)

    # StreamWriter and StreamReader classes
    class StreamWriter(Codec,codecs.StreamWriter):
        pass

    class StreamReader(Codec,codecs.StreamReader):
        pass

    # Factory functions for creating StreamWriter and
    # StreamReader objects with a given key value.

    def xor_writer_factory(stream,errors,key=0xff):
        s = StreamWriter(stream,errors)
        s.key = key
        return s;

    def xor_reader_factory(stream,errors,key=0xff):
        r = StreamReader(stream,errors)
        r.key = key
        return r

    # Function registered with the codecs module.  Recognizes any
    # encoding of the form 'xor-hh' where hh is a hexadecimal number.

    def lookup(s):
        if (s[:4] == 'xor-'):
            key = int(s[4:],16)
        # Create some functions with key set to desired value
        e = lambda x,err='strict',key=key:xor_encode(x,err,key)
        r = lambda x,err='strict',key=key:xor_reader_factory(x,err,key)
        w = lambda x,err='strict',key=key:xor_writer_factory(x,err,key)
        return (e,e,r,w)

    # Register with the codec module
    codecs.register(lookup)

Now, here’s a short program that uses the encoding:

    import xor, codecs
    f = codecs.open("foo","w","xor-37")
    f.write("Hello World\n")        # Writes an "encrypted" version
    f.close()

    (enc,dec,r,w) = codecs.lookup("xor-ae")
    a = enc("Hello World")
    # a = ('\346\313\302\302\301\216\371\301\334\302\312', 11)

##### Notes

- Further use of the `codecs` module is described in Chapter 9.

- Most of the built-in encodings are provided to support Unicode string encoding. In this case, the encoding functions produce 8-bit strings and the decoding functions produce Unicode strings.

? See Also Chapter 9.

#### `re`

The `re` module is used to perform regular-expression pattern matching and replacement in strings. Both ordinary and Unicode strings are supported. Regular-expression patterns are specified as strings containing a mix of text and special-character sequences. Since patterns often make extensive use of special characters and the backslash, they’re usually written as “raw” strings such as `r'(?P<``int` `>\d+)\.(\d*)'`. For the remainder of this section, all regular-expression patterns are denoted using the raw string syntax.

The following special-character sequences are recognized in regular expression patterns:

Character(s) | Description
--- | ---
`text` | Matches the literal string `text` .
`.` | Matches any character except newline.
`^` | Matches the start of a string.
`$` | Matches the end of the string.
`*` | Matches zero or more repetitions of the preceding expression, matching as many repetitions as possible.
`+` | Matches one or more repetitions of the preceding expression, matching as many repetitions as possible.
`?` | Matches zero repetitions or one repetition of the preceding expression.
`*?` | Matches zero or more repetitions of the preceding expression, matching as few repetitions as possible.
`+?` | Matches one or more repetitions of the preceding expression, matching as few repetitions as possible.
`??` | Matches zero or one repetitions of the preceding expression, matching as few repetitions as possible.
`{``m` `,` `n` `}` | Matches from `m` to `n` repetitions of the preceding expression, matching as many repetitions as possible.
`{``m` `,` `n` `}?` | Matches from `m` to `n` repetitions of the preceding expression, matching as few repetitions as possible.
`[...]` | Matches a set of characters such as `r'[abcdef]'` or `r'[a-zA-z]'`. Special characters such as `*` are not active inside a set.
`[^...]` | Matches the characters not in the set, such as `r'[^0-9]'`.
`A` `|``B` | Matches either `A` or `B` where `A` and `B` are both regular expressions.
`(...)` | Matches the regular expression inside the parentheses as a group and saves the matched substring.
`(?iLmsux)` | Interprets the letters `'i'`, `'L'`, `'m'`, `'s'`, `'u'`, and `'x'` as flag settings corresponding to the `re.I`, `re.L`, `re.M`, `re.S`, `re.U`, `re.X` flag settings given to `re.compile()`.
`(?:...)` | Matches the regular expression inside the parentheses, but discards the matched substring.
`(?P<``name` `>...)` | Matches the regular expression in the parentheses and creates a named group.The group name must be a valid Python identifier.
`(?P=``name` `)` | Matches the same text that was matched by an earlier named group.
`(?#...)` | A comment. The contents of the parentheses are ignored.
`(?=...)` | Matches the preceding expression only if followed by the pattern in the parentheses. For example, `r'Hello (?=World)'` matches `'Hello '` only if followed by `'World'`.
`(?!...)` | Matches the preceding expression only if it’s not followed by the pattern in parentheses. For example, `r'Hello (?!World)'` matches `'Hello '` only if it’s not followed by `'World'`.
`(?<=...)` | Matches the following expression if it’s preceded by a match of the pattern in parentheses. For example, `r'(?<=abc)def'` matches `'def'` only if it’s preceded by `'abc'`.
`(?<!...)` | Matches the following expression only if it’s not preceded by a match of the pattern in parentheses. For example, `r'(?<!abc)def'` matches `'def'` only if it’s not preceded by `'abc'`.

Standard character escape sequences such as `'\n'` or `'\t'` are recognized as standard characters in a regular expression; for example, `r'\n+'` would match one or more newline characters. In addition, literal symbols that normally have special meaning in a regular expression can be specified by preceding them with a backslash. For example, `r'\*'` matches the character `*`. In addition, a number of backslash sequences correspond to special sets of characters:

Character(s) | Description
--- | ---
`\``number` | Matches the text that was matched by a previous group number. Groups are numbered from 1 to 99, starting from the left.
`\A` | Matches only at the start of the string.
`\b` | Matches the empty string at the beginning or end of a word. A word is a sequence of alphanumeric characters terminated by whitespace or any other non-alphanumeric character.
`\B` | Matches the empty string not at the beginning or end of a word.
`\d` | Matches any decimal digit. Same as `r'[0-9]'`.
`\D` | Matches any non-digit character. Same as `r'[^0-9]'`.
`\s` | Matches any whitespace character. Same as `r'[ \t\n\r\f\v]'`.
`\S` | Matches any non-whitespace character. Same as `r'[^ \t\n\r\f\v]'`.
`\w` | Matches any alphanumeric character.
`\W` | Matches any character not contained in the set defined by `\w`.
`\Z` | Matches only at the end of the string.
`\\` | Matches a literal backslash.

The following functions are used to perform pattern matching and replacement.

    compile(str [, flags])

Compiles a regular-expression pattern string into a regular-expression object. This object can be passed as the pattern argument to all the functions that follow. `flags` is the bitwise-or of the following:

Flag | Description
--- | ---
`I` or `IGNORECASE` | Performs non–case-sensitive matching.
`L` or `LOCALE` | Uses locale settings for `\w`, `\W`, `\b`, and `\B`.
`M` or `MULTILINE` | Makes `^` and `$` apply to each line in addition to the beginning and end of the entire string. (Normally `^` and `$` apply only to the beginning and end of an entire string.)
`S` or `DOTALL` | Makes the dot (`.`) character match all characters, including the newline.
`U` or `UNICODE` | Uses information from the Unicode character properties database for `\w`, `\W`, `\b`, and `\B`.
`X` or `VERBOSE` | Ignores unescaped whitespace and comments.

    search(pattern, string [, flags])

Searches `string` for the first match of `pattern` . `flags` has the same meaning as for `compile()`. Returns a `MatchObject` on success, `None` if no match was found.

    match(pattern, string [, flags])

Checks whether zero or more characters at the beginning of `string` match `pattern` . Returns a `MatchObject` on success, or `None`.

    split(pattern, string [, maxsplit = 0])

Splits `string` by the occurrences of `pattern` . Returns a list of strings including the text matched by any groups in the pattern. `maxsplit` is the maximum number of splits to perform. By default, all possible splits are performed.

    findall(pattern, string)

Returns a list of all non-overlapping matches of `pattern` in `string` , including empty matches. If the pattern has groups, a list of the text matched by the groups is returned. If more than one group is used, each item in the list is a tuple containing the text for each group.

    sub(pattern, repl, string [, count = 0])

Replaces the leftmost non-overlapping occurrences of `pattern` in `string` by the replacement `repl` . `repl` can be a string or a function. If it’s a function, it’s called with a `MatchObject` and should return the replacement string. If `repl` is a string, back references such as `'\6'` are used to refer to groups in the pattern.The sequence `'\g<``name` `>'` is used to refer to a named group. `count` is the maximum number of substitutions to perform. By default, all occurrences are replaced. Although these functions don’t accept a `flags` parameter like `compile()`, the same effect can be achieved by using the `(?iLmsux)` notation described earlier in this section.

    subn(pattern, repl, string [, count = 0])

Same as `sub()`, but returns a tuple containing the new string and the number of substitutions.

    escape(string)

Returns a string with all non-alphanumerics backslashed.

A compiled regular-expression object `r` created by the `compile()` function has the following methods and attributes:

    r.search(string [, pos [, endpos]])

Searches `string` for a match. `pos` and `endpos` specify the starting and ending positions for the search. Returns a `MatchObject` for a match, `None` otherwise.

    r.match(string [, pos [, endpos]])

Checks whether zero or more characters at the beginning of `string` match. `pos` and `endpos` specify the range of `string` to be searched. Returns a `MatchObject` for a match, `None` otherwise.

    r.split(string [, maxsplit = 0])

Identical to the `split()` function.

    r.findall(string)

Identical to the `findall()` function.

    r.sub(repl, string [, count = 0])

Identical to the `sub()` function.

    r.subn(repl, string [, count = 0])

Identical to the `subn()` function.

    r.flags

The `flags` argument used when the regular expression object was compiled, or `0`.

    r.groupindex

A dictionary mapping symbolic group names defined by `r'(?P<``id` `>)'` to group numbers.

    r.pattern

The pattern string from which the regular expression object was compiled.

The `MatchObject` instances returned by `search()` and `match()` contain information about the contents of groups as well as positional data about where matches occurred. A `MatchObject` instance `m` has the following methods and attributes:

    m. expand(template)

Returns a string that would be obtained by doing regular-expression backslash substitution on the string `template` . Numeric back-references such as `"\1"` and `"\2"` and named references such as `"\g<``n` `>"` and `"\g<``name` `>"` are replaced by the contents of the corresponding group. Note: These sequences should be specified using raw strings or with a literal backslash character such as `r'\1'` or `'\\1'`.

    m.group([group1, group2, ...])

Returns one or more subgroups of the match. The arguments specify group numbers or group names. If no group name is given, the entire match is returned. If only one group is given, a string containing the text matched by the group is returned. Otherwise, a tuple containing the text matched by each of the requested groups is returned. An `IndexError` is raised if an invalid group number or name is given.

    m.groups([default])

Returns a tuple containing the text matched by all groups in a pattern. `default` is the value returned for groups that didn’t participate in the match (the default is `None`).

    m.groupdict([default])

Returns a dictionary containing all the named subgroups of the match. `default` is the value returned for groups that didn’t participate in the match (the default is `None`).

    m.start([group])
    m.end([group])

Returns the indices of the start and end of the substring matched by a group. If `group` is omitted, the entire matched substring is used. Returns `None` if the group exists but didn’t participate in the match.

    m.span([group])

Returns a 2-tuple `(``m` `.start(``group` `),` `m` `.end(``group` `))`. If `group` didn’t contribute to the match, this returns `(None, None)`. If `group` is omitted, the entire matched substring is used.

    m.pos

The value of `pos` passed to the `search()` or `match()` function.

    m.endpos

The value of `endpos` passed to the `search()` or `match()` function.

    m.re

The regular-expression object whose `match()` or `search()` method produced this `MatchObject` instance.

    m.string

The string passed to `match()` or `search()`.

When pattern strings don’t specify a valid regular expression, the `re.error` exception is raised.

    m.lastindex

The integer index of the last matched capturing group. `None` if no group was matched.

    m.lastgroup

The name of the last matched capturing group. `None` if the group didn’t have a name or no group was matched.

##### Examples

    import re
    s = open('foo').read()         # Read some text

    # Replace all occurrences of 'foo' with 'bar'
    t = re.sub('foo','bar',s)

    # Get the title of an HTML document
    tmatch = re.search(r'<title>(.*?)</title>',s, re.IGNORECASE)
    if tmatch: title = tmatch.group(1)

    # Extract a list of possible e-mail addresses from s
    pat = re.compile(r'([a-zA-Z][\w-]*@[\w-]+(?:\.[\w-]+)*)')
    addrs = re.findall(pat,s)

    # Replace strings that look like URLs such as 'http://www.python.org'
    # with an HTML anchor tag of the form
    # <a href='http://www.python.org'>http://www.python.org</a>

    pat = re.compile(r'((ftp|http)://[\w-]+(?:\.[\w-]+)*(?:/[\w-]*)*)')
    t = pat.sub('<a href='\\1'>\\1</a>', s)

##### Notes

- The implementation of the `re` module is actually found in the module `sre`, which provides support for standard and Unicode strings. An older implementation of `re` that supports only standard strings is available in the `pre` module.

- Detailed information about the theory and implementation of regular expressions can be found in textbooks on compiler construction. The book Mastering Regular Expressions by Jeffrey Friedl (O’Reilly & Associates, 1997) may also be useful.

- The `re` module is 8-bit clean and can process strings that contain null bytes and characters whose high bit is set. Regular expression patterns cannot contain null bytes, but can match against the null byte by writing the escape sequence `'\000'`.

? See Also string (142).

#### `string`

The `string` module contains a number of useful constants and functions for manipulating strings. Most of the functionality of this module is also available in the form of string methods. The following constants are defined:

Constant | Description
--- | ---
`digits` | The string `'0123456789'`.
`hexdigits` | The string `'0123456789abcdefABCDEF'`.
`letters` | Concatenation of lowercase and uppercase.
`lowercase` | String containing all lowercase letters.
`octdigits` | The string `'01234567'`.
`punctuation` | String of ASCII punctuation characters.
`printable` | String of ASCII characters considered to be printable.
`uppercase` | String containing all uppercase letters.
`whitespace` | String containing all whitespace characters. This usually includes space, tab, linefeed, return, formfeed, and vertical tab.

The following functions are available:

    atof(s)

Converts string `s` to a floating-point number. See the built-in `float()` function.

    atoi(s [, base])

Converts string `s` to an integer. `base` is an optional base. See the built-in `int()` function.

    atol(s [, base])

Converts string `s` to a long integer. `base` is an optional base. See the built-in `long()` function.

    capitalize(s)

Capitalizes the first character of `s` . Same as `s` `.capitalize()`.

    capwords(s)

Capitalizes the first letter of each word in `s` , replaces repeated whitespace characters with a single space, and removes leading and trailing whitespace.

    count(s, sub [, start [, end]])

Counts the number of non-overlapping occurrences of `sub` in `s` `[``start` `:``end` `]`. Same as `s` `.count(``sub` `,` `start` `,` `end` `)`.

    expandtabs(s [, tabsize=8])

Expands tabs in string `s` with whitespace. `tabsize` specifies the number of characters between tab stops. Same as `s` `.expandtab(``tabsize` `)`.

    find(s, sub [, start [, end]])
    index(s, sub [, start [, end]])

Returns the first index in `s` `[``start` `:``end` `]` where the substring `sub` is found. If `start` and `end` are omitted, the entire string is searched. `find()` returns `-1` if not found, while `index()` raises a `ValueError` exception. Same as `s` `.find(``sub` `,``start` `,``end` `)` and `s` `.index(``sub` `,``start` `,``end` `)`.

    rfind(s, sub [, start [, end]])
    rindex(s, sub [, start [, end]])

Like `find()` and `index()`, but finds the highest index. Same as `s` `.rfind(``sub` `,``start` `,``end` `)` and `s` `.rindex(``sub` `,``start` `,``end` `)`.

    lower(s)

Converts all uppercase characters in `s` to lowercase. Same as `s` `.lower().`

    maketrans(from, to)

Creates a translation table that maps each character in `from` to the character in the same position in `to` . `from` and `to` must be the same length.

    split(s [, sep [, maxsplit]])
    splitfields(s [, sep [, maxsplit]])

Returns a list of words in `s` . If `sep` is omitted or `None`, the words are separated by whitespace. Otherwise, the string in `sep` is used as a delimiter. `maxsplit` specifies the maximum number of splits that can occur.The remainder of the string will be returned as the last element. `split()` is the same as `s` `.split(``sep` `,``maxsplit` `)`.

    join(words [, sep])
    joinfields(words [, sep])

Concatenates a sequence of words into a string, with words separated by the string in `sep` . If omitted, the words are separated by whitespace. Same as `sep` `.join(``words` `)`.

    lstrip(s)
    rstrip(s)
    strip(s)

Strips leading and/or trailing whitespace from `s` . Same as `s` `.lstrip()`, `s` `.rstrip()`, and `s` `.strip()`.

    swapcase(s)

Changes uppercase to lowercase and lowercase to uppercase in `s` . Same as `s` `.swapcase()`.

    translate(s, table [, delchars])

Deletes all characters from `s` that are in `delchars` and translates the remaining characters using `table` . `table` must be a 256-character string mapping characters to characters as created by `maketrans()`. Same as `s` `.translate(``table` `,``delchars` `)`.

    upper(s)

Converts all lowercase characters in `s` to uppercase. Same as `s` `.upper()`.

    ljust(s, width)
    rjust(s, width)
    center(s, width)

Left-aligns, right-aligns, or centers `s` in a field of width `width` . Same as `s` `.ljust(``width` `)`, `s` `.rjust(``width` `)`, and `s` `.center(``width` `)`.

    zfill(s, width)

Pads a numeric string on the left with `0` digits up to the given width.

    replace(str, old, new [, max])

Replaces `max` occurrences of `old` with `new` in `str` . If `max` is omitted, all occurrences are replaced. Same as `s` `.replace(``old` `,``new` `,``max` `)`.

##### Notes

- This module is officially considered to be deprecated due to the addition of string methods in Python 2.0, but is still widely used in existing Python programs.

- Unicode and standard strings are supported by the module, but standard strings are coerced to Unicode when necessary.

? See Also re (136), Chapter 3.

#### `StringIO` and `cStringIO`

The `StringIO` and `cStringIO` modules define an object that behaves like a file but reads and writes data from a string buffer.

    StringIO([buffer])

Creates a new `StringIO` object. `buffer` is an initial value (by default, the empty string).

A `StringIO` object supports all the standard file operations— `read()`, `write()`, and so on—and the following methods:

    s.getvalue()

Returns the contents of the string buffer before `close()` is called.

    s.close()

Releases the memory buffer.

##### Note

- The `StringIO` module defines `StringIO` as a class. `cStringIO` defines it as an extension type and provides significantly faster performance.

? See Also Chapter 9, the section “Files” (for file methods).

#### `struct`

The `struct` module is used to convert data between Python and binary data structures (represented as Python strings). These data structures are often used when interacting with functions written in C or with binary network protocols.

    pack(fmt, v1, v2, ...)

Packs the values `v1` , `v2` , and so on into a string according to the format string in `fmt` .

    unpack(fmt, string)

Unpacks the contents of `string` according to the format string in `fmt` . Returns a tuple of the unpacked values.

    calcsize(fmt)

Calculates the size in bytes of the structure corresponding to a format string `fmt` .

The format string is a sequence of characters with the following interpretations:

Format | C Type | Python Type
--- | --- | ---
`'x'` | `pad byte` | No value
`'c'` | `char` | String of length 1
`'b'` | `signed char` | Integer
`'B'` | `unsigned char` | Integer
`'h'` | `short` | Integer
`'H'` | `unsigned short` | Integer
`'i'` | `int` | Integer
`'I'` | `unsigned int` | Integer
`'l'` | `long` | Integer
`'L'` | `unsigned long` | Integer
`'f'` | `float` | Float
`'d'` | `double` | Float
`'s'` | `char[]` | String
`'p'` | `char[]` | String with length encoded in the first byte
`'P'` | `void *` | Integer

Each format character can be preceded by an integer to indicate a repeat count (for example, `'4i'` is the same as `'iiii'`). For the `'s'` format, the count represents the maximum length of the string, so `'10s'` represents a 10-byte string. A format of `'0s'` indicates a string of zero length. The `'p'` format is used to encode a string in which the length appears in the first byte, followed by the string data. This is useful when dealing with Pascal code, as is sometimes necessary on the Macintosh. Note: The length of the string in this case is limited to 255 characters.

When the `'I'` and `'L'` formats are used to unpack a value, the return value is a Python long integer. In addition, the `'P'` format may return an integer or long integer, depending on the word size of the machine.

The first character of each format string can also specify a byte ordering and alignment of the packed data, as shown in the following table.

Format | Byte Order | Size and Alignment
--- | --- | ---
`'@'` | Native | Native
`'='` | Native | Standard
`'<'` | Little-endian | Standard
`'>'` | Big-endian | Standard
`'!'` | Network (big-endian) | Standard

Native byte ordering may be little-endian or big-endian, depending on the machine architecture. The native sizes and alignment correspond to the values used by the C compiler and are implementation-specific. The standard alignment assumes that no alignment is needed for any type. The standard size assumes that `short` is 2 bytes, `int` is 4 bytes, `long` is 4 bytes, `float` is 32 bits, and `double` is 64 bits. The `'P'` format can only use native byte ordering.

Sometimes it’s necessary to align the end of a structure to the alignment requirements of a particular type. To do this, end the structure-format string with the code for that type with a repeat count of zero. For example, the format `'llh0l'` specifies a structure that ends on a four-byte boundary (assuming that longs are aligned on four-byte boundaries). In this case, two pad bytes would be inserted after the short value specified by the `'h'` code. This only works when native size and alignment are being used—standard size and alignment don’t enforce alignment rules.

? See Also array (127), xdrlib (277).

#### `unicodedata`

The `unicodedata` modules provide access to the Unicode character database, which contains character properties for all Unicode characters.

    decimal(unichr[, default])

Returns the decimal integer value assigned to the character `unichr` . If `unichr` is not a decimal digit, `default` is returned or `ValueError` is raised.

    digit(unichr[, default])

Returns the integer digit value assigned to the character `unichr` . If `unichr` is not a digit, `default` is returned or `ValueError` is raised. This function differs from `decimal()` in that it works with characters that may represent digits, but that are not decimal digits.

    numeric(unichr[, default])

Returns the value assigned to the Unicode character `unichr` as a floating-point number. If no numeric value is defined, `default` is returned or `ValueError` is raised. For example, the numeric value of U+2155 (the character for the fraction `'1/5'`) is `0.2`.

    category(unichr)

Returns a string describing the general category of `unichr` . The returned string is one of the following values:

Value | Description
--- | ---
`Lu` | Letter, Uppercase
`Ll` | Letter, Lowercase
`Lt` | Letter,Titlecase
`Mn` | Mark, Non-Spacing
`Mc` | Mark, Spacing Combining
`Me` | Mark, Enclosing
`Nd` | Number, Decimal Digit
`Nl` | Number, Letter
`No` | Number, Other
`Zs` | Separator, Space
`Zl` | Separator, Line
`Zp` | Separator, Paragraph
`Cc` | Other, Control
`Cf` | Other, Format
`Cs` | Other, Surrogate
`Co` | Other, Private Use
`Cn` | Other, Not Assigned
`Lm` | Letter, Modifier
`Lo` | Letter, Other
`Pc` | Punctuation, Connector
`Pd` | Punctuation, Dash
`Ps` | Punctuation, Open
`Pe` | Punctuation, Close
`Pi` | Punctuation, Initial quote
`Pf` | Punctuation, Final quote
`Po` | Punctuation, Other
`Sm` | Symbol, Math
`Sc` | Symbol, Currency
`Sk` | Symbol, Modifier
`So` | Symbol, Other

    bidirectional(unichr)

Returns the bi-directional category assigned to `unichr` as a string, or an empty string if no such value is defined. Returns one of the following:

Value | Description
--- | ---
`L` | Left-to-Right
`LRE` | Left-to-Right Embedding
`LRO` | Left-to-Right Override
`R` | Right-to-Left
`AL` | Right-to-Left Arabic
`RLE` | Right-to-Left Embedding
`RLO` | Right-to-Left Override
`PDF` | Pop Directional Format
`EN` | European Number
`ES` | European Number Separator
`ET` | European Number Terminator
`AN` | Arabic Number
`CS` | Common Number Separator
`NSM` | Non-Spacing Mark
`BN` | Boundary Neutral
`B` | Paragraph Separator
`S` | Segment Separator
`WS` | Whitespace
`ON` | Other Neutrals

    combining(unichr)

Returns an integer describing the combining class for `unichr` , or `0` if no combining class is defined. One of the following values is returned:

Value | Description
--- | ---
`0` | Spacing, split, enclosing, reordrant, and Tibetan subjoined
`1` | Overlays and interior
`7` | Nuktas
`8` | Hiragana/Katakana voicing marks
`9` | Viramas
`10-199` | Fixed position classes
`200` | Below left attached
`202` | Below attached
`204` | Below right attached
`208` | Left attached
`210` | Right attached
`212` | Above left attached
`214` | Above attached
`216` | Above right attached
`218` | Below left
`220` | Below
`222` | Below right
`224` | Left
`226` | Right
`228` | Above left
`230` | Above
`232` | Above right
`233` | Double below
`234` | Double above
`240` | Below (iota subscript)

    mirrored(unichr)

Returns `1` if `unichr` is a “mirrored” character in bi-directional text, `0` otherwise.

    decomposition(unichr)

Returns a string containing the decomposition mapping of `unichr` , or the empty string if no such mapping is defined. Typically, characters containing accent marks can be decomposed into multicharacter sequences. For example, `decomposition(u'\u00fc') ('ü')` returns the string `'0075 0308'` corresponding to the letter u and the umlaut (¨) accent mark. The string returned by this function may also include the following strings:

Value | Description
--- | ---
`<font>` | A font variant (for example, a blackletter form)
`<noBreak>` | A nonbreaking version of a space or hyphen
`<initial>` | An initial presentation form (Arabic)
`<medial>` | A medial presentation form (Arabic)
`<final>` | A final presentation form (Arabic)
`<isolated>` | An isolated presentation form (Arabic)
`<circle>` | An encircled form
`<super>` | A superscript form
`<sub>` | A subscript form
`<vertical>` | A vertical layout presentation form
`<wide>` | A wide (or zenkaku) compatibility character
`<narrow>` | A narrow (or hankaku) compatibility character
`<small>` | A small variant form (CNS compatibility)
`<square>` | A CJK squared-font variant
`<fraction>` | A vulgar fraction form
`<compat>` | Otherwise unspecified compatibility character

##### Note

- For further details about the Unicode character database, see http://www.unicode.org.

| Last updated on 3/28/2002 Python Essential Reference, Second Edition, © 2002 New Riders Publishing

< BACK | Make Note | Bookmark | CONTINUE >

© 2002, O'Reilly & Associates, Inc.