Various utilities to help read and process files


Bases: exceptions.IOError

Raised if the a file can’t be opened with the correct module

New in version 0.1.13.

Tries to wrap a file handle in the appropriate compressed file class.

Parameters:file_handle (str) – file handle
Returns:the same file handle if no suitable compressed file class is found or the new file_handle which supports the compression
Return type:file
Raises:UnsupportedFormat – if the module to open the file is not available, key_func=None, skip_elements=0)

New in version 0.3.1.

Group the elements of an iterator by a key and yields the grouped elements. The elements yielded by the iterator are assumed to be a list or tuple, with the default key (when key_func is None) being the first of the of the objects inside that element. This behaviour can be customised by passing to key_func a function that accept an element and returns the key to be used.


the iterable assumen that the elements are already sorted by their keys

  • iterator (iterable) – iterator to be grouped
  • key_func (func) – function that accepts a element and returns its associated key
  • skip_elements (int) – number of elements to skip at the start

list – a list of the grouped elements by key, mode='r')

New in version 0.1.12.

Changed in version 0.3.4: using, always in binary mode

Opens a file using the extension as a guide to which module to use.


Unicode makes for a slower .translate method in Python2, so it’s best to use the open builtin.

  • file_name (str) – file name
  • mode (str) – mode used to open the file

file handle

Return type:



UnsupportedFormat – if the module to open the file is not available, name_mask, write_func, num_files=2)

New in version 0.1.13.

Splits the writing of a number of records in a series of files. The name_mask is used as template for the file names. A string like “split-files-{0}” can be specified and the function applies format with the index of the pieces.

  • records (iterable) – an iterable that returns a object to be saved
  • name_mask (str) – a string used as template for the output file names on which the function applies string.format()
  • write_func (func) – a function that is called to write to the files. It needs to accept a file handles as first argument and the record returned by records as the second argument
  • num_files (int) – the number of files to split the records