Filesystem manipulation¶

The components of datreant documented here are those designed for working directly with filesystem objects, namely directories and files.

Tree¶

The class datreant.Tree is an interface to a directory in the filesystem.

class datreant.Tree(dirpath)¶

A directory.

abspath¶: Absolute path of self.path.

children(hidden=False)¶

Return a View of all files and directories in this Tree.

Parameters:	hidden (bool) – If True, include hidden files and directories.
Returns:	A View with files and directories in this Tree as members.
Return type:	View

draw(depth=None, hidden=False)¶

Print an ASCII-fied visual of the tree.

Parameters:	depth (int, optional) – Maximum directory depth to display. `None` indicates no limit. hidden (bool) – If True, show hidden files and directories.

exists¶: Check existence of this path in filesystem.

glob(pattern)¶

Return a View of all child Leaves and Trees matching given globbing pattern.

Parameters:	pattern – globbing pattern to match files and directories with

leafloc¶

Get Leaf at relative path.

Use with getitem syntax, e.g. .treeloc['some name']

Allowed inputs are: - A single name - A list or array of names

If the given path resolves to an existing directory, then a ValueError will be raised.

leaves(hidden=False)¶

Return a View of the files in this Tree.

Parameters:	hidden (bool) – If True, include hidden files.
Returns:	A View with files in this Tree as members.
Return type:	View

loc¶

Get Tree/Leaf at relative path.

Use with getitem syntax, e.g. .loc['some name']

Allowed inputs are: - A single name - A list or array of names

If directory/file does not exist at the given path, then whether a Tree or Leaf is given is determined by the path semantics, i.e. a trailing separator (“/”).

Using e.g. Tree.loc['some name'] is equivalent to doing Tree['some name']. .loc is included for parity with View and Bundle API semantics.

make()¶

Make the directory if it doesn’t exist. Equivalent to makedirs().

Returns:	This Tree.
Return type:	Tree

makedirs()¶

Make all directories along path that do not currently exist.

Returns:	This Tree.
Return type:	Tree

name¶: Basename for this path.

parent¶: Parent directory for this path.

path¶: Filesystem path as a pathlib2.Path.

relpath¶: Relative path of self.path from current working directory.

sync(other, mode='upload', compress=True, checksum=True, backup=False, dry=False, include=None, exclude=None, overwrite=False, rsync_path='/usr/bin/rsync')¶

Synchronize directories using rsync.

Parameters:	other (str or Tree) – Other end of the sync, can be either a path or another Tree. mode (str) – Either `"upload"` if uploading to other, or `"download"` if downloading from other

The other options are described in the datreant.rsync.rsync() documentation.

treeloc¶

Get Tree at relative path.

Use with getitem syntax, e.g. .treeloc['some name']

Allowed inputs are: - A single name - A list or array of names

If the given path resolves to an existing file, then a ValueError will be raised.

trees(hidden=False)¶

Return a View of the directories in this Tree.

Parameters:	hidden (bool) – If True, include hidden directories.
Returns:	A View with directories in this Tree as members.
Return type:	View

walk(topdown=True, onerror=None, followlinks=False)¶

Walk through the contents of the tree.

For each directory in the tree (including the root itself), yields a 3-tuple (dirpath, dirnames, filenames).

Parameters:	topdown (Boolean, optional) – If False, walks directories from the bottom-up. onerror (function, optional) – Optional function to be called on error. followlinks (Boolean, optional) – If False, excludes symbolic file links.
Returns:	Wrapped scandir.walk() generator yielding datreant objects
Return type:	generator

Leaf¶

The class datreant.Leaf is an interface to a file in the filesystem.

class datreant.Leaf(filepath)¶

A file in the filesystem.

abspath¶: Absolute path.

exists¶: Check existence of this path in filesystem.

make()¶

Make the file if it doesn’t exit. Equivalent to touch().

Returns:	leaf – this leaf
Return type:	Leaf

makedirs()¶

Make all directories along path that do not currently exist.

Returns:	leaf – this leaf
Return type:	Leaf

name¶: Basename for this path.

parent¶: Parent directory for this path.

path¶: Filesystem path as a pathlib2.Path.

read(size=None)¶

Read file, or up to size in bytes.

Parameters:	size (int) – extent of the file to read, in bytes

relpath¶: Relative path from current working directory.

touch()¶: Make file if it doesn’t exist.

View¶

The class datreant.View is an ordered set of Trees and Leaves. It allows for convenient operations on its members as a collective, as well as providing mechanisms for filtering and subselection.

class datreant.View(*vegs)¶

An ordered set of Trees and Leaves.

Parameters:	vegs (Tree, Leaf, or list) – Trees and/or Leaves to be added, which may be nested lists of Trees and Leaves. Trees and Leaves can be given as either objects or paths.

abspaths¶: List of absolute paths for the members in this View.

children(hidden=False)¶

Return a View of all files and directories within the member Trees.

Parameters:	hidden (bool) – If True, include hidden files and directories.
Returns:	A View giving the files and directories in the member Trees.
Return type:	View

draw(depth=None, hidden=False)¶

Print an ASCII-fied visual of all member Trees.

Parameters:	depth (int) – Maximum directory depth to display. `None` indicates no limit. hidden (bool) – If False, do not show hidden files; hidden directories are still shown if they contain non-hidden files or directories.

exists¶: List giving existence of each member as a boolean.

glob(pattern)¶

Return a View of all child Leaves and Trees of members matching given globbing pattern.

Parameters:	pattern (string) – globbing pattern to match files and directories with

globfilter(pattern)¶

Return a View of members that match by name the given globbing pattern.

Parameters:	pattern (string) – globbing pattern to match member names with

leafloc¶

Get a View giving Leaf at path relative to each Tree in collection.

Use with getitem syntax, e.g. .loc['some name']

Allowed inputs are: - A single name - A list or array of names

If the given path resolves to an existing directory for any Tree, then a ValueError will be raised.

leaves(hidden=False)¶

Return a View of the files within the member Trees.

Parameters:	hidden (bool) – If True, include hidden files.
Returns:	A View giving the files in the member Trees.
Return type:	View

loc¶

Get a View giving Tree/Leaf at path relative to each Tree in collection.

Use with getitem syntax, e.g. .loc['some name']

Allowed inputs are: - A single name - A list or array of names

If directory/file does not exist at the given path, then whether a Tree or Leaf is given is determined by the path semantics, i.e. a trailing separator (“/”).

make()¶

Make the Trees and Leaves in this View if they don’t already exist.

Returns:	This View.
Return type:	View

map(function, processes=1, **kwargs)¶

Apply a function to each member, perhaps in parallel.

A pool of processes is created for processes > 1; for example, with 40 members and processes=4, 4 processes will be created, each working on a single member at any given time. When each process completes work on a member, it grabs another, until no members remain.

kwargs are passed to the given function when applied to each member

Parameters:	function (function) – Function to apply to each member. Must take only a single Treant instance as input, but may take any number of keyword arguments. processes (int) – How many processes to use. If 1, applies function to each member in member order in serial.
Returns:	results – List giving the result of the function for each member, in member order. If the function returns `None` for each member, then only `None` is returned instead of a list.
Return type:	list

memberleaves¶: A View giving only members that are Leaves (or subclasses).

membertrees¶: A View giving only members that are Trees (or subclasses).

names¶: List the basenames for the members in this View.

parents()¶

Return a View of the parent directories for each member.

Because a View functions as an ordered set, and some members of this collection may share a parent, the View of parents may contain fewer elements than this collection.

relpaths¶: List of relative paths from the current working directory for the members in this View.

treeloc¶

Get a View giving Tree at path relative to each Tree in collection.

Use with getitem syntax, e.g. .loc['some name']

Allowed inputs are: - A single name - A list or array of names

If the given path resolves to an existing file for any Tree, then a ValueError will be raised.

trees(hidden=False)¶

Return a View of the directories within the member Trees.

Parameters:	hidden (bool) – If True, include hidden directories.
Returns:	A View giving the directories in the member Trees.
Return type:	View