dymaxionlabs package

Submodules

dymaxionlabs.files module

class dymaxionlabs.files.File(name, path, metadata, **extra_attributes)[source]

Bases: object

The File class represents files stored in Dymaxion Labs.

Files are owned by the authenticated user.

Parameters:
  • name (str) – file name
  • path (str) – file path in storage
  • metadata (dict) – file metadata
  • extra_attributes (dict) – extra attributes from API endpoint
classmethod all(path='')[source]

Fetches all files found in path.

Glob patterns are allowed, to search recursively in directories:

File.all("foo/b*/images/*.tif")
#=> [<dymaxionlabs.file.File name="foo/bar/images/01.tif">, ...]
Parameters:path (str) – path glob pattern (default: “”)
Returns:a list of File of files found in path
Return type:list
base_path = '/storage'
delete()[source]

Deletes the file in storage.

Returns:True if file was succesfully deleted
Return type:bool
download(output_dir='.')[source]

Downloads the file and stores it on output_dir.

If output_dir does not exist, it will be created.

Parameters:output_dir (str) – directory path where file will be stored
classmethod get(path, raise_error=True)[source]

Gets a specific file in path.

Parameters:
  • path (str) – file path
  • raise_error (bool) – If False, do not raise NotFoundError if file not found
Return type:

File

tiling(output_path, tile_size=500)[source]

Starts an image tiling job on the current file.

By default, tiles are 500x500, but you can set the tile_size to a different size.

output_path should be a directory path in storage where tiles will be generated.

Parameters:
  • output_path (str) – tiles output directory
  • tile_size (int) – tile size (default: 500)
Returns:

a Task with info about the new tiling job

Raises:

RuntimeError if output path is None

Return type:

Task

classmethod upload(input_path, storage_path='', chunk_size=None)[source]

Uploads a file to storage

Parameters:
  • input_path (str) – path of local file to upload
  • storage_path (str) – destination path in storage
  • chunk_size (int) – size (in MB) of chunks for resumable uploading
Raises:

FileNotFoundError

Returns:

uploaded file

Return type:

File

dymaxionlabs.models module

class dymaxionlabs.models.Estimator(*, uuid, name, classes, estimator_type, metadata, image_files, configuration, training_tasks=[], prediction_tasks=[], **extra_attributes)[source]

Bases: object

The Estimator class represents a Model that can be trained to solve different kinds of tasks, like object detection or classification.

To instance an Estimator, you should use one of the following class methods: all(), get(), or create().

Parameters:
  • uuid (str) – internal id
  • name (str) – name
  • classes (list) – list of labels/classes
  • estimator_type (str) – type of estimator (i.e. )
  • image_files (list) – list of associated image files used for training
  • metadata (dict) – user metadata
  • configuration (dict) – estimator configuration
  • extra_attributes (dict) – extra attributes from API endpoint
TYPES = {'object_detection': 'OD', 'segmentation': 'SG'}
add_image(*images)[source]

Adds an image File to the estimator, for training.

Parameters:images – one or multiple image File instances
Returns:itself
add_labels_for(vector_file, image_file, label=None, label_property=None)[source]

Adds labels from an already uploaded vector_file related to image_file and tags these labels like label or with the value of label_property.

Parameters:
  • vector_file (File) – a GeoJSON file with polygons representing annotated objects
  • image_file (File) – the corresponding image file to be annotated.
  • label (str) – the class/label to use for the annotations
  • label_property (str) – the property that stores the label value of each annotation
Returns:

a dict with info about the new AnnotationJob

classmethod all()[source]

Fetches all estimators in the current project.

base_path = '/estimators'
clone()[source]

Clone an estimator.

Return type:Estimator
classmethod create(*, name, type, classes=[], metadata=None, configuration={})[source]

Creates a new Estimator named name of type with classes as labels with training_hours hour of training job.

A metadata dictionary can be added via the metadata parameter. A configuration dictionary can be added via the configuration parameter.

Parameters:
  • str (type) – A name to identify the estimator
  • str – Type of estimator (“object_detection”)
  • list (classes) – Optional list of classes/labels of objects
  • dict (configuration) – Optional metadata dictionary to store extra attributes
  • dict – Optional configuration dictionary for training
Return type:

Estimator

delete()[source]

Deletes the estimator.

Returns:True if estimator was succesfully deleted.
describe_annotations()[source]

Describe annotations.

Return type:dict
classmethod get(uuid)[source]

Gets an estimator identified by uuid.

Parameters:str (uuid) – Estimator UUID
Return type:Estimator
latest_prediction_task

Returns the most recent prediction task.

Returns:a Task instance with information about the latest prediction task
Return type:Task
latest_training_task

Returns the most recent training task.

Returns:a Task instance with information about the latest training task
Return type:Task
predict_files(tile_dirs, confidence=0.2)[source]

Starts a prediction job with the tile images stored in tile_dirs.

Results are filtered through a confidence threshold.

Parameters:
  • tile_dirs (list) – list of directories with tiles to predict
  • confidence (float) – confidence value for results
Returns:

a dict with info about the new PredictionJob

save()[source]

Updates the estimator.

Returns:itself
train()[source]

Start a training job using this estimator.

It will build a custom model based on all the images from the estimator and the associates annotations.

Returns:a dict with info about the new TrainingJob

dymaxionlabs.tasks module

class dymaxionlabs.tasks.Task(*, id, state, name, args, kwargs, created_at, updated_at, finished_at, metadata, duration, estimated_duration, error, **extra_attributes)[source]

Bases: object

A Task represents a long running job.

Currently, Tasks are used to query about the status of a model training or prediction job (see Estimator.train() and Estimator.predict_files()), or an image tiling process (see File.tiling()).

Parameters:
  • id (int) – internal id
  • state (str) – job state
  • name (str) – task name
  • args (list) – args
  • kwargs (dict) – kwargs
  • created_at (datetime) – created datetime
  • updated_at (datetime) – updated datetime
  • finished_at (datetime) – finished datetime
  • metadata (dict) – job metadata
  • duration (int) – duration (in seconds)
  • estimated_duration (int) – estimated duration (in seconds)
  • error (str) – latest error message (if task failed)
  • extra_attributes (dict) – extra attributes from API response
classmethod all(path='*')[source]

Fetches all tasks.

Returns:a list of Task
Return type:list
base_path = '/tasks'
cancel()[source]

Cancel the task if it is possible.

Returns:itself
Return type:Task
download_artifacts(output_dir='.')[source]

Downloads output artifacts in a compressed Zip file, and stores it on output_dir.

If output_dir does not exist, it will be created.

Parameters:output_dir (str) – directory path where file will be stored
Returns:path to the artifacts zip file
Return type:str
export_artifacts(storage_dir)[source]

Stores output artifacts in storage_dir.

Parameters:storage_dir (str) – directory path where file will be stored
classmethod get(id)[source]

Gets an task identified by id.

Parameters:int (id) – Task id
Returns:the specified Task instance
Return type:Task
has_artifacts()[source]

Checks if completed task has generated output artifacts.

Returns:True if it has output artifacts, False if not.
Return type:bool
is_running()[source]

Decides whether a task is running or not, and update the task attributes if is necesary.

Return type:bool
list_artifacts()[source]

Returns a list of the generated output artifacts.

Returns:list of file paths (strings)
Rtpe:list
refresh()[source]

Refreshes attributes of the task.

Returns:itself
Return type:Task

dymaxionlabs.upload module

class dymaxionlabs.upload.CustomResumableUpload(upload_url, chunk_size, headers=None)[source]

Bases: google.resumable_media.requests.upload.ResumableUpload

initiate(stream, metadata, content_type, resumable_url, total_bytes=None, stream_final=True)[source]

Initiate a resumable upload.

By default, this method assumes your stream is in a “final” state ready to transmit. However, stream_final=False can be used to indicate that the size of the resource is not known. This can happen if bytes are being dynamically fed into stream, e.g. if the stream is attached to application logs.

If stream_final=False is used, chunk_size bytes will be read from the stream every time transmit_next_chunk() is called. If one of those reads produces strictly fewer bites than the chunk size, the upload will be concluded.

Parameters:
  • transport (Session) – A requests object which can make authenticated requests.
  • stream (IO[bytes]) – The stream (i.e. file-like object) that will be uploaded. The stream must be at the beginning (i.e. stream.tell() == 0).
  • metadata (Mapping[str, str]) – The resource metadata, such as an ACL list.
  • content_type (str) – The content type of the resource, e.g. a JPEG image has content type image/jpeg.
  • total_bytes (Optional[int]) – The total number of bytes to be uploaded. If specified, the upload size will not be determined from the stream (even if stream_final=True).
  • stream_final (Optional[bool]) – Indicates if the stream is “final” (i.e. no more bytes will be added to it). In this case we determine the upload size from the size of the stream. If total_bytes is passed, this argument will be ignored.
Returns:

The HTTP response returned by transport.

Return type:

Response

transmit_next_chunk()[source]

Transmit the next chunk of the resource to be uploaded.

If the current upload was initiated with stream_final=False, this method will dynamically determine if the upload has completed. The upload will be considered complete if the stream produces fewer than chunk_size bytes when a chunk is read from it.

In the case of failure, an exception is thrown that preserves the failed response:

>>> error = None
>>> try:
...     upload.transmit_next_chunk(transport)
... except resumable_media.InvalidResponse as caught_exc:
...     error = caught_exc
...
>>> error
InvalidResponse('Request failed with status code', 400,
                'Expected one of', <HTTPStatus.OK: 200>, 308)
>>> error.response
<Response [400]>
Parameters:transport (Session) – A requests object which can make authenticated requests.
Returns:The HTTP response returned by transport.
Return type:Response
Raises:InvalidResponse – If the status code is not 200 or 308.

dymaxionlabs.utils module

exception dymaxionlabs.utils.BadRequestError[source]

Bases: Exception

exception dymaxionlabs.utils.InternalServerError[source]

Bases: Exception

exception dymaxionlabs.utils.NotFoundError[source]

Bases: Exception

class dymaxionlabs.utils.TimeoutHTTPAdapter(*args, **kwargs)[source]

Bases: requests.adapters.HTTPAdapter

send(request, **kwargs)[source]

Sends PreparedRequest object. Returns Response object.

Parameters:
  • request – The PreparedRequest being sent.
  • stream – (optional) Whether to stream the request content.
  • timeout (float or tuple or urllib3 Timeout object) – (optional) How long to wait for the server to send data before giving up, as a float, or a (connect timeout, read timeout) tuple.
  • verify – (optional) Either a boolean, in which case it controls whether we verify the server’s TLS certificate, or a string, in which case it must be a path to a CA bundle to use
  • cert – (optional) Any user-provided SSL certificate to be trusted.
  • proxies – (optional) The proxies dictionary to apply to the request.
Return type:

requests.Response

dymaxionlabs.utils.fetch_from_list_request(path, params={})[source]

Fetches all entities from a paginated result

dymaxionlabs.utils.get_api_key()[source]

Get current API Key from environment

dymaxionlabs.utils.get_api_url()[source]

Get current API URL from environment

dymaxionlabs.utils.request(method, path, body=None, files=None, params={}, headers={}, binary=False, parse_response=True)[source]

Makes an HTTP request to the API

Module contents

Package to integrate the DymaxionLabs’s funcionality:

  • Upload images
  • Predict imagenes based in object detection models
  • Download results