Utilities

Some common utility functions.

Remote

Functions related to networking.

Containing functions to download files from the internet. Supports download zipped files from Google Drive.

exception omnizart.remote.GDFileAccessLimited

Custom exception on failing to download GD file.

This exception is raised when the GD file is overly accessed during a certain period.

omnizart.remote.SIZE_MAPPING = [(1, 'B'), (1024, 'KB'), (1048576, 'MB'), (1073741824, 'GB'), (1099511627776, 'TB')]

Mapping bytes to human-readable size unit.

omnizart.remote.download(url, file_length=None, save_path='./', save_name=None, cookie_file=None, unzip=False)

Download file from the internet.

Download file from the remote URL, with progress visualization and dynamic downloading rate adjustment. Uses pure python built-in packages, no additional package requirement.

Parameters

url: URL

The file download url.

file_length: float

In bytes. If the length can’t be retrieved from the response header, but can be obtained by other approaches, you can explicitly specify the length for progress visualization.

save_path: Path

The path to store the donwloaded file.

save_name: str

Explicitly specify the file name to be stored. If not given, default to parse the name from the given url.

cookie_file: Path

Path to the cookie file. Suitable for stateful download (e.g. Google Drive).

unzip: bool

Whether to unzip (decompress) the downloaded file (assumed zipped). Will not delete the original downloaded file.

Returns

path: Path

The absolute path to the downloaded/extracted folder/file.

omnizart.remote.download_large_file_from_google_drive(url, file_length=None, save_path='./', save_name=None, unzip=False)

Google Drive file downloader.

Download function dedicated for Google Drive files. Mainly to deal with download large files and the confirmation page.

Parameters

url: URL

Could be a full google drive download url or the file ID.

file_length: float

In bytes. If the length can’t be retrieved from the response header, but can be obtained by other approaches, you can explicitly specify the length for progress visualization.

save_path: Path

Path to store the downloaded file.

save_name: str

Explicitly specify the file name to be stored. If not given, default to parse the name from the given url.

unzip: bool

Whether to unzip (decompress) the downloaded file (assumed zipped). Will not delete the original downloaded file.

Returns

path: Path

The absolute path to the downloaded/extracted folder/file.

omnizart.remote.format_byte(size, digit=2)

Format the given byte size into human-readable string.

Utility Functions

Various utility functions for this project.

class omnizart.utils.LazyLoader(local_name, parent_module_globals, name, warning=None)

Lazily import a module, mainly to avoid pulling in large dependencies.

Original implementations are from tensorflow [1].

References

1

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/util/lazy_loader.py

omnizart.utils.aggregate_f0_info(pred, t_unit)

Aggregation function of F0 contour.

Aggregate the repeated frequencies in continuous frames into higher-level representation, with information of start time, end time, and frequency.

Parameters

pred: 1D numpy array

Array that contains F0 information (Hz) in frame-level.

t_unit: float

Time unit of each frame.

Returns

agg_f0: list[dict]

Aggregated F0 information. Each element in the list represents a single freqeuncy with start time, end time, and frequency recorded in dict.

omnizart.utils.camel_to_snake(string)

Convert a camel case to snake case

omnizart.utils.ensure_path_exists(path)

omnizart.utils.get_filename(path)

omnizart.utils.get_logger(name=None, level='warn')

Get the logger for printing informations.

Used for layout the information of various stages while executing the program. Set the environment variable LOG_LEVEL to change the default level.

Parameters

name: str

Name of the logger.

level: {‘debug’, ‘info’, ‘warn’, ‘warning’, ‘error’, ‘critical’}

Level of the logger. The level ‘warn’ and ‘warning’ are different. The former is the default level and the actual level is set to logging.INFO, and for ‘warning’ which will be set to true logging.WARN level. The purpose behind this design is to categorize the message layout into several different formats.

omnizart.utils.json_serializable(key_path='./', value_path='./')

Class-level decorator for making a class json-serializable object.

This decorator makes a class serializable as a json object. All attributes will be outputed as a key-value pair recursively if there are also attributes that is type of json-serializable class. Inject two functions to the decorated class: from_json, to_json.

Parameters

key_path: Path

Access sub-object according to the path. E.g. Assume you have a dictionary: d = {“a”: {“b”: {“c”: “d”}}}, and the path: p = “a/b”, then the sub-object after being propagated would be {“c”: “d”}.

value_path: Path

The relative path to the key_path. This parameter makes you able to access the value that is not at the same level as the key. E.g. assume you have a sub-object after the propagation of key_path: d = {“a”: {“b”: {“c”: “d”}}}, and the value_path: vp = “a/b/c”, the corresponding value of the key should be “d”.

See also

tests.test_utils.test_normal_serializable

Unit test of this decorator.

Notes

The attributes should be defined inside ‘__init__’, or those defined as class attributes will not be serialized as thery are invisible for __dict__ attribute.

Examples

>>> @json_serializable()
    class A:
        def __init__(self):
            self.a = 1
            self.b = "Hello"
            self.c = [10, 20, 30]

    @json_serializable()
    class B:
        def __init__(self):
            self.a_instance = A()
            self.d = "World"

>>> inst = B()
>>> inst.to_json()
{
    "d": "World",
    "a_instance": {
        "a": 1,
        "b": "Hello",
        "c": [10, 20, 30]
    }
}

omnizart.utils.parallel_generator(func, input_list, max_workers=2, use_thread=False, chunk_size=None, timeout=600, **kwargs)

omnizart.utils.resolve_dataset_type(dataset_path, keywords)

omnizart.utils.snake_to_camel(string)

Convert a snake case to camel case

omnizart.utils.synth_midi(midi_path, output_path, sampling_rate=44100, sf2_path='/home/runner/work/omnizart/omnizart/omnizart/resource/soundfonts.sf2')

Synthesize MIDI into wav audio.