PyTorch Autotuning API¶
TorchApp(app_name, module, tune_dataloader, test_dataloader, knobs, tensor_to_qos, combine_qos=<function mean>, target_device=None, torch_device='cpu', model_storage_folder=None)¶
Adaptor for approximable PyTorch Modules with tensor output.
A TorchApp stores the PyTorch Module, datasets for tuning and calibration, set of available
TorchApproxKnobeach of which may be applied to some layer in the Module, and the quality of service (QoS) metric of application (e.g., accuracy). It provides empirical tuning and predictive tuning capability, automatically supporting
app_name – Name of the application, which is used as an identifier in tuning sessions, etc.
module – The PyTorch module to tune.
tune_dataloader – A
torch.utils.data.Datasetdataset to use as inputs to module during tuning.
test_dataloader – A
torch.utils.data.Datasetdataset used for QoS testing (see
knobs – A set of
TorchApproxKnobto be considered. Each knob has an
is_applicable()method which is used to determine which layer it can apply to.
approxes.get_knobs_from_filereturns a set of builtin knobs that will exactly fit here.
tensor_to_qos – QoS metric function which computes QoS from the module’s output.
torchutil.accuracycomputes the classification accuracy which can be applied here.
combine_qos – A function to combine each batch’s QoS into one value. When QoS is Classification Accuracy, this will most likely be
numpy.mean(which is the default value).
target_device – The target device that this application should be tuned on.
torch_device – The PyTorch device where the model inference is run on. This device should be able to run the implementations of the knobs available for this app on
model_storage_folder – A folder to store the serialized QoS models into.
QoSModelP1will be serialized into
model_storage_folder / "p1.pkl", and
model_storage_folder / "p2.json".
Constructs and returns a set of
TorchApproxKnobfrom a knob declaration file.
default_knob_filepoints to a file that is contained in the predtuner package, so just calling
get_knobs_from_file()should provide a set of predefined knobs already.
extra_name_to_class (Dict[str, Type[TorchApproxKnob]]) – a mapping from the name of the approximation to the class (implementation) of the approximation. If not given, only the builtin approximations will be considered when parsing the declaration file.
- Return type
The “classification accuracy” metric (return value is between 0 and 100).
Defining New Approximation Knobs¶
TorchApproxKnob(name, devices=None, baseline_priority=None)¶
Defines an approximation knob that knows its own expected speedup ratio and what Modules it can apply to, and can be applied to a torch.nn.Module to return an approximated Module.
Applies knob to a Module and returns an approximated Module.
Returns true if approx knob does not contain randomness.
The speedup this knob is expected to provide. Used for cost prediction.