Running Locally

Make predictions on-device.

Function has experimental support for creating prediction functions that run entirely on the local device. These are called edge predictors, and enable building highly interactive, cross-platform functionality.

Creating the Predictor

Creating an edge predictor follows the same steps as cloud predictors. Create a predictor.ipynb notebook and add the following code cell:

from math import pi
def predict (radius: float) -> float:
Compute the area of a circle given its radius.
return pi * radius ** 2

Now, we can provision the predictor on Function in order to run it on any device, fully locally:

# Open a terminal and run the following command:
fxn create @username/area predictor.ipynb --edge --overwrite

Make sure to replace @username with your Function username.

The --edge flag creates an edge predictor instead of a cloud predictor.

There is absolutely no difference in your code when making a prediction with an edge predictor vs. a cloud predictor. Our client libraries will handle all the details for you.

Current Limitations

Edge predictors are currently in alpha. As such, there are major limitations compared to cloud predictors:

  1. Only a very limited subset of Python is supported. We will be expanding support for more Python language features over time. See GitHub
  2. For security, all file I/O is restricted. Predictors must only operate on data explicitly passed in as an argument.

Technical Considerations

At its core, Function is a Python compiler and optimizer. Given a Python function, we transpile the function to C++ then cross-compile for the following platforms:

Platform Architectures Notes
Android armeabi-v7a, arm64-v8a, x86, x86_64 None
iOS arm64 Device-only
macOS arm64, x86_64 None
Web wasm32 None
Windows x86_64 None

To transpile Python to C++, we map each Python operation to a set of equivalent implementations in C++. We then take the cartesian product of these mappings to generate a set of C++ functions for a single Python function.

Our approach means that we often emit hundreds-to-thousands of equivalent C++ functions for a single Python function.

The combinatorics involved require us to take a different approach to performance optimization. We compile all emitted C++ functions, serve them to users, and use telemetry data to choose the best C++ function for a unique device.

Performance optimization is a purely empirical science. We're taking this concept of auto-tuning code to the max.

See Function.h and Dialect.hpp for more information about on-device predictions.

Creating Predictors