Skip to content

Commit f2ed614

Browse files
committed
added description for python backend
1 parent 73dde96 commit f2ed614

File tree

1 file changed

+11
-0
lines changed

1 file changed

+11
-0
lines changed

Quick_Deploy/JAX/README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,17 @@
3030

3131
This README showcases how to deploy a simple ResNet model on Triton Inference Server. While Triton doesn't yet have a dedicated JAX backend, JAX/Flax models can be deployed using [Python Backend](https://github.com/triton-inference-server/python_backend). If you are new to Triton, it is recommended to watch this [getting started video](https://www.youtube.com/watch?v=NQDtfSi5QF4) and review [Part 1](https://github.com/triton-inference-server/tutorials/tree/main/Conceptual_Guide/Part_1-model_deployment) of the conceptual guide before proceeding. For the purposes of demonstration, we are using a pre-trained model provided by [flaxmodels](https://github.com/matthias-wright/flaxmodels).
3232

33+
Before diving into the specifics execution, an understanding of the underlying structure is needed. To use a JAX or a Flax model, the recommended path for this is using a ["Python Model"](https://github.com/triton-inference-server/python_backend#python-backend). Python models in Triton are basically classes with three Triton-specific functions: `initialize`, `execute` and `finalize`. Users can customize this class to serve any python function they write or any model they want as long as it can be loaded in python runtime. The `initialize` function runs when the python model is loaded into memory, and the `finalize` function runs when the model is unloaded from memory. Both of these functions are optional to define. For the purposes of this example, we will use the `initialize` and the `execute` functions to load and run(respectively) a `resnet18` model.
34+
35+
```
36+
class TritonPythonModel:
37+
def initialize(self, args):
38+
...
39+
40+
def execute(self, requests):
41+
...
42+
```
43+
3344
## Step 1: Set Up Triton Inference Server
3445

3546
To use Triton, we need to build a model repository. The structure of the repository as follows:

0 commit comments

Comments
 (0)