Mlflow transformer

Deploy any Transformer–based model to DataBricks MLFlow¶

This notebook shows how to deploy transformer-based model to DataBricks MLFlow and get prediction via batch and online HTTP serving API.

In [0]:

%pip install sentence-transformers 
%pip install cloudpickle==2.0.0

%pip install sentence-transformers %pip install cloudpickle==2.0.0

Note: you may need to restart the kernel using dbutils.library.restartPython() to use updated packages.
Requirement already satisfied: sentence-transformers in /databricks/python3/lib/python3.10/site-packages (2.2.2)
Requirement already satisfied: tqdm in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (4.64.1)
Requirement already satisfied: huggingface-hub>=0.4.0 in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (0.14.1)
Requirement already satisfied: numpy in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (1.23.5)
Requirement already satisfied: torch>=1.6.0 in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (2.0.1+cpu)
Requirement already satisfied: scipy in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (1.10.0)
Requirement already satisfied: nltk in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (3.7)
Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (4.31.0)
Requirement already satisfied: sentencepiece in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (0.1.99)
Requirement already satisfied: torchvision in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (0.15.2+cpu)
Requirement already satisfied: scikit-learn in /databricks/python3/lib/python3.10/site-packages (from sentence-transformers) (1.1.1)
Requirement already satisfied: packaging>=20.9 in /databricks/python3/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers) (22.0)
Requirement already satisfied: pyyaml>=5.1 in /databricks/python3/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers) (6.0)
Requirement already satisfied: filelock in /databricks/python3/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers) (3.9.0)
Requirement already satisfied: fsspec in /databricks/python3/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers) (2022.11.0)
Requirement already satisfied: requests in /databricks/python3/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers) (2.28.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /databricks/python3/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers) (4.4.0)
Requirement already satisfied: sympy in /databricks/python3/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers) (1.11.1)
Requirement already satisfied: networkx in /databricks/python3/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers) (2.8.4)
Requirement already satisfied: jinja2 in /databricks/python3/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers) (3.1.2)
Requirement already satisfied: regex!=2019.12.17 in /databricks/python3/lib/python3.10/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (2022.7.9)
Requirement already satisfied: safetensors>=0.3.1 in /databricks/python3/lib/python3.10/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (0.3.3)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /databricks/python3/lib/python3.10/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers) (0.13.3)
Requirement already satisfied: joblib in /databricks/python3/lib/python3.10/site-packages (from nltk->sentence-transformers) (1.2.0)
Requirement already satisfied: click in /databricks/python3/lib/python3.10/site-packages (from nltk->sentence-transformers) (8.0.4)
Requirement already satisfied: threadpoolctl>=2.0.0 in /databricks/python3/lib/python3.10/site-packages (from scikit-learn->sentence-transformers) (2.2.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /databricks/python3/lib/python3.10/site-packages (from torchvision->sentence-transformers) (9.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in /databricks/python3/lib/python3.10/site-packages (from jinja2->torch>=1.6.0->sentence-transformers) (2.1.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /databricks/python3/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (1.26.14)
Requirement already satisfied: certifi>=2017.4.17 in /databricks/python3/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in /databricks/python3/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in /databricks/python3/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers) (2.0.4)
Requirement already satisfied: mpmath>=0.19 in /databricks/python3/lib/python3.10/site-packages (from sympy->torch>=1.6.0->sentence-transformers) (1.2.1)
Note: you may need to restart the kernel using dbutils.library.restartPython() to use updated packages.
Note: you may need to restart the kernel using dbutils.library.restartPython() to use updated packages.
Requirement already satisfied: cloudpickle==2.0.0 in /databricks/python3/lib/python3.10/site-packages (2.0.0)
Note: you may need to restart the kernel using dbutils.library.restartPython() to use updated packages.

In [0]:

import time
import os
import logging

os.environ["DATABRICKS_TOKEN"] = "<your_token_here>"

import cloudpickle
import requests
import pandas as pd
import mlflow
import mlflow.pytorch
from mlflow.models import infer_signature
from mlflow.tracking.client import MlflowClient
from mlflow.entities.model_registry.model_version_status import ModelVersionStatus
from mlflow.utils.environment import _mlflow_conda_env

import sentence_transformers
from sentence_transformers import SentenceTransformer

import time import os import logging os.environ["DATABRICKS_TOKEN"] = "" import cloudpickle import requests import pandas as pd import mlflow import mlflow.pytorch from mlflow.models import infer_signature from mlflow.tracking.client import MlflowClient from mlflow.entities.model_registry.model_version_status import ModelVersionStatus from mlflow.utils.environment import _mlflow_conda_env import sentence_transformers from sentence_transformers import SentenceTransformer

Wrap Transformer model into MLFlow api and log into to MLFlow.¶

Before deploying the model we need to make it compatible with MLFlow.
This is done via wrapping a model into mlflow.pyfunc.PythonModel subclass, create conda_env and log model to MLFlow.

In [0]:

class TransformerWrapper(mlflow.pyfunc.PythonModel):
    def __init__(self):
        self.model = SentenceTransformer("sentence-transformers/all-MiniLM-L12-v2")

    def predict(self, context, model_input):
        inputs = model_input["text"]
        return self.model.encode(inputs)


with mlflow.start_run() as run:
    pyfunc_model = TransformerWrapper()

    input_schema = mlflow.types.Schema([
        mlflow.types.ColSpec(name="text", type=mlflow.types.DataType.string)
    ])
    output_schema = mlflow.types.Schema([
        mlflow.types.ColSpec(name="embedding", type=mlflow.types.DataType.float)
    ])
    signature = mlflow.models.signature.ModelSignature(
        inputs=input_schema,
        outputs=output_schema,
    )

    conda_env =  _mlflow_conda_env(
        additional_conda_deps=None,
        additional_pip_deps=[
            "cloudpickle=={}".format(cloudpickle.__version__), 
            "torch==1.13.1",
            "sentence-transformers=={}".format(sentence_transformers.__version__),
        ],
        additional_conda_channels=None,
    )

    model_name = "transformer-model"
    log_result = mlflow.pyfunc.log_model(
        model_name,
        python_model=pyfunc_model,
        signature=signature,
        conda_env=conda_env,
    )
    run_id = mlflow.active_run().info.run_id

class TransformerWrapper(mlflow.pyfunc.PythonModel): def __init__(self): self.model = SentenceTransformer("sentence-transformers/all-MiniLM-L12-v2") def predict(self, context, model_input): inputs = model_input["text"] return self.model.encode(inputs) with mlflow.start_run() as run: pyfunc_model = TransformerWrapper() input_schema = mlflow.types.Schema([ mlflow.types.ColSpec(name="text", type=mlflow.types.DataType.string) ]) output_schema = mlflow.types.Schema([ mlflow.types.ColSpec(name="embedding", type=mlflow.types.DataType.float) ]) signature = mlflow.models.signature.ModelSignature( inputs=input_schema, outputs=output_schema, ) conda_env = _mlflow_conda_env( additional_conda_deps=None, additional_pip_deps=[ "cloudpickle=={}".format(cloudpickle.__version__), "torch==1.13.1", "sentence-transformers=={}".format(sentence_transformers.__version__), ], additional_conda_channels=None, ) model_name = "transformer-model" log_result = mlflow.pyfunc.log_model( model_name, python_model=pyfunc_model, signature=signature, conda_env=conda_env, ) run_id = mlflow.active_run().info.run_id

Uploading artifacts:   0%|          | 0/5 [00:00<?, ?it/s]

Register model in MLFlow model registry¶

In [0]:

model_details = mlflow.register_model(model_uri=log_result.model_uri, name=model_name)

model_details = mlflow.register_model(model_uri=log_result.model_uri, name=model_name)

Registered model 'transformer-model' already exists. Creating a new version of this model...
2023/11/14 11:30:31 INFO mlflow.tracking._model_registry.client: Waiting up to 300 seconds for model version to finish creation. Model name: transformer-model, version 28
Created version '28' of model 'transformer-model'.

In [0]:

def wait_until_ready(model_name, model_version):
    client = MlflowClient()
    for _ in range(10):
        model_version_details = client.get_model_version(
            name=model_name,
            version=model_version,
        )
        status = ModelVersionStatus.from_string(model_version_details.status)
        print("Model status: %s" % ModelVersionStatus.to_string(status))
        if status == ModelVersionStatus.READY:
            break
        time.sleep(1)

wait_until_ready(model_details.name, model_details.version)

def wait_until_ready(model_name, model_version): client = MlflowClient() for _ in range(10): model_version_details = client.get_model_version( name=model_name, version=model_version, ) status = ModelVersionStatus.from_string(model_version_details.status) print("Model status: %s" % ModelVersionStatus.to_string(status)) if status == ModelVersionStatus.READY: break time.sleep(1) wait_until_ready(model_details.name, model_details.version)

Model status: READY

Move model to "Production" stage.¶

This step is optional. We can also use non-Production models. For example non-production models can be used via uris like models:/transformer-model/<model_version>, whereas production models can be used via odels:/transformer-model/production.

Archiving previous version is also optional. If you don't have previous version or want to deploy multiple version, feel free to comment this step out.

In [0]:

client = MlflowClient()

client.transition_model_version_stage(
  name=model_details.name,
  version=str(int(model_details.version) - 1),
  stage="Archived"
)

client.transition_model_version_stage(
    name=model_details.name,
    version=model_details.version,
    stage='Production',
)

client = MlflowClient() client.transition_model_version_stage( name=model_details.name, version=str(int(model_details.version) - 1), stage="Archived" ) client.transition_model_version_stage( name=model_details.name, version=model_details.version, stage='Production', )

Out[0]:

<ModelVersion: aliases=[], creation_timestamp=1699961431238, current_stage='Production', description='', last_updated_timestamp=1699961443390, name='transformer-model', run_id='730460b252fc449081aed709316d075b', run_link='', source='dbfs:/databricks/mlflow-tracking/150882855439300/730460b252fc449081aed709316d075b/artifacts/transformer-model', status='READY', status_message='', tags={}, user_id='6738268205827713', version='28'>

In [0]:

model_version_details = client.get_model_version(
    name=model_details.name,
    version=model_details.version,
)
model_version_details.current_stage

model_version_details = client.get_model_version( name=model_details.name, version=model_details.version, ) model_version_details.current_stage

Out[0]:

'Production'

Get production model¶

In [0]:

model = mlflow.pyfunc.load_model(f"models:/{model_name}/production")

model = mlflow.pyfunc.load_model(f"models:/{model_name}/production")

Downloading artifacts:   0%|          | 0/5 [00:00<?, ?it/s]

2023/11/14 11:30:53 WARNING mlflow.pyfunc: Detected one or more mismatches between the model's dependencies and the current Python environment:
 - torch (current: 2.0.1+cpu, required: torch==1.13.1)
To fix the mismatches, call `mlflow.pyfunc.get_model_dependencies(model_uri)` to fetch the model's environment and install dependencies using the resulting environment file.

Compute prediction¶

To make sure model works as expected

In [0]:

model.predict(pd.DataFrame({"text": ["Hello", "world"]}))

model.predict(pd.DataFrame({"text": ["Hello", "world"]}))

Out[0]:

array([[-7.49069974e-02, -1.44330626e-02,  4.86498363e-02,
        -2.71380823e-02, -4.30882052e-02, -1.47763416e-01,
         6.88330382e-02, -1.37038296e-02, -5.35686500e-02,
         2.69752834e-02, -6.28339546e-03, -3.98834497e-02,
         7.65673770e-03, -3.78089435e-02, -1.17557272e-02,
        -3.46409529e-02,  1.29357114e-01, -2.52778418e-02,
        -1.52099021e-02,  7.30297179e-04, -8.06888193e-02,
         2.69378778e-02, -9.87357870e-02, -3.41076069e-02,
        -2.70293076e-02, -7.32003748e-02,  5.08588217e-02,
        -1.72563046e-02,  7.28218630e-02, -1.23288728e-01,
        -1.09712547e-02,  1.54055757e-02,  1.61925405e-01,
        -3.71392109e-02, -1.72857773e-02,  2.61619966e-02,
         7.30954856e-02, -4.66715954e-02, -4.06146385e-02,
         5.91708459e-02,  1.65670700e-02,  7.34634837e-03,
        -4.89753895e-02,  6.89893588e-02,  4.28648442e-02,
        -7.57247433e-02, -1.50943510e-02,  4.19088565e-02,
         5.49149886e-02, -5.63954599e-02, -1.02605252e-02,
         1.87230646e-03, -7.89721403e-03,  5.32481931e-02,
         8.37045163e-02,  3.31942663e-02, -1.54266031e-02,
         6.00020960e-02,  2.80920584e-02,  2.25260723e-02,
         1.45363016e-02,  2.43535116e-02, -6.68660505e-03,
        -7.50119612e-02, -9.48023051e-03, -1.15375957e-02,
        -1.92416329e-02, -1.36323050e-02,  8.45558345e-02,
        -4.68212701e-02, -2.46619992e-03, -2.13895608e-02,
        -1.17153185e-03, -7.55426064e-02, -4.24708053e-02,
         1.41823133e-02,  1.82692539e-02,  4.45626974e-02,
        -6.14465214e-03, -1.16020227e-02,  3.17072682e-02,
        -9.66820419e-02,  4.12958264e-02,  3.99302617e-02,
         4.54294719e-02,  3.89315933e-02,  4.95433286e-02,
         4.89371456e-02,  3.06886323e-02,  2.03035139e-02,
        -6.82089105e-02,  3.90502512e-02,  1.89298261e-02,
        -1.60383247e-02, -1.06201079e-02,  3.29713300e-02,
         3.54809314e-02,  2.79160887e-02, -1.11540526e-01,
         3.48968625e-01, -3.55612896e-02, -4.91050584e-03,
        -5.67586944e-02,  8.80380869e-02, -4.17035855e-02,
         1.85913164e-02, -3.91423330e-02, -2.43817596e-03,
         8.89101401e-02,  1.87534431e-03,  3.37357074e-02,
        -6.68416731e-03, -7.49676973e-02, -3.72564979e-03,
         1.87743083e-02, -1.57694388e-02,  2.78985053e-02,
         2.58560106e-02, -5.71869221e-03,  1.68670192e-02,
        -1.32057024e-02, -2.69698296e-02, -1.92995481e-02,
        -7.41479173e-02,  4.91741076e-02, -1.21244408e-01,
        -2.41899863e-02,  4.11933884e-02,  2.86937528e-03,
        -1.51861571e-02,  5.77711277e-02,  1.00579247e-01,
        -3.06599420e-02, -4.14960347e-02, -5.28491847e-03,
         1.30650690e-02,  4.40504625e-02, -5.18948957e-03,
         2.06486229e-02, -5.28720170e-02, -3.61683890e-02,
         6.24842122e-02, -1.48781529e-03,  9.26209707e-03,
         4.92057465e-02, -8.04261565e-02,  8.98516551e-03,
         4.29460630e-02,  7.89718516e-03,  4.35357876e-02,
         2.58034132e-02,  5.88308536e-02,  4.74047326e-02,
         1.49990944e-02,  6.57414794e-02, -3.74123082e-02,
         6.79621771e-02, -4.06212313e-03,  9.12263170e-02,
        -3.30900960e-02, -3.94272767e-02,  3.20153646e-02,
         1.38845574e-02, -1.99709293e-02,  8.31901003e-03,
        -6.09774962e-02,  2.52220780e-02, -4.71705645e-02,
         9.22820624e-03,  1.38197010e-02,  6.14670515e-02,
        -6.11599609e-02, -5.32357730e-02,  5.35366940e-04,
         1.61767714e-02, -1.74247678e-02,  1.96990147e-02,
         1.21120103e-02, -2.49442738e-02,  2.62113307e-02,
        -9.60381627e-02,  4.06903662e-02, -6.90064207e-02,
        -5.19040413e-03, -3.72856446e-02,  4.18422977e-03,
        -1.45950783e-02,  4.66426089e-02,  1.72513537e-02,
         7.02733546e-02,  1.66991018e-02,  2.47594342e-02,
         1.83374975e-02, -1.44565515e-02, -8.25000694e-04,
         3.54981348e-02,  5.94661422e-02, -1.98835302e-02,
         2.67210063e-02, -3.42742652e-02, -8.47949311e-02,
         4.46633324e-02,  3.93131860e-02,  1.05438329e-01,
         1.48746103e-01, -8.42517689e-02,  3.81163545e-02,
         3.42347138e-02,  3.05974800e-02, -2.80073397e-02,
        -1.57944821e-02,  1.61785930e-02, -4.09205165e-03,
         2.18856204e-02, -8.32906440e-02, -7.71572739e-02,
        -3.63875553e-02,  6.62894025e-02, -6.13491647e-02,
         1.16128176e-02,  2.28852108e-02, -5.58810458e-02,
        -7.11629959e-03,  1.71528283e-32,  7.95187578e-02,
         4.03285623e-02, -9.23589095e-02, -7.32000023e-02,
        -1.68406088e-02,  2.98280641e-02,  2.72444300e-02,
         8.62754285e-02, -7.15340599e-02,  2.70222127e-02,
        -2.35113855e-02, -2.06820201e-02,  1.52852554e-02,
         2.17891242e-02, -1.26570957e-02, -4.81002331e-02,
         9.91885886e-02, -3.34858298e-02, -2.84026898e-02,
         4.43787463e-02, -3.44951898e-02,  1.78413726e-02,
        -1.55072529e-02,  3.24161090e-02, -1.51708964e-02,
         2.08107289e-02,  1.58074275e-02,  2.55570412e-02,
        -7.90592283e-02,  2.03256477e-02,  1.99132301e-02,
         1.15450896e-01,  1.73321050e-02,  8.00544098e-02,
         3.50806117e-02,  5.38774244e-02, -5.25988601e-02,
        -1.15043938e-01,  5.64412121e-03, -6.01647422e-02,
        -8.95632356e-02, -2.12106481e-02, -3.71850170e-02,
         7.30711371e-02, -1.12248342e-02, -2.83620916e-02,
        -7.11176991e-02,  2.01000851e-02,  2.51403693e-02,
        -8.02900866e-02, -8.17444548e-02, -3.37668099e-02,
        -1.36561692e-02, -6.50677904e-02, -7.09494278e-02,
        -6.51876722e-03, -8.32077563e-02,  5.15002795e-02,
        -1.00222707e-03, -2.11973228e-02, -5.27224923e-03,
         4.24491130e-02,  1.23829752e-01,  6.15164563e-02,
         4.51583192e-02, -8.26849253e-04,  4.92508523e-02,
         1.94461737e-02,  5.08161560e-02, -2.82611698e-02,
        -4.82448414e-02, -3.01309209e-02,  2.53796242e-02,
         2.65739057e-02, -3.88689786e-02, -2.08398444e-03,
        -1.52737424e-02, -7.54594132e-02,  4.11442667e-03,
         4.41256724e-03,  3.54635157e-02,  4.88590449e-02,
        -3.12298015e-02,  2.79445872e-02,  7.31522124e-03,
         1.59383640e-02,  5.15587814e-02,  5.07781133e-02,
        -3.35258618e-02, -2.44219750e-02,  1.26994634e-02,
         2.98856255e-02, -2.98038460e-02,  7.59720162e-04,
         9.33398586e-03, -3.77792365e-32, -2.39051022e-02,
        -2.28878818e-02,  3.86543274e-02, -4.30122502e-02,
         8.96073654e-02,  9.54524651e-02, -4.57702717e-03,
         4.72065806e-02, -6.21844493e-02, -1.19305318e-02,
         1.98797807e-02,  4.54092063e-02, -2.91915964e-02,
        -9.23778266e-02, -2.91774590e-02,  4.51411381e-02,
        -8.98976773e-02,  4.10241410e-02,  3.40340808e-02,
        -1.57277398e-02,  1.77023280e-02,  3.60726006e-02,
        -4.42338437e-02,  4.37921099e-02,  6.02683052e-02,
         2.81218011e-02,  1.11317979e-02, -2.04167087e-02,
        -4.76017557e-02,  2.56126858e-02,  1.08534917e-02,
         8.50801915e-02, -1.23381652e-02,  3.73017415e-03,
        -1.08977770e-02, -4.22941968e-02,  5.40373735e-02,
         4.60671494e-03,  4.81107272e-02, -1.18950218e-01,
        -4.22098339e-02,  4.28498315e-04, -4.60487511e-03,
        -6.80256411e-02,  2.47772522e-02,  5.72778434e-02,
         3.39871272e-02, -3.80931981e-02,  5.46790799e-03,
         1.50440093e-02, -1.71862950e-03, -4.49503772e-03,
         8.36174190e-02,  3.61522511e-02, -2.27608569e-02,
         1.09307664e-02, -4.64580245e-02, -2.51197778e-02,
         3.10343932e-02,  1.40037062e-02,  2.80777067e-02,
        -7.75458151e-03, -3.13466080e-02,  5.54159097e-02],
       [-4.36194129e-02,  1.24172762e-03, -6.61618076e-03,
        -3.61949168e-02,  3.40310186e-02, -1.04177609e-01,
         1.18855357e-01,  8.33758935e-02, -5.25901606e-03,
         7.02770874e-02,  7.44197592e-02, -1.92101318e-02,
         4.22310419e-02,  4.77747768e-02, -1.00599825e-01,
         6.59855502e-03, -7.06020044e-03, -5.78204431e-02,
        -7.44119510e-02, -5.77753447e-02,  8.12651031e-03,
        -1.05400132e-02, -9.23642442e-02,  2.72354651e-02,
        -3.27182934e-02,  1.47155644e-02,  1.84647162e-02,
         5.37966564e-02,  4.39366139e-03, -6.69799894e-02,
        -1.14284344e-02,  1.01684709e-03,  5.48181273e-02,
        -2.64709909e-03, -4.73516434e-02,  4.10526246e-03,
         7.96800107e-03, -9.52281430e-03,  3.66992578e-02,
         7.25318938e-02, -1.83794107e-02, -2.02359688e-02,
        -1.38983671e-02, -3.02274544e-02,  7.08662346e-03,
        -4.88152243e-02, -3.61666121e-02, -6.50164112e-02,
        -2.85622329e-02, -9.76050794e-02,  1.71331838e-02,
        -9.90536436e-02, -1.65603142e-02, -8.66569672e-03,
         7.57850781e-02, -7.43833333e-02, -4.66334112e-02,
         5.27071534e-03,  2.56000999e-02,  2.67603360e-02,
        -9.87996720e-03, -1.49469904e-03,  1.99468639e-02,
         9.64489765e-03,  4.01734374e-02,  4.46488522e-02,
        -4.24120668e-03,  7.03190565e-02, -9.25385505e-02,
        -8.32328722e-02, -5.34532182e-02,  2.76684482e-02,
         6.16889074e-02,  9.97963082e-03,  1.30488172e-01,
        -7.88195431e-02,  5.48367016e-02, -5.62950596e-02,
        -1.81357656e-02, -1.50051271e-03,  1.65783092e-02,
        -5.79604022e-02,  4.23630401e-02,  5.91657348e-02,
        -4.30757180e-04, -6.59449492e-03,  2.93029081e-02,
         4.89060804e-02, -8.74299742e-03, -2.56793201e-02,
        -8.70074630e-02, -8.48818868e-02,  5.86376712e-02,
         2.11759862e-02, -3.37857231e-02, -3.58547904e-02,
         2.04802584e-02, -7.34518841e-02, -6.23515472e-02,
         3.66165340e-01, -3.83556336e-02,  3.74274962e-02,
         9.02117509e-03,  1.85298603e-02,  4.67687361e-02,
         4.06234153e-02, -7.93087669e-03,  6.76765889e-02,
        -3.08420267e-02, -4.76172306e-02, -6.74149990e-02,
         5.09078577e-02,  2.75326427e-02, -2.45814007e-02,
         8.87249038e-03, -7.46029019e-02,  4.39203270e-02,
         2.12147217e-02, -1.49640292e-02,  5.55184335e-02,
         1.81750990e-02, -1.19720632e-02,  4.10598293e-02,
         2.85306107e-02, -3.33044007e-02, -5.93986586e-02,
         2.64546946e-02,  5.21890186e-02,  6.39567301e-02,
        -1.74697433e-02,  6.53330758e-02,  8.44450071e-02,
         1.46098370e-02,  2.77910642e-02,  2.18893085e-02,
         3.81155387e-02,  5.00992686e-02,  6.02446310e-02,
        -5.22410832e-02, -3.40476520e-02,  4.75198776e-02,
         7.90159255e-02,  5.34493588e-02,  2.22588014e-02,
        -3.05935331e-02, -3.82138453e-02,  5.47443032e-02,
        -1.31002264e-02, -2.66934186e-03,  1.02440208e-01,
        -9.34778377e-02,  3.55974957e-02,  2.56248191e-03,
         1.96040161e-02,  1.76399481e-02, -6.37880564e-02,
         2.10898425e-02, -6.21709824e-02,  5.36505878e-02,
        -1.70017756e-03, -6.84831738e-02,  4.26300056e-02,
        -2.26416159e-02, -3.44322622e-02, -2.49186363e-02,
        -5.42054279e-03, -8.74495357e-02, -4.67348173e-02,
        -4.39893790e-02,  2.52047572e-02,  6.16509989e-02,
         3.28155831e-02,  3.02153099e-02, -3.01138349e-02,
         5.70706576e-02,  3.71646509e-02, -2.55125165e-02,
         3.86465080e-02,  2.56903786e-02, -2.10288428e-02,
        -3.99890654e-02,  9.25337244e-03,  2.41863281e-02,
         2.41978429e-02,  3.20683196e-02,  9.09368042e-03,
        -6.19303025e-02, -2.52461410e-04,  5.78331240e-02,
         1.95390899e-02,  4.56315242e-02,  5.65279126e-02,
         4.05420847e-02,  4.74190665e-03,  3.66835259e-02,
        -4.67121229e-02, -2.21480839e-02,  9.98883601e-03,
         6.83738524e-03,  9.44694225e-03, -6.52247295e-03,
         4.46486138e-02,  6.32576421e-02, -2.71030553e-02,
         1.25889286e-01, -6.38752729e-02, -4.20950651e-02,
         4.77776863e-02, -4.44841869e-02, -1.71494000e-02,
         2.39695292e-02,  3.07548009e-02, -2.57944614e-02,
        -3.82859223e-02, -5.06828958e-03, -2.40310319e-02,
        -2.80019008e-02, -1.89768523e-02, -9.35744420e-02,
         3.23699415e-02,  1.44522469e-02, -2.76773255e-02,
        -1.74342338e-02,  1.82664475e-32, -4.42017987e-02,
         3.35162841e-02, -2.57008187e-02, -3.35008949e-02,
         8.01714230e-03,  1.55776809e-03, -4.58200043e-03,
         9.85373184e-03,  6.60310984e-02,  6.78686276e-02,
         4.81087603e-02,  1.47069022e-02,  8.06113258e-02,
        -3.42555754e-02, -3.26205743e-03, -1.04056671e-02,
         6.25951141e-02,  3.04815006e-02, -4.93760332e-02,
         4.47589867e-02,  2.63141328e-03, -4.71894518e-02,
        -6.37669712e-02,  1.10650128e-02, -3.39160499e-04,
         3.07911448e-02,  2.02939333e-03, -2.82918811e-02,
         6.19909726e-03,  4.30973172e-02, -3.54050547e-02,
         3.62614132e-02, -8.36105049e-02,  9.83160809e-02,
        -1.41894063e-02, -2.16325577e-02, -5.90247102e-02,
        -7.55953863e-02, -9.38706624e-04, -8.17082822e-03,
        -3.67824105e-03,  2.24777479e-02, -1.08326465e-01,
         3.02967113e-02, -1.06457040e-01,  2.84283534e-02,
        -4.95225228e-02,  7.92975649e-02,  2.33978573e-02,
        -6.50940016e-02,  3.94302793e-02, -1.58559773e-02,
        -1.06494585e-02, -8.48078281e-02, -6.19805716e-02,
         4.77777347e-02, -2.71212701e-02, -4.78804223e-02,
        -3.58883776e-02,  6.07158095e-02,  2.55190879e-02,
         1.44867378e-03,  7.76931527e-04,  8.79666731e-02,
         4.31706160e-02,  5.69744520e-02,  1.89088266e-02,
         9.71703157e-02, -3.89649905e-02, -6.06040657e-02,
        -8.53621513e-02,  4.91853803e-02, -5.25183119e-02,
        -8.83182138e-03,  1.13622947e-02,  1.24287196e-01,
         4.96101081e-02,  2.04096232e-02,  3.96146849e-02,
         2.58915708e-03,  2.56173741e-02,  2.37003732e-02,
         1.09122217e-01,  5.56674339e-02, -2.25001965e-02,
        -2.06263680e-02,  4.14560512e-02,  3.01600248e-03,
         2.19394639e-02, -4.92630433e-03,  8.85566697e-03,
         8.06230004e-04, -3.42080668e-02, -1.81107745e-02,
        -4.62028123e-02, -3.07123966e-32, -1.18308902e-01,
         2.00220589e-02,  2.60014329e-02, -5.92399947e-03,
         6.24213554e-02,  1.60534922e-02,  9.21495110e-02,
         1.08808803e-03,  9.76104755e-03,  5.86248823e-02,
        -2.83072703e-02,  6.73462823e-02,  1.28318155e-02,
        -2.28054312e-04,  4.11208421e-02, -6.43146709e-02,
        -4.27600965e-02,  4.50460287e-03,  6.53191209e-02,
         1.47612626e-02,  7.81147629e-02,  4.49998751e-02,
        -7.58399721e-03, -2.98436265e-02,  7.98099115e-02,
        -4.78304736e-02, -5.87424412e-02, -1.09326988e-01,
        -2.58829203e-02, -6.28677756e-02,  1.17499873e-01,
        -6.22401237e-02, -7.96295702e-02, -5.29674888e-02,
        -1.55016165e-02,  3.20038572e-02, -1.53178070e-02,
        -4.22602929e-02,  3.42442580e-02,  1.23892340e-03,
         2.29211543e-02,  3.36142480e-02,  6.51369765e-02,
        -2.17055362e-02,  2.06596069e-02, -3.44996410e-03,
        -1.52502479e-02,  2.87910402e-02,  2.59440504e-02,
        -3.58282849e-02, -4.73467298e-02,  2.20816340e-02,
         2.24817712e-02, -4.73678187e-02,  3.65528949e-02,
        -1.30275693e-02, -5.71539346e-03, -2.98424754e-02,
        -9.99975726e-02,  7.44161159e-02, -3.12952884e-02,
        -6.30516186e-02, -4.87828217e-02,  1.25095109e-02]], dtype=float32)

Call http inference endpoint¶

This option will only be awailable after enabling serving and waiting for model to be deployed.

In [0]:

response = requests.post(
    '<your_databricks_workspace_uri>/model/transformer-model/Production/invocations',
    headers={
        'Authorization': f'Bearer {os.environ.get("DATABRICKS_TOKEN")}',
        'Content-Type': 'application/json',
    },
    json={"inputs": {"text": ["hello", "world"]}},
)

response = requests.post( '/model/transformer-model/Production/invocations', headers={ 'Authorization': f'Bearer {os.environ.get("DATABRICKS_TOKEN")}', 'Content-Type': 'application/json', }, json={"inputs": {"text": ["hello", "world"]}}, )

In [0]:

response.status_code

response.status_code

Out[0]:

200

In [0]:

response.json()

response.json()

Out[0]:

{'predictions': [[-0.07490702718496323,
   -0.01443308126181364,
   0.04864983260631561,
   -0.027138084173202515,
   -0.043088190257549286,
   -0.14776340126991272,
   0.06883306056261063,
   -0.013703802600502968,
   -0.05356861650943756,
   0.026975270360708237,
   -0.006283464841544628,
   -0.039883457124233246,
   0.007656700909137726,
   -0.037808965891599655,
   -0.011755705811083317,
   -0.034640949219465256,
   0.12935706973075867,
   -0.025277869775891304,
   -0.015209854580461979,
   0.000730327214114368,
   -0.08068884164094925,
   0.02693787030875683,
   -0.0987357422709465,
   -0.034107550978660583,
   -0.027029285207390785,
   -0.07320033013820648,
   0.050858885049819946,
   -0.017256317660212517,
   0.07282191514968872,
   -0.12328874319791794,
   -0.010971251875162125,
   0.015405580401420593,
   0.16192536056041718,
   -0.03713918477296829,
   -0.017285751178860664,
   0.026161983609199524,
   0.07309548556804657,
   -0.04667157679796219,
   -0.040614672005176544,
   0.05917086452245712,
   0.016567112877964973,
   0.007346309255808592,
   -0.04897543415427208,
   0.0689893513917923,
   0.04286481440067291,
   -0.07572471350431442,
   -0.015094270929694176,
   0.04190893471240997,
   0.05491500347852707,
   -0.056395452469587326,
   -0.01026054099202156,
   0.0018723044777289033,
   -0.007897173054516315,
   0.05324820429086685,
   0.08370450884103775,
   0.03319425508379936,
   -0.015426648780703545,
   0.06000211834907532,
   0.028092052787542343,
   0.02252606861293316,
   0.014536287635564804,
   0.024353498592972755,
   -0.006686619482934475,
   -0.07501193881034851,
   -0.009480259381234646,
   -0.011537599377334118,
   -0.019241614267230034,
   -0.013632320798933506,
   0.08455581218004227,
   -0.04682128503918648,
   -0.0024662339128553867,
   -0.021389620378613472,
   -0.0011715105501934886,
   -0.07554260641336441,
   -0.04247080907225609,
   0.014182341285049915,
   0.01826930046081543,
   0.044562745839357376,
   -0.006144660525023937,
   -0.011601999402046204,
   0.0317072831094265,
   -0.09668204188346863,
   0.041295867413282394,
   0.03993024304509163,
   0.045429427176713943,
   0.038931675255298615,
   0.049543365836143494,
   0.04893714562058449,
   0.030688703060150146,
   0.020303519442677498,
   -0.06820891052484512,
   0.03905028477311134,
   0.018929757177829742,
   -0.016038360074162483,
   -0.010620114393532276,
   0.03297124803066254,
   0.03548092767596245,
   0.027916131541132927,
   -0.11154043674468994,
   0.348968505859375,
   -0.035561323165893555,
   -0.004910491872578859,
   -0.056758638471364975,
   0.08803820610046387,
   -0.04170358553528786,
   0.01859135366976261,
   -0.03914238139986992,
   -0.0024381878320127726,
   0.08891010284423828,
   0.001875291345641017,
   0.03373574838042259,
   -0.006684185471385717,
   -0.07496771961450577,
   -0.0037256991490721703,
   0.018774284049868584,
   -0.015769420191645622,
   0.027898505330085754,
   0.025856001302599907,
   -0.00571862468495965,
   0.016866987571120262,
   -0.0132057536393404,
   -0.02696983329951763,
   -0.01929955743253231,
   -0.07414791733026505,
   0.04917411506175995,
   -0.12124430388212204,
   -0.024190016090869904,
   0.041193414479494095,
   0.0028693845961242914,
   -0.015186102129518986,
   0.05777110531926155,
   0.10057920962572098,
   -0.03065994754433632,
   -0.04149605706334114,
   -0.005284885875880718,
   0.013065098784863949,
   0.044050540775060654,
   -0.005189500283449888,
   0.020648593083024025,
   -0.05287201702594757,
   -0.036168381571769714,
   0.06248431280255318,
   -0.0014878440415486693,
   0.009262116625905037,
   0.049205753952264786,
   -0.08042621612548828,
   0.00898516271263361,
   0.0429459884762764,
   0.007897159084677696,
   0.043535832315683365,
   0.02580338343977928,
   0.0588308647274971,
   0.04740474373102188,
   0.014999138191342354,
   0.06574150174856186,
   -0.03741232305765152,
   0.0679621770977974,
   -0.00406209472566843,
   0.09122636169195175,
   -0.03309011459350586,
   -0.039427321404218674,
   0.03201528266072273,
   0.013884488493204117,
   -0.019970934838056564,
   0.008319037035107613,
   -0.0609775073826313,
   0.02522209659218788,
   -0.047170523554086685,
   0.00922818947583437,
   0.013819699175655842,
   0.061467044055461884,
   -0.061159972101449966,
   -0.05323580279946327,
   0.0005353130982257426,
   0.016176819801330566,
   -0.017424793913960457,
   0.01969904825091362,
   0.012111999094486237,
   -0.024944283068180084,
   0.02621140331029892,
   -0.09603817015886307,
   0.04069036245346069,
   -0.06900647282600403,
   -0.0051904721185564995,
   -0.0372856929898262,
   0.004184163641184568,
   -0.01459512673318386,
   0.04664258286356926,
   0.017251335084438324,
   0.07027335464954376,
   0.01669912412762642,
   0.02475942298769951,
   0.018337512388825417,
   -0.014456609264016151,
   -0.0008249963284470141,
   0.03549818694591522,
   0.05946619063615799,
   -0.019883597269654274,
   0.0267210453748703,
   -0.034274302423000336,
   -0.08479492366313934,
   0.04466329514980316,
   0.03931320086121559,
   0.10543838888406754,
   0.14874614775180817,
   -0.08425174653530121,
   0.038116369396448135,
   0.034234702587127686,
   0.030597509816288948,
   -0.028007302433252335,
   -0.01579442247748375,
   0.016178537160158157,
   -0.004092061426490545,
   0.021885637193918228,
   -0.08329066634178162,
   -0.07715727388858795,
   -0.03638743609189987,
   0.06628938764333725,
   -0.06134912744164467,
   0.011612867936491966,
   0.022885186597704887,
   -0.05588102713227272,
   -0.007116286549717188,
   1.715283266898202e-32,
   0.07951880991458893,
   0.04032856225967407,
   -0.09235890954732895,
   -0.07320000231266022,
   -0.016840578988194466,
   0.029828021302819252,
   0.027244456112384796,
   0.08627542853355408,
   -0.07153405249118805,
   0.027022212743759155,
   -0.023511454463005066,
   -0.02068202942609787,
   0.01528528518974781,
   0.021789105609059334,
   -0.01265703048557043,
   -0.048100218176841736,
   0.09918858110904694,
   -0.033485863357782364,
   -0.028402796015143394,
   0.04437867924571037,
   -0.03449517861008644,
   0.017841415479779243,
   -0.01550727803260088,
   0.03241603821516037,
   -0.015170935541391373,
   0.020810693502426147,
   0.01580740511417389,
   0.02555708773434162,
   -0.07905929535627365,
   0.020325616002082825,
   0.019913246855139732,
   0.11545094102621078,
   0.017332149669528008,
   0.0800543874502182,
   0.03508063033223152,
   0.05387740582227707,
   -0.052598875015974045,
   -0.11504394561052322,
   0.005644132848829031,
   -0.06016472354531288,
   -0.08956325799226761,
   -0.021210599690675735,
   -0.03718499466776848,
   0.07307110726833344,
   -0.011224858462810516,
   -0.02836211957037449,
   -0.07111760228872299,
   0.02010011300444603,
   0.02514035254716873,
   -0.08029015362262726,
   -0.08174445480108261,
   -0.0337667390704155,
   -0.013656162656843662,
   -0.06506780534982681,
   -0.07094952464103699,
   -0.006518752314150333,
   -0.08320780098438263,
   0.05150030553340912,
   -0.0010022292844951153,
   -0.021197330206632614,
   -0.0052722240798175335,
   0.04244908690452576,
   0.12382970750331879,
   0.06151643022894859,
   0.045158322900533676,
   -0.0008269379613921046,
   0.04925081506371498,
   0.019446199759840965,
   0.05081614479422569,
   -0.028261205181479454,
   -0.04824479669332504,
   -0.030130889266729355,
   0.02537967637181282,
   0.026573896408081055,
   -0.038869019597768784,
   -0.002083965577185154,
   -0.015273724682629108,
   -0.07545942068099976,
   0.0041144113056361675,
   0.004412523936480284,
   0.03546347841620445,
   0.04885905981063843,
   -0.031229782849550247,
   0.02794460766017437,
   0.007315249647945166,
   0.01593834161758423,
   0.05155876278877258,
   0.05077815055847168,
   -0.03352585807442665,
   -0.024421995505690575,
   0.01269952766597271,
   0.029885638505220413,
   -0.029803799465298653,
   0.0007597118383273482,
   0.0093339579179883,
   -3.7779236450964515e-32,
   -0.023905085399746895,
   -0.02288786694407463,
   0.03865434601902962,
   -0.043012164533138275,
   0.0896073654294014,
   0.09545248746871948,
   -0.00457707280293107,
   0.0472065806388855,
   -0.062184497714042664,
   -0.011930582113564014,
   0.019879771396517754,
   0.04540923237800598,
   -0.029191553592681885,
   -0.09237777441740036,
   -0.029177455231547356,
   0.04514108970761299,
   -0.08989764750003815,
   0.04102415591478348,
   0.03403407707810402,
   -0.015727724879980087,
   0.017702341079711914,
   0.036072611808776855,
   -0.04423394799232483,
   0.043792083859443665,
   0.060268305242061615,
   0.028121890500187874,
   0.011131782084703445,
   -0.020416704937815666,
   -0.04760175570845604,
   0.02561267651617527,
   0.010853544808924198,
   0.08508022129535675,
   -0.012338175438344479,
   0.00373013480566442,
   -0.01089775562286377,
   -0.04229416325688362,
   0.05403739959001541,
   0.004606758244335651,
   0.0481107272207737,
   -0.11895021796226501,
   -0.042209893465042114,
   0.00042848577140830457,
   -0.004604817368090153,
   -0.06802567094564438,
   0.024777276441454887,
   0.057277861982584,
   0.033987049013376236,
   -0.038093239068984985,
   0.0054679070599377155,
   0.015043937601149082,
   -0.0017186368349939585,
   -0.004495053086429834,
   0.08361741155385971,
   0.03615223243832588,
   -0.022760875523090363,
   0.010930780321359634,
   -0.04645800217986107,
   -0.025119764730334282,
   0.031034383922815323,
   0.014003745280206203,
   0.028077738359570503,
   -0.007754610851407051,
   -0.031346630305051804,
   0.05541591718792915],
  [-0.043619439005851746,
   0.0012417675461620092,
   -0.0066161612048745155,
   -0.036194950342178345,
   0.03403099253773689,
   -0.10417766869068146,
   0.11885537207126617,
   0.08337590843439102,
   -0.005258981138467789,
   0.07027710974216461,
   0.0744197890162468,
   -0.019210102036595345,
   0.04223107546567917,
   0.04777480289340019,
   -0.10059982538223267,
   0.00659859087318182,
   -0.007060152944177389,
   -0.05782044306397438,
   -0.07441196590662003,
   -0.057775381952524185,
   0.008126454427838326,
   -0.010540014132857323,
   -0.09236421436071396,
   0.027235446497797966,
   -0.03271837905049324,
   0.014715492725372314,
   0.018464764580130577,
   0.0537966787815094,
   0.004393658135086298,
   -0.0669800415635109,
   -0.011428463272750378,
   0.0010168250882998109,
   0.054818153381347656,
   -0.002647096524015069,
   -0.04735160619020462,
   0.004105263855308294,
   0.007968010380864143,
   -0.009522794745862484,
   0.03669930249452591,
   0.07253183424472809,
   -0.018379392102360725,
   -0.020235970616340637,
   -0.013898422010242939,
   -0.030227402225136757,
   0.007086604367941618,
   -0.048815254122018814,
   -0.03616663068532944,
   -0.06501639634370804,
   -0.02856222353875637,
   -0.09760506451129913,
   0.01713320054113865,
   -0.09905365854501724,
   -0.01656031422317028,
   -0.008665705099701881,
   0.0757850781083107,
   -0.07438331097364426,
   -0.04663340747356415,
   0.005270721390843391,
   0.02560015581548214,
   0.026760326698422432,
   -0.00987992249429226,
   -0.0014946406008675694,
   0.0199468694627285,
   0.009644858539104462,
   0.040173523128032684,
   0.0446489118039608,
   -0.004241177346557379,
   0.07031906396150589,
   -0.09253856539726257,
   -0.08323289453983307,
   -0.053453199565410614,
   0.027668451890349388,
   0.061688926070928574,
   0.009979595430195332,
   0.13048815727233887,
   -0.0788196250796318,
   0.05483662709593773,
   -0.056295059621334076,
   -0.018135806545615196,
   -0.0015005044406279922,
   0.01657830737531185,
   -0.05796034634113312,
   0.042363088577985764,
   0.059165749698877335,
   -0.000430751038948074,
   -0.006594479084014893,
   0.029302923008799553,
   0.04890604317188263,
   -0.008742937818169594,
   -0.02567925676703453,
   -0.08700745552778244,
   -0.08488183468580246,
   0.05863768979907036,
   0.021175967529416084,
   -0.03378571942448616,
   -0.035854771733284,
   0.02048029936850071,
   -0.07345185428857803,
   -0.062351491302251816,
   0.3661653399467468,
   -0.038355614989995956,
   0.037427518516778946,
   0.009021146222949028,
   0.01852988637983799,
   0.04676869884133339,
   0.04062345623970032,
   -0.007930937223136425,
   0.0676765963435173,
   -0.030842063948512077,
   -0.04761726036667824,
   -0.0674150288105011,
   0.05090788006782532,
   0.02753262408077717,
   -0.024581395089626312,
   0.008872441947460175,
   -0.0746028870344162,
   0.04392031952738762,
   0.02121473103761673,
   -0.01496405340731144,
   0.055518459528684616,
   0.018175147473812103,
   -0.011972054839134216,
   0.04105980694293976,
   0.028530606999993324,
   -0.033304363489151,
   -0.059398774057626724,
   0.026454642415046692,
   0.052188947796821594,
   0.06395678222179413,
   -0.0174697358161211,
   0.06533306837081909,
   0.08444499224424362,
   0.014609841629862785,
   0.02779104933142662,
   0.021889295428991318,
   0.03811546415090561,
   0.050099294632673264,
   0.06024466082453728,
   -0.05224113166332245,
   -0.03404765948653221,
   0.04751987382769585,
   0.07901587337255478,
   0.053449392318725586,
   0.022258764132857323,
   -0.030593503266572952,
   -0.0382138267159462,
   0.05474429205060005,
   -0.013100207783281803,
   -0.0026693735271692276,
   0.10244017839431763,
   -0.09347783029079437,
   0.03559752181172371,
   0.002562473760917783,
   0.0196040328592062,
   0.017639940604567528,
   -0.06378813832998276,
   0.021089863032102585,
   -0.062171004712581635,
   0.0536506250500679,
   -0.0017002058448269963,
   -0.06848315894603729,
   0.04262998700141907,
   -0.02264164760708809,
   -0.03443225473165512,
   -0.0249185673892498,
   -0.005420503672212362,
   -0.08744952827692032,
   -0.04673484340310097,
   -0.04398934170603752,
   0.025204747915267944,
   0.06165103614330292,
   0.0328156054019928,
   0.030215345323085785,
   -0.030113888904452324,
   0.05707065761089325,
   0.0371645987033844,
   -0.02551243081688881,
   0.038646526634693146,
   0.025690356269478798,
   -0.02102884277701378,
   -0.039989035576581955,
   0.009253344498574734,
   0.024186307564377785,
   0.024197813123464584,
   0.03206832334399223,
   0.009093702770769596,
   -0.06193032115697861,
   -0.0002524311712477356,
   0.05783309042453766,
   0.019539114087820053,
   0.04563150554895401,
   0.05652792006731033,
   0.040542103350162506,
   0.00474184425547719,
   0.036683544516563416,
   -0.04671211913228035,
   -0.022148095071315765,
   0.009988845326006413,
   0.00683737825602293,
   0.009446938522160053,
   -0.006522531621158123,
   0.044648606330156326,
   0.06325763463973999,
   -0.027103088796138763,
   0.12588930130004883,
   -0.06387532502412796,
   -0.04209502786397934,
   0.0477776937186718,
   -0.0444842204451561,
   -0.01714935153722763,
   0.023969532921910286,
   0.030754782259464264,
   -0.025794504210352898,
   -0.03828589245676994,
   -0.00506824953481555,
   -0.024031037464737892,
   -0.02800186537206173,
   -0.018976889550685883,
   -0.09357450902462006,
   0.03236991912126541,
   0.014452234841883183,
   -0.027677329257130623,
   -0.017434250563383102,
   1.8266447527809084e-32,
   -0.044201772660017014,
   0.03351627290248871,
   -0.02570079080760479,
   -0.03350091353058815,
   0.008017141371965408,
   0.0015577750746160746,
   -0.0045820181258022785,
   0.009853705763816833,
   0.06603112071752548,
   0.06786862760782242,
   0.048108723014593124,
   0.014706892892718315,
   0.08061131834983826,
   -0.034255560487508774,
   -0.0032620818819850683,
   -0.010405679233372211,
   0.06259512901306152,
   0.030481504276394844,
   -0.04937608912587166,
   0.04475899040699005,
   0.002631417941302061,
   -0.04718944430351257,
   -0.06376693397760391,
   0.011064964346587658,
   -0.0003391142236068845,
   0.030791139230132103,
   0.0020293579436838627,
   -0.02829189971089363,
   0.006199076771736145,
   0.043097302317619324,
   -0.035405054688453674,
   0.0362614244222641,
   -0.08361049741506577,
   0.09831609576940536,
   -0.014189419336616993,
   -0.021632561460137367,
   -0.05902468040585518,
   -0.0755954161286354,
   -0.000938678567763418,
   -0.00817082915455103,
   -0.0036782254464924335,
   0.022477734833955765,
   -0.10832643508911133,
   0.030296701937913895,
   -0.10645707696676254,
   0.028428370133042336,
   -0.049522515386343,
   0.07929755747318268,
   0.023397838696837425,
   -0.0650940015912056,
   0.039430271834135056,
   -0.015855984762310982,
   -0.010649463161826134,
   -0.08480788767337799,
   -0.06198057904839516,
   0.04777773469686508,
   -0.027121294289827347,
   -0.04788042977452278,
   -0.03588831424713135,
   0.060715846717357635,
   0.02551906369626522,
   0.0014486659783869982,
   0.0007769512594677508,
   0.08796665817499161,
   0.04317057132720947,
   0.05697440356016159,
   0.01890881545841694,
   0.09717033803462982,
   -0.038964975625276566,
   -0.06060405448079109,
   -0.08536211401224136,
   0.04918540641665459,
   -0.05251830816268921,
   -0.008831857703626156,
   0.011362235061824322,
   0.12428716570138931,
   0.049610111862421036,
   0.0204095970839262,
   0.03961470350623131,
   0.0025891505647450686,
   0.025617379695177078,
   0.02370036579668522,
   0.1091221421957016,
   0.055667489767074585,
   -0.022500155493617058,
   -0.020626356825232506,
   0.04145609587430954,
   0.0030160103924572468,
   0.021939417347311974,
   -0.0049263350665569305,
   0.008855664171278477,
   0.0008061827393248677,
   -0.034208111464977264,
   -0.018110832199454308,
   -0.04620274901390076,
   -3.071238775774758e-32,
   -0.1183088943362236,
   0.020021982491016388,
   0.026001442223787308,
   -0.005924020893871784,
   0.06242140755057335,
   0.016053486615419388,
   0.092149518430233,
   0.0010881092166528106,
   0.009761064313352108,
   0.058624882251024246,
   -0.02830728329718113,
   0.06734628975391388,
   0.012831822969019413,
   -0.0002280419139424339,
   0.04112083092331886,
   -0.06431466341018677,
   -0.04276016727089882,
   0.004504664335399866,
   0.06531913578510284,
   0.014761260710656643,
   0.07811471819877625,
   0.04499988630414009,
   -0.007583981845527887,
   -0.02984366938471794,
   0.07980986684560776,
   -0.04783043637871742,
   -0.058742448687553406,
   -0.10932699590921402,
   -0.02588292397558689,
   -0.06286779046058655,
   0.1174999251961708,
   -0.06224008649587631,
   -0.07962951064109802,
   -0.052967485040426254,
   -0.015501629561185837,
   0.032003872096538544,
   -0.015317845158278942,
   -0.04226025938987732,
   0.03424423560500145,
   0.0012389487819746137,
   0.022921131923794746,
   0.03361426666378975,
   0.06513690948486328,
   -0.021705513820052147,
   0.020659610629081726,
   -0.003449993906542659,
   -0.01525021716952324,
   0.028791045770049095,
   0.02594403177499771,
   -0.035828299820423126,
   -0.04734674096107483,
   0.022081643342971802,
   0.022481778636574745,
   -0.047367777675390244,
   0.03655289113521576,
   -0.013027514331042767,
   -0.005715371109545231,
   -0.029842434450984,
   -0.09999755769968033,
   0.07441609352827072,
   -0.0312952995300293,
   -0.06305162608623505,
   -0.048782773315906525,
   0.012509501539170742]]}

In [0]: