Mlflow transformer
Deploy any Transformer–based model to a serving endpoint in DataBricks¶
This notebook shows how to deploy transformer-based model to a DataBricks serving endpoint using MLFlow
In [0]:
Copied!
%pip install sentence-transformers
%pip install sentence-transformers
Requirement already satisfied: sentence-transformers in /databricks/python3/lib/python3.11/site-packages (2.7.0)
Requirement already satisfied: transformers<5.0.0,>=4.34.0 in /databricks/python3/lib/python3.11/site-packages (from sentence-transformers) (4.41.2)
Requirement already satisfied: tqdm in /databricks/python3/lib/python3.11/site-packages (from sentence-transformers) (4.65.0)
Requirement already satisfied: torch>=1.11.0 in /databricks/python3/lib/python3.11/site-packages (from sentence-transformers) (2.3.1+cpu)
Requirement already satisfied: numpy in /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages (from sentence-transformers) (1.24.4)
Requirement already satisfied: scikit-learn in /databricks/python3/lib/python3.11/site-packages (from sentence-transformers) (1.3.0)
Requirement already satisfied: scipy in /databricks/python3/lib/python3.11/site-packages (from sentence-transformers) (1.11.1)
Requirement already satisfied: huggingface-hub>=0.15.1 in /databricks/python3/lib/python3.11/site-packages (from sentence-transformers) (0.27.1)
Requirement already satisfied: Pillow in /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages (from sentence-transformers) (10.4.0)
Requirement already satisfied: filelock in /databricks/python3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (3.13.4)
Requirement already satisfied: fsspec>=2023.5.0 in /databricks/python3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (2023.5.0)
Requirement already satisfied: packaging>=20.9 in /databricks/python3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /databricks/python3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (6.0)
Requirement already satisfied: requests in /databricks/python3/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (2.31.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (4.12.2)
Requirement already satisfied: sympy in /databricks/python3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers) (1.11.1)
Requirement already satisfied: networkx in /databricks/python3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers) (3.1)
Requirement already satisfied: jinja2 in /databricks/python3/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers) (3.1.2)
Requirement already satisfied: regex!=2019.12.17 in /databricks/python3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.34.0->sentence-transformers) (2022.7.9)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /databricks/python3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.34.0->sentence-transformers) (0.19.0)
Requirement already satisfied: safetensors>=0.4.1 in /databricks/python3/lib/python3.11/site-packages (from transformers<5.0.0,>=4.34.0->sentence-transformers) (0.4.2)
Requirement already satisfied: joblib>=1.1.1 in /databricks/python3/lib/python3.11/site-packages (from scikit-learn->sentence-transformers) (1.2.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /databricks/python3/lib/python3.11/site-packages (from scikit-learn->sentence-transformers) (2.2.0)
Requirement already satisfied: MarkupSafe>=2.0 in /databricks/python3/lib/python3.11/site-packages (from jinja2->torch>=1.11.0->sentence-transformers) (2.1.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /databricks/python3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /databricks/python3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /databricks/python3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /databricks/python3/lib/python3.11/site-packages (from requests->huggingface-hub>=0.15.1->sentence-transformers) (2023.7.22)
Requirement already satisfied: mpmath>=0.19 in /databricks/python3/lib/python3.11/site-packages (from sympy->torch>=1.11.0->sentence-transformers) (1.3.0)
Note: you may need to restart the kernel using %restart_python or dbutils.library.restartPython() to use updated packages.
Wrap Transformer model into MLFlow api and log it to MLFlow.¶
Before deploying the model we need to make it compatible with MLFlow.
This is done via wrapping a model into mlflow.pyfunc.PythonModel subclass, create conda_env and log model to MLFlow.
In [0]:
Copied!
import pandas as pd
import mlflow
import sentence_transformers
from mlflow.models import infer_signature
from sentence_transformers import SentenceTransformer
class TransformerWrapper(mlflow.pyfunc.PythonModel):
def __init__(self):
self.model_name = "sentence-transformers/all-MiniLM-L12-v2"
def load_context(self, context : dict):
self.model = SentenceTransformer(self.model_name)
def predict(self, context : dict, model_input : pd.DataFrame):
return self.model.encode(model_input["text"])
import pandas as pd
import mlflow
import sentence_transformers
from mlflow.models import infer_signature
from sentence_transformers import SentenceTransformer
class TransformerWrapper(mlflow.pyfunc.PythonModel):
def __init__(self):
self.model_name = "sentence-transformers/all-MiniLM-L12-v2"
def load_context(self, context : dict):
self.model = SentenceTransformer(self.model_name)
def predict(self, context : dict, model_input : pd.DataFrame):
return self.model.encode(model_input["text"])
/databricks/python/lib/python3.11/site-packages/huggingface_hub/file_download.py:795: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn(
Out[0]:
array([[-7.49069825e-02, -1.44330934e-02, 4.86498103e-02,
-2.71380432e-02, -4.30882052e-02, -1.47763386e-01,
6.88330382e-02, -1.37037653e-02, -5.35686426e-02,
2.69752610e-02, -6.28342759e-03, -3.98834683e-02,
7.65673863e-03, -3.78089547e-02, -1.17557859e-02,
-3.46409269e-02, 1.29357129e-01, -2.52779089e-02,
-1.52098928e-02, 7.30281230e-04, -8.06888267e-02,
2.69378833e-02, -9.87357423e-02, -3.41075659e-02,
-2.70293411e-02, -7.32003674e-02, 5.08588888e-02,
-1.72562227e-02, 7.28218481e-02, -1.23288780e-01,
-1.09712686e-02, 1.54055692e-02, 1.61925435e-01,
-3.71391810e-02, -1.72857344e-02, 2.61619836e-02,
7.30955228e-02, -4.66715395e-02, -4.06146608e-02,
5.91708682e-02, 1.65671837e-02, 7.34632788e-03,
-4.89753485e-02, 6.89893663e-02, 4.28648256e-02,
-7.57247284e-02, -1.50942868e-02, 4.19089496e-02,
5.49149513e-02, -5.63954599e-02, -1.02605559e-02,
1.87228876e-03, -7.89719820e-03, 5.32482155e-02,
8.37045312e-02, 3.31942365e-02, -1.54265985e-02,
6.00020960e-02, 2.80920826e-02, 2.25260668e-02,
1.45363053e-02, 2.43535191e-02, -6.68663485e-03,
-7.50119388e-02, -9.48021840e-03, -1.15375975e-02,
-1.92416590e-02, -1.36322994e-02, 8.45558047e-02,
-4.68212925e-02, -2.46622181e-03, -2.13895701e-02,
-1.17153139e-03, -7.55425319e-02, -4.24707793e-02,
1.41823869e-02, 1.82693042e-02, 4.45626862e-02,
-6.14464981e-03, -1.16020218e-02, 3.17072906e-02,
-9.66820121e-02, 4.12957855e-02, 3.99302468e-02,
4.54295091e-02, 3.89316715e-02, 4.95432727e-02,
4.89371903e-02, 3.06886621e-02, 2.03034990e-02,
-6.82089329e-02, 3.90502624e-02, 1.89297646e-02,
-1.60383042e-02, -1.06201377e-02, 3.29712890e-02,
3.54809538e-02, 2.79161222e-02, -1.11540437e-01,
3.48968446e-01, -3.55612971e-02, -4.91052400e-03,
-5.67586869e-02, 8.80381539e-02, -4.17036004e-02,
1.85913499e-02, -3.91423330e-02, -2.43820064e-03,
8.89101401e-02, 1.87533081e-03, 3.37356888e-02,
-6.68419339e-03, -7.49677122e-02, -3.72568471e-03,
1.87742598e-02, -1.57694183e-02, 2.78985128e-02,
2.58559883e-02, -5.71870152e-03, 1.68670155e-02,
-1.32057145e-02, -2.69698557e-02, -1.92995910e-02,
-7.41478801e-02, 4.91740592e-02, -1.21244356e-01,
-2.41899900e-02, 4.11934145e-02, 2.86935410e-03,
-1.51861086e-02, 5.77710532e-02, 1.00579195e-01,
-3.06599475e-02, -4.14960459e-02, -5.28487284e-03,
1.30650867e-02, 4.40505072e-02, -5.18952776e-03,
2.06485391e-02, -5.28720357e-02, -3.61683518e-02,
6.24842308e-02, -1.48780015e-03, 9.26208403e-03,
4.92057353e-02, -8.04261565e-02, 8.98520183e-03,
4.29459885e-02, 7.89717585e-03, 4.35358137e-02,
2.58033723e-02, 5.88308573e-02, 4.74047549e-02,
1.49991428e-02, 6.57414794e-02, -3.74122970e-02,
6.79621398e-02, -4.06213384e-03, 9.12263915e-02,
-3.30900885e-02, -3.94272804e-02, 3.20153497e-02,
1.38845639e-02, -1.99709572e-02, 8.31900258e-03,
-6.09775186e-02, 2.52220556e-02, -4.71705347e-02,
9.22823325e-03, 1.38197336e-02, 6.14670403e-02,
-6.11599460e-02, -5.32357544e-02, 5.35383122e-04,
1.61767676e-02, -1.74248312e-02, 1.96989998e-02,
1.21120447e-02, -2.49442980e-02, 2.62113735e-02,
-9.60381627e-02, 4.06903177e-02, -6.90065026e-02,
-5.19047910e-03, -3.72856706e-02, 4.18422837e-03,
-1.45951351e-02, 4.66426462e-02, 1.72513369e-02,
7.02733770e-02, 1.66991390e-02, 2.47593988e-02,
1.83375552e-02, -1.44566316e-02, -8.24987306e-04,
3.54981124e-02, 5.94661348e-02, -1.98835805e-02,
2.67210379e-02, -3.42742912e-02, -8.47949162e-02,
4.46632877e-02, 3.93132418e-02, 1.05438367e-01,
1.48746148e-01, -8.42517167e-02, 3.81164141e-02,
3.42346951e-02, 3.05974968e-02, -2.80073173e-02,
-1.57944132e-02, 1.61785651e-02, -4.09205630e-03,
2.18856502e-02, -8.32906589e-02, -7.71572515e-02,
-3.63875180e-02, 6.62893951e-02, -6.13492057e-02,
1.16127953e-02, 2.28851754e-02, -5.58810197e-02,
-7.11630657e-03, 1.71528327e-32, 7.95188025e-02,
4.03285585e-02, -9.23589468e-02, -7.32000023e-02,
-1.68405697e-02, 2.98280362e-02, 2.72444859e-02,
8.62753987e-02, -7.15340823e-02, 2.70222295e-02,
-2.35114135e-02, -2.06820164e-02, 1.52853029e-02,
2.17891596e-02, -1.26571264e-02, -4.81002107e-02,
9.91885662e-02, -3.34858112e-02, -2.84027159e-02,
4.43787202e-02, -3.44952121e-02, 1.78413671e-02,
-1.55071933e-02, 3.24161090e-02, -1.51709123e-02,
2.08107550e-02, 1.58074386e-02, 2.55570989e-02,
-7.90592507e-02, 2.03256626e-02, 1.99132673e-02,
1.15450941e-01, 1.73321329e-02, 8.00543725e-02,
3.50806154e-02, 5.38774990e-02, -5.25988117e-02,
-1.15043998e-01, 5.64412540e-03, -6.01646937e-02,
-8.95631835e-02, -2.12106332e-02, -3.71850021e-02,
7.30711594e-02, -1.12248510e-02, -2.83621158e-02,
-7.11176693e-02, 2.01000869e-02, 2.51403581e-02,
-8.02901238e-02, -8.17444250e-02, -3.37667949e-02,
-1.36561627e-02, -6.50678053e-02, -7.09495097e-02,
-6.51874533e-03, -8.32077414e-02, 5.15003316e-02,
-1.00220670e-03, -2.11973302e-02, -5.27226413e-03,
4.24490832e-02, 1.23829789e-01, 6.15164042e-02,
4.51583713e-02, -8.26959033e-04, 4.92508635e-02,
1.94461998e-02, 5.08161411e-02, -2.82611754e-02,
-4.82447781e-02, -3.01309209e-02, 2.53796838e-02,
2.65738703e-02, -3.88689935e-02, -2.08402588e-03,
-1.52736846e-02, -7.54594356e-02, 4.11440060e-03,
4.41256072e-03, 3.54635119e-02, 4.88590673e-02,
-3.12298350e-02, 2.79445928e-02, 7.31527014e-03,
1.59383491e-02, 5.15587777e-02, 5.07781319e-02,
-3.35258432e-02, -2.44219527e-02, 1.26995277e-02,
2.98856571e-02, -2.98038423e-02, 7.59773480e-04,
9.33393650e-03, -3.77792541e-32, -2.39051189e-02,
-2.28878502e-02, 3.86543348e-02, -4.30122055e-02,
8.96073431e-02, 9.54525098e-02, -4.57705837e-03,
4.72065918e-02, -6.21844418e-02, -1.19306045e-02,
1.98797379e-02, 4.54091839e-02, -2.91915964e-02,
-9.23778042e-02, -2.91774776e-02, 4.51411642e-02,
-8.98976848e-02, 4.10241038e-02, 3.40340845e-02,
-1.57277696e-02, 1.77023411e-02, 3.60725932e-02,
-4.42339331e-02, 4.37920578e-02, 6.02683015e-02,
2.81218141e-02, 1.11317988e-02, -2.04167254e-02,
-4.76017557e-02, 2.56126169e-02, 1.08535150e-02,
8.50802660e-02, -1.23381717e-02, 3.73017252e-03,
-1.08977873e-02, -4.22941968e-02, 5.40374368e-02,
4.60671121e-03, 4.81106900e-02, -1.18950218e-01,
-4.22098562e-02, 4.28526546e-04, -4.60485043e-03,
-6.80256188e-02, 2.47772243e-02, 5.72778247e-02,
3.39871198e-02, -3.80932167e-02, 5.46783023e-03,
1.50439683e-02, -1.71865243e-03, -4.49507032e-03,
8.36173818e-02, 3.61521989e-02, -2.27608792e-02,
1.09307785e-02, -4.64579873e-02, -2.51198392e-02,
3.10343467e-02, 1.40037164e-02, 2.80776825e-02,
-7.75458617e-03, -3.13465744e-02, 5.54159246e-02],
[-4.36194427e-02, 1.24176452e-03, -6.61615515e-03,
-3.61949764e-02, 3.40309553e-02, -1.04177661e-01,
1.18855372e-01, 8.33759233e-02, -5.25899744e-03,
7.02771470e-02, 7.44198263e-02, -1.92101020e-02,
4.22310680e-02, 4.77747917e-02, -1.00599825e-01,
6.59858715e-03, -7.06018973e-03, -5.78204431e-02,
-7.44119734e-02, -5.77753857e-02, 8.12644418e-03,
-1.05399890e-02, -9.23641995e-02, 2.72354316e-02,
-3.27183269e-02, 1.47155374e-02, 1.84647106e-02,
5.37966453e-02, 4.39368188e-03, -6.69799820e-02,
-1.14284605e-02, 1.01684907e-03, 5.48181348e-02,
-2.64710258e-03, -4.73516062e-02, 4.10528108e-03,
7.96805322e-03, -9.52281803e-03, 3.66993546e-02,
7.25318864e-02, -1.83794349e-02, -2.02359334e-02,
-1.38984248e-02, -3.02274209e-02, 7.08663929e-03,
-4.88152206e-02, -3.61666456e-02, -6.50164187e-02,
-2.85622478e-02, -9.76050645e-02, 1.71332173e-02,
-9.90536734e-02, -1.65603124e-02, -8.66571441e-03,
7.57850930e-02, -7.43833557e-02, -4.66333851e-02,
5.27069485e-03, 2.56001130e-02, 2.67603137e-02,
-9.87999048e-03, -1.49471418e-03, 1.99468490e-02,
9.64487065e-03, 4.01734933e-02, 4.46489155e-02,
-4.24114196e-03, 7.03191087e-02, -9.25385654e-02,
-8.32328647e-02, -5.34532070e-02, 2.76684705e-02,
6.16889596e-02, 9.97955073e-03, 1.30488157e-01,
-7.88196325e-02, 5.48366569e-02, -5.62950522e-02,
-1.81357767e-02, -1.50049117e-03, 1.65783260e-02,
-5.79603836e-02, 4.23630998e-02, 5.91657534e-02,
-4.30740678e-04, -6.59449119e-03, 2.93028932e-02,
4.89060879e-02, -8.74297693e-03, -2.56792922e-02,
-8.70074555e-02, -8.48818645e-02, 5.86376712e-02,
2.11759973e-02, -3.37857194e-02, -3.58547904e-02,
2.04802714e-02, -7.34518692e-02, -6.23515546e-02,
3.66165340e-01, -3.83556224e-02, 3.74275111e-02,
9.02115367e-03, 1.85298733e-02, 4.67686504e-02,
4.06234711e-02, -7.93088786e-03, 6.76765740e-02,
-3.08420043e-02, -4.76172380e-02, -6.74150065e-02,
5.09078540e-02, 2.75326315e-02, -2.45813709e-02,
8.87245592e-03, -7.46029168e-02, 4.39203195e-02,
2.12147366e-02, -1.49640366e-02, 5.55184335e-02,
1.81751400e-02, -1.19720697e-02, 4.10597995e-02,
2.85306498e-02, -3.33043486e-02, -5.93987033e-02,
2.64547132e-02, 5.21889888e-02, 6.39567301e-02,
-1.74697433e-02, 6.53330758e-02, 8.44449475e-02,
1.46098491e-02, 2.77910829e-02, 2.18893010e-02,
3.81155498e-02, 5.00993021e-02, 6.02446608e-02,
-5.22411205e-02, -3.40476073e-02, 4.75198925e-02,
7.90158957e-02, 5.34494109e-02, 2.22587548e-02,
-3.05935331e-02, -3.82138602e-02, 5.47443107e-02,
-1.31001957e-02, -2.66939006e-03, 1.02440163e-01,
-9.34778973e-02, 3.55974995e-02, 2.56248051e-03,
1.96040589e-02, 1.76399574e-02, -6.37881011e-02,
2.10898649e-02, -6.21709935e-02, 5.36505878e-02,
-1.70020177e-03, -6.84831440e-02, 4.26299907e-02,
-2.26416197e-02, -3.44322436e-02, -2.49185842e-02,
-5.42053720e-03, -8.74495506e-02, -4.67347987e-02,
-4.39893492e-02, 2.52047163e-02, 6.16509877e-02,
3.28156352e-02, 3.02153379e-02, -3.01138554e-02,
5.70706949e-02, 3.71646397e-02, -2.55124811e-02,
3.86465229e-02, 2.56904233e-02, -2.10288204e-02,
-3.99890617e-02, 9.25333891e-03, 2.41863634e-02,
2.41978820e-02, 3.20683494e-02, 9.09369998e-03,
-6.19302951e-02, -2.52413127e-04, 5.78331277e-02,
1.95391588e-02, 4.56315428e-02, 5.65279275e-02,
4.05421220e-02, 4.74190759e-03, 3.66835110e-02,
-4.67121229e-02, -2.21480671e-02, 9.98884439e-03,
6.83738431e-03, 9.44691896e-03, -6.52255071e-03,
4.46486063e-02, 6.32576346e-02, -2.71030832e-02,
1.25889271e-01, -6.38752878e-02, -4.20950092e-02,
4.77776937e-02, -4.44841944e-02, -1.71493925e-02,
2.39695460e-02, 3.07548344e-02, -2.57944651e-02,
-3.82859744e-02, -5.06829470e-03, -2.40309928e-02,
-2.80018896e-02, -1.89768746e-02, -9.35745165e-02,
3.23699228e-02, 1.44522507e-02, -2.76773721e-02,
-1.74342338e-02, 1.82664475e-32, -4.42017838e-02,
3.35162766e-02, -2.57008132e-02, -3.35009322e-02,
8.01710505e-03, 1.55778555e-03, -4.58203023e-03,
9.85369831e-03, 6.60311207e-02, 6.78686276e-02,
4.81087193e-02, 1.47069143e-02, 8.06112811e-02,
-3.42556052e-02, -3.26208980e-03, -1.04057109e-02,
6.25951067e-02, 3.04814987e-02, -4.93760630e-02,
4.47589569e-02, 2.63141748e-03, -4.71894406e-02,
-6.37669340e-02, 1.10649662e-02, -3.39122867e-04,
3.07911392e-02, 2.02936120e-03, -2.82919053e-02,
6.19907910e-03, 4.30973247e-02, -3.54050770e-02,
3.62614393e-02, -8.36105272e-02, 9.83160883e-02,
-1.41893774e-02, -2.16325819e-02, -5.90246916e-02,
-7.55953789e-02, -9.38678568e-04, -8.17080867e-03,
-3.67823034e-03, 2.24777292e-02, -1.08326405e-01,
3.02967057e-02, -1.06457062e-01, 2.84283571e-02,
-4.95225005e-02, 7.92975575e-02, 2.33978480e-02,
-6.50940388e-02, 3.94302718e-02, -1.58559792e-02,
-1.06494315e-02, -8.48078728e-02, -6.19805977e-02,
4.77776974e-02, -2.71212626e-02, -4.78804484e-02,
-3.58883590e-02, 6.07158653e-02, 2.55191140e-02,
1.44867611e-03, 7.76907080e-04, 8.79666954e-02,
4.31705862e-02, 5.69744147e-02, 1.89087819e-02,
9.71703306e-02, -3.89649533e-02, -6.06040657e-02,
-8.53620917e-02, 4.91853729e-02, -5.25182597e-02,
-8.83183535e-03, 1.13622351e-02, 1.24287233e-01,
4.96100970e-02, 2.04096250e-02, 3.96146439e-02,
2.58912006e-03, 2.56173536e-02, 2.37003770e-02,
1.09122135e-01, 5.56674488e-02, -2.25001667e-02,
-2.06263755e-02, 4.14560810e-02, 3.01604299e-03,
2.19393875e-02, -4.92635183e-03, 8.85570049e-03,
8.06190306e-04, -3.42081115e-02, -1.81108322e-02,
-4.62027900e-02, -3.07123789e-32, -1.18308932e-01,
2.00220402e-02, 2.60014459e-02, -5.92406467e-03,
6.24214038e-02, 1.60534885e-02, 9.21495184e-02,
1.08812598e-03, 9.76104010e-03, 5.86248823e-02,
-2.83072796e-02, 6.73462749e-02, 1.28318369e-02,
-2.28056437e-04, 4.11208421e-02, -6.43146262e-02,
-4.27601524e-02, 4.50465688e-03, 6.53191581e-02,
1.47612672e-02, 7.81147033e-02, 4.49998751e-02,
-7.58396974e-03, -2.98437029e-02, 7.98098817e-02,
-4.78304438e-02, -5.87424710e-02, -1.09326974e-01,
-2.58829035e-02, -6.28677979e-02, 1.17499895e-01,
-6.22401349e-02, -7.96295106e-02, -5.29675223e-02,
-1.55016296e-02, 3.20039093e-02, -1.53178414e-02,
-4.22603078e-02, 3.42442207e-02, 1.23893470e-03,
2.29211226e-02, 3.36142704e-02, 6.51369318e-02,
-2.17055287e-02, 2.06595957e-02, -3.44997901e-03,
-1.52502526e-02, 2.87910309e-02, 2.59440895e-02,
-3.58282961e-02, -4.73467149e-02, 2.20816545e-02,
2.24817935e-02, -4.73678149e-02, 3.65528986e-02,
-1.30275814e-02, -5.71536878e-03, -2.98424512e-02,
-9.99975577e-02, 7.44160563e-02, -3.12953182e-02,
-6.30516559e-02, -4.87827696e-02, 1.25094848e-02]], dtype=float32)
In [0]:
Copied!
with mlflow.start_run() as run:
model_name = "transformer-model"
registered_model_name=f"development.default.{model_name}"
model = TransformerWrapper()
model.load_context({})
pip_reqs = mlflow.pyfunc.get_default_conda_env()
pip_reqs["dependencies"][-1]["pip"] += [
"sentence-transformers=={}".format(sentence_transformers.__version__),
]
sample_inputs = pd.DataFrame({"text": ["Hello", "world"]})
log_result = mlflow.pyfunc.log_model(
model_name,
python_model=model,
conda_env=pip_reqs,
input_example=sample_inputs,
signature=infer_signature(sample_inputs, model.predict(None, sample_inputs)),
registered_model_name=registered_model_name,
)
with mlflow.start_run() as run:
model_name = "transformer-model"
registered_model_name=f"development.default.{model_name}"
model = TransformerWrapper()
model.load_context({})
pip_reqs = mlflow.pyfunc.get_default_conda_env()
pip_reqs["dependencies"][-1]["pip"] += [
"sentence-transformers=={}".format(sentence_transformers.__version__),
]
sample_inputs = pd.DataFrame({"text": ["Hello", "world"]})
log_result = mlflow.pyfunc.log_model(
model_name,
python_model=model,
conda_env=pip_reqs,
input_example=sample_inputs,
signature=infer_signature(sample_inputs, model.predict(None, sample_inputs)),
registered_model_name=registered_model_name,
)
/databricks/python/lib/python3.11/site-packages/huggingface_hub/file_download.py:795: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn( /databricks/python/lib/python3.11/site-packages/huggingface_hub/file_download.py:795: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. warnings.warn(
Uploading artifacts: 0%| | 0/11 [00:00<?, ?it/s]
Registered model 'development.default.transformer-model' already exists. Creating a new version of this model...
Uploading artifacts: 0%| | 0/11 [00:00<?, ?it/s]
Created version '27' of model 'development.default.transformer-model'.
🏃 View run overjoyed-skunk-870 at: https://dbc-f44ef3dc-50d6.cloud.databricks.com/ml/experiments/3140073489088096/runs/24397d5256d44909855605a149c4cf95 🧪 View experiment at: https://dbc-f44ef3dc-50d6.cloud.databricks.com/ml/experiments/3140073489088096
Deploy model to serving endpoint¶
In [0]:
Copied!
from mlflow.deployments import get_deploy_client
client = get_deploy_client("databricks")
endpoint = client.create_endpoint(
config={
"name": model_name,
"config": {
"served_entities": [
{
"entity_name": registered_model_name,
"entity_version": log_result.registered_model_version,
"workload_size": "Small",
"scale_to_zero_enabled": True
}
],
}
}
)
from mlflow.deployments import get_deploy_client
client = get_deploy_client("databricks")
endpoint = client.create_endpoint(
config={
"name": model_name,
"config": {
"served_entities": [
{
"entity_name": registered_model_name,
"entity_version": log_result.registered_model_version,
"workload_size": "Small",
"scale_to_zero_enabled": True
}
],
}
}
)
Test serving endpoint¶
In [0]:
Copied!
import requests
response = requests.post(
'<your_databricks_workspace_uri>/serving-endpoints/transformer-model/invocations',
headers={
'Authorization': 'Bearer <your_databricks_access_token>',
'Content-Type': 'application/json',
},
json={"inputs": {"text": ["hello", "world"]}},
)
import requests
response = requests.post(
'/serving-endpoints/transformer-model/invocations',
headers={
'Authorization': 'Bearer ',
'Content-Type': 'application/json',
},
json={"inputs": {"text": ["hello", "world"]}},
)
In [0]:
Copied!
response.json()
response.json()
Out[0]:
{'predictions': [[-0.07490699738264084,
-0.014433098025619984,
0.04864982143044472,
-0.027138084173202515,
-0.04308821260929108,
-0.14776337146759033,
0.06883306056261063,
-0.013703778386116028,
-0.05356861278414726,
0.026975223794579506,
-0.006283458322286606,
-0.039883460849523544,
0.007656780071556568,
-0.037808965891599655,
-0.011755740270018578,
-0.034640952944755554,
0.12935712933540344,
-0.025277819484472275,
-0.015209856443107128,
0.0007302461308427155,
-0.08068878948688507,
0.02693794295191765,
-0.09873572736978531,
-0.03410757705569267,
-0.027029380202293396,
-0.07320035248994827,
0.050858866423368454,
-0.017256302759051323,
0.0728219524025917,
-0.12328876554965973,
-0.010971282608807087,
0.015405534766614437,
0.16192539036273956,
-0.03713918477296829,
-0.017285741865634918,
0.026162028312683105,
0.07309547066688538,
-0.046671558171510696,
-0.04061464965343475,
0.05917086452245712,
0.016567181795835495,
0.007346348371356726,
-0.04897535219788551,
0.06898930668830872,
0.04286482557654381,
-0.07572472840547562,
-0.015094233676791191,
0.04190894961357117,
0.05491500347852707,
-0.05639542266726494,
-0.010260515846312046,
0.0018722371896728873,
-0.007897211238741875,
0.05324821174144745,
0.08370453864336014,
0.033194251358509064,
-0.015426664613187313,
0.06000210717320442,
0.028092045336961746,
0.022526009008288383,
0.014536309987306595,
0.024353453889489174,
-0.0066866218112409115,
-0.07501190900802612,
-0.009480273351073265,
-0.011537594720721245,
-0.019241631031036377,
-0.013632326386868954,
0.08455577492713928,
-0.04682129994034767,
-0.0024661505594849586,
-0.021389581263065338,
-0.001171510899439454,
-0.07554257661104202,
-0.042470790445804596,
0.014182256534695625,
0.0182692501693964,
0.0445626899600029,
-0.0061446442268788815,
-0.011601988226175308,
0.031707294285297394,
-0.09668205678462982,
0.04129583761096001,
0.03993024304509163,
0.045429471880197525,
0.038931671530008316,
0.049543287605047226,
0.04893716052174568,
0.03068866953253746,
0.020303497090935707,
-0.06820891052484512,
0.03905018791556358,
0.01892983540892601,
-0.01603829860687256,
-0.010620113462209702,
0.032971322536468506,
0.03548093140125275,
0.02791607938706875,
-0.11154036968946457,
0.3489684760570526,
-0.03556126728653908,
-0.004910522140562534,
-0.056758683174848557,
0.08803814649581909,
-0.041703563183546066,
0.018591314554214478,
-0.03914238139986992,
-0.0024382700212299824,
0.08891012519598007,
0.001875347807072103,
0.03373568877577782,
-0.006684228777885437,
-0.07496774196624756,
-0.0037256807554513216,
0.018774284049868584,
-0.015769436955451965,
0.027898509055376053,
0.025856008753180504,
-0.005718715023249388,
0.016867058351635933,
-0.013205739669501781,
-0.02696979232132435,
-0.019299568608403206,
-0.07414783537387848,
0.04917405545711517,
-0.12124433368444443,
-0.02418995089828968,
0.04119341820478439,
0.002869400894269347,
-0.015186090022325516,
0.05777108669281006,
0.10057926923036575,
-0.030659982934594154,
-0.04149603098630905,
-0.005284868646413088,
0.01306508481502533,
0.04405050352215767,
-0.0051895249634981155,
0.020648524165153503,
-0.052872005850076675,
-0.03616846352815628,
0.06248427927494049,
-0.0014878313522785902,
0.00926207099109888,
0.04920574277639389,
-0.0804261863231659,
0.008985195308923721,
0.04294600710272789,
0.007897174917161465,
0.043535731732845306,
0.025803374126553535,
0.05883082002401352,
0.04740481451153755,
0.01499905250966549,
0.06574149429798126,
-0.03741220757365227,
0.06796222925186157,
-0.0040621040388941765,
0.09122632443904877,
-0.03309017792344093,
-0.039427366107702255,
0.0320153534412384,
0.013884534128010273,
-0.019970964640378952,
0.008319060318171978,
-0.06097748130559921,
0.02522207796573639,
-0.047170598059892654,
0.009228195995092392,
0.013819687999784946,
0.06146702170372009,
-0.061159905046224594,
-0.05323576182126999,
0.0005353422602638602,
0.016176732257008553,
-0.017424793913960457,
0.019698934629559517,
0.012111975811421871,
-0.02494429424405098,
0.026211420074105263,
-0.09603825956583023,
0.040690306574106216,
-0.06900645047426224,
-0.00519044604152441,
-0.03728567436337471,
0.00418417202308774,
-0.014595062471926212,
0.04664258286356926,
0.017251286655664444,
0.07027334719896317,
0.016699088737368584,
0.024759477004408836,
0.01833745278418064,
-0.014456599950790405,
-0.0008250173414126039,
0.035498131066560745,
0.0594661645591259,
-0.01988360285758972,
0.026721032336354256,
-0.03427431359887123,
-0.08479491621255875,
0.04466331750154495,
0.03931319713592529,
0.10543838143348694,
0.14874611794948578,
-0.08425173163414001,
0.038116391748189926,
0.03423473984003067,
0.030597474426031113,
-0.028007375076413155,
-0.015794433653354645,
0.01617855206131935,
-0.004092052578926086,
0.02188558503985405,
-0.08329065889120102,
-0.07715725898742676,
-0.03638751059770584,
0.06628945469856262,
-0.06134922057390213,
0.011612842790782452,
0.02288518100976944,
-0.05588105320930481,
-0.007116343826055527,
1.7152831199614082e-32,
0.07951879501342773,
0.040328603237867355,
-0.09235884994268417,
-0.07319997251033783,
-0.016840526834130287,
0.029828080907464027,
0.027244415134191513,
0.08627544343471527,
-0.07153411954641342,
0.027022214606404305,
-0.02351139672100544,
-0.020681945607066154,
0.015285310335457325,
0.021789120510220528,
-0.012657093815505505,
-0.04810015484690666,
0.09918852895498276,
-0.033485833555459976,
-0.028402702882885933,
0.04437868669629097,
-0.03449517861008644,
0.017841428518295288,
-0.015507213771343231,
0.03241613134741783,
-0.015170932747423649,
0.02081073448061943,
0.01580744981765747,
0.025557080283761024,
-0.07905921339988708,
0.020325614139437675,
0.019913263618946075,
0.11545098572969437,
0.017332082614302635,
0.0800543874502182,
0.035080667585134506,
0.05387743189930916,
-0.05259888991713524,
-0.11504394561052322,
0.005644102115184069,
-0.06016472354531288,
-0.08956320583820343,
-0.021210627630352974,
-0.03718503937125206,
0.07307112962007523,
-0.01122485101222992,
-0.028362151235342026,
-0.07111763954162598,
0.020100124180316925,
0.02514040656387806,
-0.08029009401798248,
-0.081744484603405,
-0.033766765147447586,
-0.01365615800023079,
-0.0650678500533104,
-0.07094944268465042,
-0.006518734153360128,
-0.08320773392915726,
0.05150030553340912,
-0.0010022938949987292,
-0.021197376772761345,
-0.0052722059190273285,
0.042449068278074265,
0.12382979691028595,
0.061516474932432175,
0.04515837877988815,
-0.0008268920355476439,
0.04925084486603737,
0.019446123391389847,
0.05081614851951599,
-0.028261229395866394,
-0.04824485629796982,
-0.030130919069051743,
0.025379657745361328,
0.02657392807304859,
-0.03886903449892998,
-0.0020840074867010117,
-0.01527369860559702,
-0.07545941323041916,
0.00411444203928113,
0.0044125961139798164,
0.03546355664730072,
0.04885902255773544,
-0.03122977912425995,
0.027944518253207207,
0.007315258029848337,
0.01593836583197117,
0.05155874788761139,
0.05077812820672989,
-0.03352586179971695,
-0.02442193031311035,
0.012699450366199017,
0.02988562174141407,
-0.029803799465298653,
0.0007597211515530944,
0.009333984926342964,
-3.777926289958741e-32,
-0.023905033245682716,
-0.022887805476784706,
0.03865432366728783,
-0.04301220178604126,
0.0896073505282402,
0.0954524502158165,
-0.004577081184834242,
0.0472065694630146,
-0.0621844120323658,
-0.01193063985556364,
0.01987975463271141,
0.045409198850393295,
-0.029191559180617332,
-0.09237784147262573,
-0.029177412390708923,
0.04514112323522568,
-0.08989769220352173,
0.04102414846420288,
0.034034136682748795,
-0.01572774536907673,
0.017702341079711914,
0.03607259690761566,
-0.04423388093709946,
0.043792106211185455,
0.06026824936270714,
0.02812189981341362,
0.011131846345961094,
-0.02041669376194477,
-0.04760183021426201,
0.02561265043914318,
0.0108534786850214,
0.08508025854825974,
-0.012338134460151196,
0.0037301077973097563,
-0.01089777983725071,
-0.04229413717985153,
0.05403749272227287,
0.004606732167303562,
0.04811084270477295,
-0.11895018070936203,
-0.042209822684526443,
0.0004284721508156508,
-0.004604796413332224,
-0.0680256113409996,
0.02477726712822914,
0.05727783963084221,
0.033987127244472504,
-0.03809322416782379,
0.005467874463647604,
0.015043956227600574,
-0.0017186638433486223,
-0.004495037253946066,
0.08361738175153732,
0.0361521877348423,
-0.022760873660445213,
0.010930786840617657,
-0.04645797237753868,
-0.02511979639530182,
0.031034382060170174,
0.014003747142851353,
0.028077704831957817,
-0.007754513528198004,
-0.03134661912918091,
0.05541597679257393],
[-0.043619487434625626,
0.0012417499674484134,
-0.0066161444410681725,
-0.036194946616888046,
0.034030936658382416,
-0.10417760163545609,
0.11885534971952438,
0.08337587863206863,
-0.005258931312710047,
0.070277139544487,
0.0744197890162468,
-0.019210105761885643,
0.04223107919096947,
0.04777485132217407,
-0.10059987753629684,
0.006598544307053089,
-0.007060231640934944,
-0.05782045051455498,
-0.07441194355487823,
-0.05777536332607269,
0.008126461878418922,
-0.010540020652115345,
-0.09236419945955276,
0.02723541297018528,
-0.03271832689642906,
0.01471550390124321,
0.018464770168066025,
0.05379663035273552,
0.004393669776618481,
-0.06697997450828552,
-0.011428340338170528,
0.0010168392909690738,
0.054818131029605865,
-0.002647097920998931,
-0.04735160991549492,
0.004105249419808388,
0.007968065328896046,
-0.009522838518023491,
0.036699313670396805,
0.07253183424472809,
-0.018379341810941696,
-0.020235950127243996,
-0.013898388482630253,
-0.030227484181523323,
0.007086615078151226,
-0.04881523177027702,
-0.03616662696003914,
-0.06501641124486923,
-0.02856222726404667,
-0.09760506451129913,
0.017133252695202827,
-0.09905363619327545,
-0.01656031236052513,
-0.008665686473250389,
0.07578513771295547,
-0.07438334822654724,
-0.04663345217704773,
0.005270696710795164,
0.02560010366141796,
0.026760296896100044,
-0.009879945777356625,
-0.001494653057307005,
0.019946886226534843,
0.00964483805000782,
0.040173474699258804,
0.0446489192545414,
-0.004241237882524729,
0.07031901925802231,
-0.09253857284784317,
-0.08323287963867188,
-0.053453229367733,
0.027668427675962448,
0.06168895214796066,
0.00997957680374384,
0.13048815727233887,
-0.07881958782672882,
0.05483667179942131,
-0.05629505217075348,
-0.018135804682970047,
-0.0015004880260676146,
0.016578253358602524,
-0.057960394769907,
0.04236307367682457,
0.05916578695178032,
-0.0004307743802201003,
-0.006594537757337093,
0.029302965849637985,
0.04890609532594681,
-0.008742976002395153,
-0.025679295882582664,
-0.08700742572546005,
-0.08488191664218903,
0.058637700974941254,
0.021175989881157875,
-0.03378574550151825,
-0.03585478663444519,
0.020480262115597725,
-0.07345186173915863,
-0.06235155835747719,
0.36616525053977966,
-0.03835558891296387,
0.037427544593811035,
0.00902115274220705,
0.01852991431951523,
0.04676874354481697,
0.04062347114086151,
-0.007930848747491837,
0.0676766186952591,
-0.030842097476124763,
-0.04761723801493645,
-0.06741496175527573,
0.05090789496898651,
0.027532676234841347,
-0.02458137646317482,
0.008872517384588718,
-0.0746028795838356,
0.043920282274484634,
0.021214721724390984,
-0.014963989146053791,
0.05551842972636223,
0.018175145611166954,
-0.011972051113843918,
0.04105980321764946,
0.028530629351735115,
-0.03330439701676369,
-0.059398800134658813,
0.026454735547304153,
0.052189040929079056,
0.06395671516656876,
-0.017469698563218117,
0.06533303111791611,
0.08444498479366302,
0.014609848149120808,
0.027791092172265053,
0.021889295428991318,
0.03811546415090561,
0.05009929835796356,
0.06024470552802086,
-0.052241139113903046,
-0.034047674387693405,
0.04751984402537346,
0.07901585102081299,
0.0534493625164032,
0.02225879393517971,
-0.03059348836541176,
-0.0382138267159462,
0.054744284600019455,
-0.01310022547841072,
-0.0026693802792578936,
0.1024402305483818,
-0.09347786754369736,
0.03559749200940132,
0.002562500536441803,
0.01960403099656105,
0.017639940604567528,
-0.06378811597824097,
0.02108987607061863,
-0.062171004712581635,
0.05365058407187462,
-0.0017001954838633537,
-0.06848318874835968,
0.042629968374967575,
-0.022641649469733238,
-0.034432314336299896,
-0.02491859719157219,
-0.005420498084276915,
-0.0874495580792427,
-0.04673486575484276,
-0.04398937150835991,
0.025204751640558243,
0.06165098771452904,
0.03281563147902489,
0.03021535649895668,
-0.030113868415355682,
0.05707062408328056,
0.037164609879255295,
-0.025512410327792168,
0.03864650800824165,
0.025690311565995216,
-0.0210287943482399,
-0.03998909145593643,
0.009253322146832943,
0.024186354130506516,
0.024197867140173912,
0.03206837922334671,
0.009093680419027805,
-0.06193026527762413,
-0.0002524293668102473,
0.05783316120505333,
0.01953911781311035,
0.04563150182366371,
0.056527938693761826,
0.04054207727313042,
0.004741905257105827,
0.036683544516563416,
-0.046712130308151245,
-0.0221481341868639,
0.009988783858716488,
0.006837444845587015,
0.009446898475289345,
-0.006522462237626314,
0.04464865103363991,
0.0632576048374176,
-0.027103060856461525,
0.12588931620121002,
-0.06387529522180557,
-0.04209504649043083,
0.04777773097157478,
-0.04448413476347923,
-0.01714937575161457,
0.023969532921910286,
0.03075478971004486,
-0.02579452097415924,
-0.03828594461083412,
-0.005068281199783087,
-0.024031026288866997,
-0.028001818805933,
-0.018976887688040733,
-0.09357448667287827,
0.0323699451982975,
0.014452285133302212,
-0.027677327394485474,
-0.01743423193693161,
1.826644018096939e-32,
-0.04420176520943642,
0.03351627662777901,
-0.025700803846120834,
-0.033500876277685165,
0.008017118088901043,
0.0015577797312289476,
-0.0045820362865924835,
0.009853694587945938,
0.0660310685634613,
0.06786859780550003,
0.04810873046517372,
0.014706932939589024,
0.08061127364635468,
-0.03425562009215355,
-0.0032620378769934177,
-0.010405578650534153,
0.06259510666131973,
0.03048151358962059,
-0.04937605932354927,
0.044758982956409454,
0.0026314116548746824,
-0.04718945175409317,
-0.0637669637799263,
0.011065009981393814,
-0.0003391436766833067,
0.030791115015745163,
0.0020293472334742546,
-0.02829188108444214,
0.006199137773364782,
0.04309729114174843,
-0.03540507331490517,
0.03626145422458649,
-0.08361049741506577,
0.09831610321998596,
-0.014189377427101135,
-0.021632594987750053,
-0.05902469903230667,
-0.07559533417224884,
-0.0009387145400978625,
-0.008170796558260918,
-0.0036782182287424803,
0.02247774228453636,
-0.10832644999027252,
0.03029669262468815,
-0.10645706951618195,
0.028428398072719574,
-0.04952250048518181,
0.07929759472608566,
0.02339784801006317,
-0.06509396433830261,
0.03943025320768356,
-0.01585594192147255,
-0.010649466887116432,
-0.08480783551931381,
-0.06198066473007202,
0.047777675092220306,
-0.0271212887018919,
-0.04788042977452278,
-0.035888317972421646,
0.06071586534380913,
0.025519084185361862,
0.0014486348954960704,
0.0007769481744617224,
0.0879666805267334,
0.04317055270075798,
0.05697445198893547,
0.01890879124403,
0.09717031568288803,
-0.038964975625276566,
-0.06060411408543587,
-0.08536212891340256,
0.049185384064912796,
-0.052518267184495926,
-0.008831865154206753,
0.011362263932824135,
0.1242871880531311,
0.04961008578538895,
0.020409662276506424,
0.0396147258579731,
0.002589166397228837,
0.02561739832162857,
0.02370036579668522,
0.10912219434976578,
0.055667441338300705,
-0.02250012196600437,
-0.020626289770007133,
0.041456013917922974,
0.0030159817542880774,
0.02193944901227951,
-0.0049263178370893,
0.00885569117963314,
0.000806207477580756,
-0.03420805186033249,
-0.018110817298293114,
-0.046202801167964935,
-3.071240539016284e-32,
-0.11830893158912659,
0.020022038370370865,
0.026001455262303352,
-0.005923999007791281,
0.06242138519883156,
0.01605343259871006,
0.09214948862791061,
0.001088106888346374,
0.009761103428900242,
0.058624837547540665,
-0.028307264670729637,
0.06734627485275269,
0.012831809930503368,
-0.00022808021458331496,
0.04112078621983528,
-0.06431462615728378,
-0.04276016354560852,
0.004504645708948374,
0.06531918793916702,
0.014761262573301792,
0.07811468839645386,
0.04499991238117218,
-0.007583969738334417,
-0.02984367497265339,
0.07980983704328537,
-0.047830499708652496,
-0.05874243006110191,
-0.10932701081037521,
-0.025882873684167862,
-0.06286776065826416,
0.11749985814094543,
-0.062240105122327805,
-0.07962953299283981,
-0.052967511117458344,
-0.015501618385314941,
0.032003920525312424,
-0.015317771583795547,
-0.04226027429103851,
0.03424416854977608,
0.0012389235198497772,
0.022921176627278328,
0.033614207059144974,
0.06513688713312149,
-0.02170555107295513,
0.020659614354372025,
-0.0034499401226639748,
-0.015250272117555141,
0.028791051357984543,
0.025944093242287636,
-0.03582833707332611,
-0.047346632927656174,
0.022081641480326653,
0.022481795400381088,
-0.04736781492829323,
0.03655286505818367,
-0.013027559965848923,
-0.005715334787964821,
-0.029842428863048553,
-0.09999756515026093,
0.07441607862710953,
-0.03129531070590019,
-0.06305164843797684,
-0.04878274351358414,
0.012509497813880444]]}