Performing Sentence Similarity By Leveraging Hugging Face APIs
How To Perform API Calls to hugging Face APIs to leverage the existing trained models
Sentence Similarity is the task of determining how similar two texts are. Sentence similarity models convert input texts into vectors (embeddings) that capture semantic information and calculate how close (similar) they are between them. Techniques like cosine similarities can score the sentence against the reference sentences.
While the official website provides the details of how to download the model in a python environment and use it, API calls can also be used for quick and easy demonstration purposes. Use the following code block to perform the same
import json
import requestsAPI_URL = "https://api-inference.huggingface.co/models/sentence-transformers/msmarco-distilbert-base-tas-b"
headers = {"Authorization": f"Bearer {api_token}"}def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()data = query(
{
"inputs": {
"source_sentence": "That is a happy person",
"sentences": [
"That is a happy dog",
"That is a very happy person",
"Today is a sunny day"
]
}
}
## [0.853, 0.981, 0.655]
Q: How to get an API token to run the above code?
- Sign up for the hugging face website.
- Go to profile → setting → access tokens.
3. Copy the access token and paste it into the code block. Hence the final code block should look something like this
import json
import requestsAPI_URL = "https://api-inference.huggingface.co/models/sentence-transformers/msmarco-distilbert-base-tas-b"
headers = {"Authorization": f"Bearer {'hf_fwxDRGtZMVwaptvmHwtlnWb***********************'}"}def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return print(response.json())data = query(
{
"inputs": {
"source_sentence": "That is a happy person",
"sentences": [
"That is a happy dog",
"That is a very happy person",
"Today is a sunny day"
]
}
}
That's it, you are now good to run it in any python code environment.
The Sentence Transformers library
The Sentence Transformers library is very powerful for calculating embeddings of sentences, paragraphs, and entire documents. An embedding is just a vector representation of a text and is useful for finding how similar two texts are.
You can find and use hundreds of Sentence Transformers models from the Hub by directly using the library, playing with the widgets in the browser or using the Inference API.
I really hope you guys learned something from this post. Feel free to 👏 if you like what you learned. Let me know if there is anything you need my help with.
