Langchain
If you're using Langchain, you can log your data to Athina with just a few lines of code.
All you need to do is to add the CallbackHandler
to your LLMChain
callbacks.
Install Athina Logging SDK
pip install athina-logger
Import Athina classes and set API key
from athina_logger.api_key import AthinaApiKey
from athina_logger.langchain_handler import CallbackHandler
AthinaApiKey.set_api_key(os.getenv('ATHINA_API_KEY'))
Instantiate the CallbackHandler
with Athina metadata properties
athina_handler = CallbackHandler(
prompt_slug='customer-query-prompt/v1',
user_query='I would like to get a refund on a pair of shoes I purchased online',
environment='production',
session_id='1234',
customer_id='nike-usa',
customer_user_id='tim@apple.com',
external_reference_id='your-reference-id',
custom_attributes= {
"loggedBy": "John Doe",
"age": 24,
"isAdmin": true,
"references": null
# any other attribute to be logged
}
kwargs: Any, # Any key-value data you want to associate with the LLM calls in a chain
)
prompt_slug
: An identifier for the prompt that is being used.user_query
: The query that the user entered to the LLM.environment
: The environment in which the LLM is running. For example,production
ordevelopment
.session_id
: The session ID of the LLM. This is used to group multiple LLM calls together.customer_id
: The ID of the customer that is using the LLM.customer_user_id
: The ID of the user that is using the LLM.external_reference_id
: The ID of the external reference that is using the LLM.custom_attributes
: Any key-value data you want to associate with the LLM callkwargs
: Any key-value data you want to associate with the LLM calls in a chain. This key-value data will be stored as context in Athina Server
Sample kwargs:
context1 = "Germany is located in central europe"
context2 = "Berlin is the capital of Germany"
This will be stored as:
{
"context1": "Germany is located in central europe",
"context2": "Berlin is the capital of Germany"
}
This will be perceived as retrieved context
Add CallbackHandler
to LLMChain
callbacks
chain = LLMChain(
llm=ChatOpenAI(...otherProperties, callbacks=[athina_handler]),
prompt=chat_prompt,
)
chain.run('OpenAI')
Supported Models (Without Streaming)
text-davinci-003
gpt-3.5-turbo
gpt-3.5-turbo-0613
gpt-3.5-turbo-16k
gpt-3.5-turbo-16k-0613
gpt-3.5-turbo-1106
gpt-4
gpt-4-0613
gpt-4-32k
gpt-4-32k-0613
gpt-4-1106-preview
meta-llama/Llama-2-13b
meta-llama/Llama-2-13b-chat
meta-llama/Llama-2-13b-chat-hf
meta-llama/Llama-2-13b-hf
meta-llama/Llama-2-70b
meta-llama/Llama-2-70b-chat
meta-llama/Llama-2-70b-chat-hf
meta-llama/Llama-2-70b-hf
meta-llama/Llama-2-7b
meta-llama/Llama-2-7b-chat
meta-llama/Llama-2-7b-chat-hf
meta-llama/Llama-2-7b-hf
claude-2
Supported Models (With Streaming)
text-davinci-003
gpt-3.5-turbo
gpt-3.5-turbo-0613
gpt-3.5-turbo-16k
gpt-3.5-turbo-16k-0613
gpt-3.5-turbo-1106
gpt-4
gpt-4-0613
gpt-4-32k
gpt-4-32k-0613
gpt-4-1106-preview
Not using Python?
Reach out to us at hello@athina.ai - we're happy to add support for other stacks as well if we hear from you.