Langchain

Langchain

If you're using Langchain, you can log your data to Athina with just a few lines of code.

All you need to do is to add the CallbackHandler to your LLMChain callbacks.

Install Athina Logging SDK

pip install athina-logger

Import Athina classes and set API key

from athina_logger.api_key import AthinaApiKey
from athina_logger.langchain_handler import CallbackHandler
 
AthinaApiKey.set_api_key(os.getenv('ATHINA_API_KEY'))

Instantiate the CallbackHandler with Athina metadata properties

athina_handler = CallbackHandler(
    prompt_slug='customer-query-prompt/v1',
    user_query='I would like to get a refund on a pair of shoes I purchased online',
    environment='production',
    session_id='1234',
    customer_id='nike-usa',
    customer_user_id='tim@apple.com',
    external_reference_id='your-reference-id',
    custom_attributes= {
        "loggedBy": "John Doe",
        "age": 24,
        "isAdmin": true,
        "references": null
        # any other attribute to be logged
    }
    kwargs: Any, # Any key-value data you want to associate with the LLM calls in a chain
)
  • prompt_slug: An identifier for the prompt that is being used.
  • user_query: The query that the user entered to the LLM.
  • environment: The environment in which the LLM is running. For example, production or development.
  • session_id: The session ID of the LLM. This is used to group multiple LLM calls together.
  • customer_id: The ID of the customer that is using the LLM.
  • customer_user_id: The ID of the user that is using the LLM.
  • external_reference_id: The ID of the external reference that is using the LLM.
  • custom_attributes: Any key-value data you want to associate with the LLM call
  • kwargs: Any key-value data you want to associate with the LLM calls in a chain. This key-value data will be stored as context in Athina Server
Sample kwargs:
context1 = "Germany is located in central europe"
context2 = "Berlin is the capital of Germany"
 
This will be stored as:
 
{
    "context1": "Germany is located in central europe",
    "context2": "Berlin is the capital of Germany"
}
 
This will be perceived as retrieved context

Add CallbackHandler to LLMChain callbacks

chain = LLMChain(
    llm=ChatOpenAI(...otherProperties, callbacks=[athina_handler]),
    prompt=chat_prompt,
)
chain.run('OpenAI')

Supported Models (Without Streaming)

  • text-davinci-003
  • gpt-3.5-turbo
  • gpt-3.5-turbo-0613
  • gpt-3.5-turbo-16k
  • gpt-3.5-turbo-16k-0613
  • gpt-3.5-turbo-1106
  • gpt-4
  • gpt-4-0613
  • gpt-4-32k
  • gpt-4-32k-0613
  • gpt-4-1106-preview
  • meta-llama/Llama-2-13b
  • meta-llama/Llama-2-13b-chat
  • meta-llama/Llama-2-13b-chat-hf
  • meta-llama/Llama-2-13b-hf
  • meta-llama/Llama-2-70b
  • meta-llama/Llama-2-70b-chat
  • meta-llama/Llama-2-70b-chat-hf
  • meta-llama/Llama-2-70b-hf
  • meta-llama/Llama-2-7b
  • meta-llama/Llama-2-7b-chat
  • meta-llama/Llama-2-7b-chat-hf
  • meta-llama/Llama-2-7b-hf
  • claude-2

Supported Models (With Streaming)

  • text-davinci-003
  • gpt-3.5-turbo
  • gpt-3.5-turbo-0613
  • gpt-3.5-turbo-16k
  • gpt-3.5-turbo-16k-0613
  • gpt-3.5-turbo-1106
  • gpt-4
  • gpt-4-0613
  • gpt-4-32k
  • gpt-4-32k-0613
  • gpt-4-1106-preview

Not using Python?

Reach out to us at hello@athina.ai - we're happy to add support for other stacks as well if we hear from you.