Log via Python Sdk

Log via Python SDK

1. Install the Python SDK

Run pip install athina-logger

2. Import Athina Logger

from athina_logger.api_key import AthinaApiKey

3. Import Openai package

 
# For openai < 1
 
import openai
openai.api_key = os.getenv('OPENAI_API_KEY')
 
# For openai >=1
 
from openai import OpenAI
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))

4. Set Athina API key

AthinaApiKey.set_api_key(os.getenv('ATHINA_API_KEY'))

5. Call the logging Function

For OpenAI v0.x

from athina_logger.inference_logger import InferenceLogger
from athina_logger.exception.custom_exception import CustomException
 
response = openai.ChatCompletion.create(
        model='gpt-4-1106-preview',
        messages=[{"role": "user", "content": "What is machine learning?"}],
    )
 
try:
    InferenceLogger.log_inference(
        prompt_slug="sdk_test",
        prompt=messages,
        language_model_id="gpt-4-1106-preview",
        response=response,
        cost=cost,
        external_reference_id="abc",
        custom_attributes={
                "name": "John Doe"
                # Your custom attributes
            }
    )
except Exception as e:
    if isinstance(e, CustomException):
        print(e.status_code)
        print(e.message)
    else:
        print(e)
```python
 
#### For OpenAI 1.x
  ```python
# For openai > 1
response = client.chat.completions.create(
        model='gpt-4-1106-preview',
        messages=[{"role": "user", "content": "What is machine learning?"}],
    )
 
response = response.model_dump() # For openai > 1 version
 
try:
    InferenceLogger.log_inference(
        prompt_slug="sdk_test",
        prompt=messages,
        language_model_id="gpt-4-1106-preview",
        response=response,
        external_reference_id="abc",
        cost=0.0123,
        custom_attributes={
                "name": "John Doe"
                # Your custom attributes
            }
    )
except Exception as e:
    if isinstance(e, CustomException):
        print(e.status_code)
        print(e.message)
    else:
        print(e)

💡

Tip: Include your logging code in a try/catch block to ensure that your application doesn't crash if the logging request fails.

Logging Attributes

prompt (optional): prompt sent to the LLM

prompt: [{"role": "user", "content": "What is machine learning?"}] # for openai models
prompt: "what is machine learning?" # for other models

response (optional): LLM Response. This can be either a string or the ChatCompletion response object from OpenAI
prompt_slug (optional): Identifier for the prompt used for inference. This is useful for segmenting inference calls by prompt

prompt_slug: "customer_query"

language_model_id (optional): Language model against which inference is made. Check out all supported models here

language_model_id: "gpt-4-1106-preview"

functions (optional): functions for older versions of openai,

functions: [
  {
    "name": "get_current_weather",
    "description": "Get the current weather in a given location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "The city and state, e.g. San Francisco, CA"
        },
        "unit": {
          "type": "string",
          "enum": [
            "celsius",
            "fahrenheit"
          ]
        }
      },
      "required": [
        "location"
      ]
    }
  }
]

environment (optional): Environment your app is running in (ex: production, staging, etc). This is useful for segmenting inference calls by environment

environment: "production"

function_call_response (optional): function call for older version of openai

function_call_response: {
  "name": "get_current_weather",
  "arguments": "{\n  \"location\": \"Boston, MA\"\n}"
}

tools (optional): tools for new versions of openai

tools: [
  {
    "type": "function",
    "function": {
      "name": "getCurrentWeather",
      "description": "Get the weather in location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city and state e.g. San Francisco, CA"
          },
          "unit": {
            "type": "string",
            "enum": [
              "c",
              "f"
            ]
          }
        },
        "required": [
          "location"
        ]
      }
    }
  }
]

tool_calls (optional): tool calls for new versions of openai

tool_calls: [
  {
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "get_current_weather",
      "arguments": "{\n\"location\": \"Boston, MA\"\n}"
    }
  }
]

If tool_calls field is not present, we extract it from the openai completion response and log it in our database

external_reference_id (optional): is useful if you want to associate your own internal identifier with the inference logged to Athina

external_reference_id: "5e838eaf-7dd0-4b6f-a32c-26110dd54e58"

customer_id (optional): is your customer ID. This is useful for segmenting inference calls by customer

customer_id: "stripe"

customer_user_id (optional): is the end user ID. This is useful for segmenting inference calls by the end user

customer_user_id: "user@gmail.com"

cost (optional): is the cost incurred for this LLM inference call. Tip: If you log an entire OpenAI completion response to us, we'll automatically calculate the cost.

cost: 0.0123

session_id (optional): is the session or conversation ID. This is used for grouping different inferences into a conversation or chain. Read more

session_id: "c45g-1234-s6g4-43d3"

user_query (optional): is the user's query. For conversational applications, this is the user's last message

user_query: "what is machine learning?"

context (optional): is the context used as information for the prompt. For RAG applications, this is the "retrieved" data. You may log context as a string or as an object (dictionary)

context: {"information": "Machine learning is a branch of artificial intelligence (AI) and computer science which focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving its accuracy"}
context: "Machine learning is a branch of artificial intelligence (AI) and computer science which focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving its accuracy"

custom_attributes (optional): custom_attributes is an object (dictionary) where you can log your own custom attributes as key-value pair with the inference.

custom_attributes: {
                "name": "John Doe"
                # Any other custom_attribute
            } # OPTIONAL;

💡

Tip: For evals, you must also log user_query and context

prompt_tokens (optional): tokens in the prompt,
completion_tokens (optional): tokens in the completion response,
total_tokens (optional): prompt_tokens + completion_tokens,
response_time (optional): is the response time in milliseconds. This is useful for segmenting inference calls by response time

prompt_tokens: 50
completion_tokens: 30
total_tokens: 80
response_time: 1208

💡

Tip: If you log the entire OpenAI ChatCompletion response object to us, we'll automatically extract the token counts and cost.

expected_response (optional): is the reference response to compare against for evaluation purposes. This is useful for segmenting inference calls by expected response

expected_response: "Machine Learning is a branch of artificial intelligence"

💡

Tip: For grounded evals like Answer Similarity, you must also log a reference response (string) to compare against.

Log via API Request OpenAI Completion (1.x)