Blablador endpoint

In this notebook we will use the blablardor infrastructure at the Research Center Jülich. Before you can access it, you need to create an API key as explained on this page. You will see that also this method uses the OpenAI API and we change the base_url.

import os
import openai
openai.__version__

'1.58.1'

def prompt_blablador(message:str, model="gpt-3.5-turbo"):
    """A prompt helper function that sends a message to Blablador (FZ Jülich)
    and returns only the text response.
    """
    import os
    
    # convert message in the right format if necessary
    if isinstance(message, str):
        message = [{"role": "user", "content": message}]
    
    # setup connection to the LLM
    client = openai.OpenAI()
    client.base_url = "https://helmholtz-blablador.fz-juelich.de:8000/v1"
    
    # todo: enter your API key here:
    # client.api_key = ""
    client.api_key = os.environ.get('BLABLADOR_API_KEY')
    response = client.chat.completions.create(
        model=model,
        messages=message
    )
    
    # extract answer
    return response.choices[0].message.content

prompt_blablador("Hi!")

'\nHello! How can I assist you today?'

Exercise

List the models available in the blablador endpoint and try them out by specifying them when calling prompt_blablador().

client = openai.OpenAI()
client.base_url = "https://helmholtz-blablador.fz-juelich.de:8000/v1"
client.api_key = os.environ.get('BLABLADOR_API_KEY')

print("\n".join([model.id for model in client.models.list().data]))

1 - Llama3 405 on WestAI with 4b quantization
1 - Ministral - 8b - out best fast model as november 2024
1 - Teuken-7B-instruct-research-v0.4 - The OpenGPT-X model
10 - DeepSeek-R1-Distill-Llama-70B - the best large  model as of January 2025
10 - DeepSeek-R1-Distill-Llama-8B - the best fast model as of January 2025
3 - Qwen2.5-Coder-7B-Instruct - the best model for coding as of september 2024
4 - Cosmosage V3.1 - Answers your Cosmology and Astronomy questions (new version September 2024)
5 - GritLM-7B - For Chat AND Text Embeddings
99 - TrustLLM preview 2
alias-code
alias-cosmosage
alias-embeddings
alias-fast
alias-large
alias-llama3-huge
alias-opengptx
alias-trustllm
gpt-3.5-turbo
text-davinci-003
text-embedding-ada-002