ScaDS.AI LLM endpoint

In this notebook we will use the ScaDS.AI LLM infrastructure infrastructure at the Center for Information Services and High Performance Computing (ZIH) of TU Dresden. To use it, you must be connected via TU Dresden VPN. Also this method uses the OpenAI API and we just change the base_url.

Read more

• ScaDS.AI LLM examples

import os
import openai
openai.__version__

'1.106.1'

def prompt_scadsai_llm(message:str, model="openai/gpt-oss-120b"):
    """A prompt helper function that sends a message to ScaDS.AI LLM server at 
    ZIH TU Dresden and returns only the text response.
    """
    import os
    
    # convert message in the right format if necessary
    if isinstance(message, str):
        message = [{"role": "user", "content": message}]
    
    # setup connection to the LLM
    client = openai.OpenAI(base_url="https://llm.scads.ai/v1",
                           api_key=os.environ.get('SCADSAI_API_KEY')
    )
    response = client.chat.completions.create(
        model=model,
        messages=message
    )
    
    # extract answer
    return response.choices[0].message.content

prompt_scadsai_llm("Hi!")

'Hello! 👋 How can I assist you today?'

Exercise

List the models available in the endpoint and try them out by specifying them when calling prompt_scadsai_llm().

client = openai.OpenAI(base_url="https://llm.scads.ai/v1",
                       api_key=os.environ.get('SCADSAI_API_KEY'))

print("\n".join([model.id for model in client.models.list().data]))

alias-ha
meta-llama/Llama-3.3-70B-Instruct
meta-llama/Llama-3.1-8B-Instruct
alias-reasoning
alias-huge
alias-huge-no-thinking
Qwen/Qwen3-VL-8B-Instruct
alias-vision
openGPT-X/Teuken-7B-instruct-v0.6
Qwen/Qwen3-Coder-30B-A3B-Instruct
alias-code
google/gemma-4-31B-it
Qwen/Qwen3-Embedding-4B
BAAI/bge-reranker-v2-m3
stabilityai/stable-diffusion-3.5-large-turbo
alias-image-generation
black-forest-labs/FLUX.2-klein-9B
openai/whisper-large-v3-turbo
openai/whisper-large-v3
Kokoro-82M
tts-1-hd
openai/gpt-oss-120b
moonshotai/Kimi-K2.6
black-forest-labs/FLUX.2-dev
MiniMaxAI/MiniMax-M2.7