ScaDS.AI LLM endpoint

Contents

ScaDS.AI LLM endpoint#

In this notebook we will access large language models hosted at the ScaDS.AI LLM infrastructure infrastructure of the Center for Information Services and High Performance Computing (ZIH) of TU Dresden. To use it, you must be connected via TU Dresden VPN. This method uses the openai Python library, and the OpenAI API. We just change the base_url.

Read more#

ScaDS.AI LLM examples

import os
import openai
openai.__version__

'1.91.0'

def prompt_scadsai_llm(message:str, model="meta-llama/Llama-3.3-70B-Instruct"):
    """A prompt helper function that sends a message to ScaDS.AI LLM server at 
    ZIH TU Dresden and returns only the text response.
    """
    import os
    import openai
    
    # convert message in the right format if necessary
    if isinstance(message, str):
        message = [{"role": "user", "content": message}]
    
    # setup connection to the LLM
    client = openai.OpenAI(base_url="https://llm.scads.ai/v1",
                           api_key=os.environ.get('SCADSAI_API_KEY')
    )
    response = client.chat.completions.create(
        model=model,
        messages=message
    )
    
    # extract answer
    return response.choices[0].message.content

print(prompt_scadsai_llm("Hi!"))

How can I assist you today?

We can also use reasoning models such as the DeepSeek-R1 model hosted on the server.

print(prompt_scadsai_llm("What is the meaning of life?", model="deepseek-ai/DeepSeek-R1"))

The question "What is the meaning of life?" is one of humanity’s oldest and most profound inquiries, with answers varying widely across philosophical, religious, scientific, and personal perspectives. Here’s a concise overview:

1. **Philosophical Perspectives**:  
   - **Existentialism**: Thinkers like Jean-Paul Sartre and Albert Camus argue that life has no inherent meaning; individuals must create their own purpose through choices, passions, and actions.  
   - **Absurdism**: Camus also suggested that while life may lack intrinsic meaning, embracing the "absurd" (the conflict between our search for meaning and an indifferent universe) can lead to a fulfilling, rebellious existence.  
   - **Nihilism**: Contends that life is without objective meaning, purpose, or intrinsic value, though responses to this view range from despair to liberation.  

2. **Religious/Spiritual Views**:  
   - Many traditions propose transcendent purposes. For example, in Christianity, meaning is found in a relationship with God; Buddhism emphasizes ending suffering (dukkha) through enlightenment; Hinduism focuses on Dharma (duty) and Moksha (liberation).  

3. **Scientific Lens**:  
   - Biology suggests life’s "purpose" is survival and reproduction, driven by evolution. However, this mechanistic view doesn’t address subjective human experiences of meaning.  
   - Cosmology might frame life as a rare, fleeting phenomenon in an indifferent universe, inviting awe rather than teleology.  

4. **Personal and Subjective Meaning**:  
   - Many find purpose in relationships, creativity, knowledge, service, or personal growth. Viktor Frankl, a Holocaust survivor and psychiatrist, argued that meaning arises even in suffering when we seek responsibility and purpose.  

**Conclusion**: There is no universal answer. The meaning of life is often seen as a deeply personal journey, shaped by individual values, experiences, and cultural context. It may lie in asking the question itself—continually exploring what inspires connection, wonder, or fulfillment in your own life. As Rainer Maria Rilke wrote: *"Live the questions now."*

Exercise#

List the models available in the endpoint and try them out by specifying them when calling prompt_scadsai_llm("...", model=...).

client = openai.OpenAI(base_url="https://llm.scads.ai/v1",
                       api_key=os.environ.get('SCADSAI_API_KEY'))

print("\n".join([model.id for model in client.models.list().data]))

meta-llama/Llama-3.3-70B-Instruct
mistral-7b-q4
black-forest-labs/FLUX.1-dev
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct
deepseek-ai/DeepSeek-R1
Qwen/Qwen2-VL-7B-Instruct
tts-1-hd
meta-llama/Llama-4-Scout-17B-16E-Instruct
stabilityai/stable-diffusion-3.5-large
Kokoro-82M
en-de-translator
intfloat/multilingual-e5-large-instruct
openGPT-X/Teuken-7B-instruct-research-v0.4