Long-context prompting

One can enrich a prompt to an LLM with a knowledge base, e.g. code snippets in the bio-image analysis context. This notebook demonstrates how to use such a knowledge-base for code generation.

from llm_utilities import prompt_scadsai_llm

First we define a knowledge base. This document is use-case-agnositic. We will use the same knowledge base independent from the task we’re trying to solve.

knowledge_base = """## Code snippets

You typically assemble code snippets when working with Python. The following snippets are given:

* When asked to segment nuclei, use the cellpose library like this:

```python
from cellpose import models

model = models.CellposeModel(gpu=False)
masks, flows, styles = model.eval(image, 
                                  batch_size=32, 
                                  flow_threshold=0.4, 
                                  cellprob_threshold=0.0,
                                  normalize={"tile_norm_blocksize": 0})
label_image = masks.astype(np.uint32)
```

* When asked to segment anything else but nuclei, use otsu-thresholding:

```python
from skimage.measure import label
from skimage.filters import threshold_otsu

# process the image
labeled = label(image > threshold_otsu(image))
```

*When asked to open an image, use this code:

```python
from skimage.io import imread, imsave

image = imread('image.tif')
```

* When asekd to visualize images, use this code at the very end of the code:

```python
import stackview
stackview.insight(image)
```

When writing code make sure the code is executable, 
e.g. make sure import statements are there and variables are named consistently.
"""

Next, we formulate the specific task at hand. In this example, we aim to develop a simple image processing workflow.

task = """Please write code that does this
* loads blobs.tif, 
* segment roundish objects in the image and 
* visualize the result """

Finally, the prompt is assembled from the generic knowledge base and the specific task.

prompt = f"""You are an excellent bio-image analyst and python programmer.

Your task is to write code in Jupyter Notebook cells. 
Always write one piece of code that goes into a Jupyter cell.

{knowledge_base}

## Your task:

{task}

## Final hints
Make sure there are comments between the code blocks.
"""

The prompt can then be sent to the LLM service provider to generate code for our task.

code = prompt_scadsai_llm(prompt)

# clean output
code = code.strip("\n").strip("```python").strip("```").strip("\n")

print(code)

# Import necessary libraries
from skimage.io import imread, imsave
import numpy as np
from skimage.measure import label
from skimage.filters import threshold_otsu

# Load the image
image = imread('blobs.tif')
```

```python
# Segment roundish objects in the image using otsu-thresholding
labeled = label(image > threshold_otsu(image))
```

```python
# Visualize the result
import stackview
stackview.insight(labeled)

Exercise

Extend the knowledge base above with more code snippets, e.g. feature extraction and plotting. Prompt for an advanced image processing workflow going from a raw image to a plot of extracted features.