Spring AI Structured Output Converters (List, Map and Bean)

The Spring AI structured output converters help in converting the LLM output into a structured format such as list, map, or complex structures defined in a Java bean. These classes help in communicating the expected response formats to LLMs and later parse the response into Java classes using standard mashalling and unmarshalling features.

Spring AI provides 3 inbuilt classes MapOutputConverter, ListOutputConverter, and BeanOutputConverter that can help in communicating most of the expected response structures typecally seen in an API.

At a very high level, these converters perform two main tasks:

• It instructs the prompt for the desired response format (XML or JSON) and structure. This ensures that the response is generated in a rigorous format.
• Upon receiving the response, it parses the response into Java classes such as a POJO, a List, or a Map.

1. Setup

Before running any code on your machine, make sure you have OpenAPI project key set as an environment variable and your application reads it from there.

export OPENAI_API_KEY=[api_key_copied_from_openai_site]

Later, we can refer to this API key in the project’s properties file:

spring.ai.openai.api-key=${OPENAI_API_KEY}

Next, we need to add the required Maven dependencies that are specific to the LLM we are interacting with. In this demo, we are using OpenAI’s ChatGPT so let’s add its dependency in the application.

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>

For complete setup steps, follow the Spring AI tutorial.

2. Spring AI MapOutputConverter Example

The MapOutputConverter class instructs the prompt to request LLM output in RFC8259 compliant JSON format and the structure should be of type java.util.HashMap. Later, when we get the LLM response, this converter parses the response from JSON and populates into the Map instance.

The way MapOutputConverter works is by appending a fixed text to the user message requesting the specific format. We can see this fixed text in the MapOutputConverter class source code or calling its toFormat() method.

public class MapOutputConverter extends AbstractMessageOutputConverter<Map<String, Object>> {

	//...

	@Override
	public String getFormat() {
		String raw = """
				Your response should be in JSON format.
				The data structure for the JSON should match this Java class: %s
				Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
				 """;
		return String.format(raw, HashMap.class.getName());
	}
}

In the following example, we are supplying a list of countries and asking the LLM to return the name of the country and its capital in the Map format.

@GetMapping("/country-capital-service/map")
public Map<String, Object> getCapitalNamesInMap(@RequestParam String countryNamesCsv) {

  if (countryNamesCsv == null || countryNamesCsv.isEmpty()) {
    throw new IllegalArgumentException("Country names CSV cannot be null or empty");
  }

  MapOutputConverter converter = new MapOutputConverter();
  String format = converter.getFormat();

  PromptTemplate pt = new PromptTemplate("For these list of countries {countryNamesCsv}, return the list of capitals. {format}");
  Prompt renderedPrompt = pt.create(Map.of("countryNamesCsv", countryNamesCsv, "format", format));

  ChatResponse response = chatClient.call(renderedPrompt);
  Generation generation = response.getResult();  // call getResults() if multiple generations
  return converter.parse(generation.getOutput().getContent());
}

Upon receiving the request, API prepares the final prompt by replacing the ‘countryNamesCsv’ with the supplied list of countries, and format with the fixed text as specified in MapOutputConverter’s getFormat() method.

The response we receive from the LLM and finally returned from the API is a JSON output in the Map format.

3. Spring AI ListOutputConverter Example

The ListOutputConverter class works pretty much the same as the list output converter, except it instructs the prompt to request LLM output in a list of comma-separated values. Later, Spring AI uses Jackson to parse the CSV values into a List.

public class ListOutputConverter 
	extends AbstractConversionServiceOutputConverter<List<String>> {

	//...

	@Override
	public String getFormat() {
		return """
				Your response should be a list of comma separated values
				eg: `foo, bar, baz`
				""";
	}
}

In the following example, we are supplying a list of countries and asking for their capitals in the list.

@GetMapping("/country-capital-service/list")
	public List<String> getCapitalNamesInList(@RequestParam String countryNamesCsv) {

	if (countryNamesCsv == null || countryNamesCsv.isEmpty()) {
	  throw new IllegalArgumentException("Country names CSV cannot be null or empty");
	}

	ListOutputConverter converter = new ListOutputConverter(new DefaultConversionService());
	String format = converter.getFormat();

	PromptTemplate pt = new PromptTemplate("For these list of countries {countryNamesCsv}, return the list of capitals. {format}");
	Prompt renderedPrompt = pt.create(Map.of("countryNamesCsv", countryNamesCsv, "format", format));

	ChatResponse response = chatClient.call(renderedPrompt);
	Generation generation = response.getResult();  // call getResults() if multiple generations
	return converter.parse(generation.getOutput().getContent());
}

The LLM returns the response in a plain string of CSV values:

New Delhi, Washington D.C., Ottawa, Jerusalem

And this CSV is parsed into java.util.List using its converter.parse() method. Let’s test the API:

4. Spring AI BeanOutputConverter Example

The BeanOutputConverter helps in requesting the LLM response in the JSON format and structure matching with a Java POJO. The fields in the LLM response are always compatible with the fields in the specified Java bean.

In the following method, the JSON schema is generated from the Java bean class provided as a reference.

public class BeanOutputConverter<T> implements StructuredOutputConverter<T> {

	//...

	@Override
	public String getFormat() {
		String template = """
				Your response should be in JSON format.
				Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
				Do not include markdown code blocks in your response.
				Remove the ```json markdown from the output.
				Here is the JSON Schema instance your output must adhere to:
				```%s```
				""";
		return String.format(template, this.jsonSchema);
	}
}

Let us take another example. In this example, we are providing the name of a country, and asking the LLM to return its top 10 cities. The response should follow the Java bean structure as specified:

public record Pair(String countryName, List<String> cities) {
}

Let us see how to use BeanOutputConverter to instruct prompt and parse teh response received.

@GetMapping("/country-capital-service/bean")
public Pair getCapitalNamesInPojo(@RequestParam String countryName) {

  BeanOutputConverter<Pair> converter = new BeanOutputConverter(Pair.class);
  String format = converter.getFormat();

  PromptTemplate pt = new PromptTemplate("For these list of countries {countryName}, return the list of its 10 popular cities. {format}");
  Prompt renderedPrompt = pt.create(Map.of("countryName", countryName, "format", format));

  ChatResponse response = chatClient.call(renderedPrompt);
  Generation generation = response.getResult();  // call getResults() if multiple generations
  return converter.parse(generation.getOutput().getContent());
}

We can verify the generated response by calling the API:

5. Conclusion

As discussed in this Spring AI tutorial, LLM will not always return the response in unstructured texts. Several times, we need them to return the response in fixed formats, and that’s where these structured output converters help. I will suggest you play with these APIs and try requesting the outputs in different formats to understand and appreciate their usages better.

Happy Learning !!

Source Code on Github