Getting Started with Spring AI (with Examples)

Large Language Models (LLMs) like GPT-4, Claude, Gemini, and Mistral that power Generative AI applications like ChatGPT have evolved and increased in number, and have accelerated a new class of intelligent applications and making existing applications more intelligent. People who make decisions for businesses are trying hard to understand how to use these intelligent programs in their products and services. Spring AI helps Java developers make these intelligent programs by providing well-known autoconfiguration and client-API-based programming approaches.

Spring AI supports almost all major model providers such as OpenAI, Microsoft, Amazon, Google, and Huggingface. It supports plain chat-based interactions and conversion from Text to Image, etc., in synchronous and asynchronous (stream API) methods.

1. Setting Up the API Key

Before moving forward into the guide, you must have API access to the LLM that you want to access with Spring AI. In this guide, we are accessing OpenAI’s gpt-3.5-turbo model so we created an account in the OpenAI website and obtained the API key after paying a small amount ($5 in our case).

The Project API key (previously User API key) is a unique encoded string that identifies and authenticates a user or application. You can obtain the key for your account at page: platform.openai.com/api-keys.

Spring AI module defines separate key names for separate LLMs. So there is no fixed key name. Generally, the key name is in the format of ‘SPRING_AI_<LLM>_API_KEY‘. So for OpenAI, it is ‘spring.ai.openai.api-key‘ and for Mistral, it is ‘spring.ai.mistralai.api-key‘.

For safety reasons, we should add the API key to the Machine’s environment variables and then refer to it in our application’s configuration. Although, we can define the API key directly into the application.properties file as well. In the following example, we are exporting the API key into the environment variable ‘OPENAI_API_KEY‘.

export OPENAI_API_KEY=[api_key_copied_from_openai_site]

On a Windows machine, we can add the environment variable in System properties:

Later, we can refer to this API key in the project’s properties file:

spring.ai.openai.api-key=${OPENAI_API_KEY}

2. Project Dependencies

In a Spring boot application, we can enable support for Spring AI using the dependency: spring-ai-openai-spring-boot-starter.

<properties>
  <java.version>21</java.version>
  <spring-ai.version>1.0.0-SNAPSHOT</spring-ai.version>
</properties>

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>${spring-ai.version}</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

As of today, Spring AI is still in development mode so we can pull it from the milestone repository. You can check the official GitHub page for the latest version.

<repositories>
	<repository>
	  <id>spring-milestones</id>
	  <name>Spring Milestones</name>
	  <url>https://repo.spring.io/milestone</url>
	  <snapshots>
	    <enabled>false</enabled>
	  </snapshots>
	</repository>
	<repository>
	  <id>spring-snapshots</id>
	  <name>Spring Snapshots</name>
	  <url>https://repo.spring.io/snapshot</url>
	  <releases>
	    <enabled>false</enabled>
	  </releases>
	</repository>
</repositories>

Additionally, we will need to include spring-web, spring-webflux, and optional httpclient dependencies for creating the web components and handling HTTP connections to the backend LLMs.

<dependencies>

	<!-- other dependnecies -->
	
	<dependency>
	  <groupId>org.springframework.boot</groupId>
	  <artifactId>spring-boot-starter-web</artifactId>
	</dependency>
	<dependency>
	  <groupId>org.springframework.boot</groupId>
	  <artifactId>spring-boot-starter-webflux</artifactId>
	</dependency>
<dependencies>

3. What does Spring AI Module provide?

Spring AI is a new addition to the Spring Framework ecosystem for developing generative AI-focused applications that focus on generating new content in response to input prompts. In the backend, it makes an API call to OpenAI and presents the response back.

At its heart, Spring AI offers a text-based generative AI system. It means that users input the text, and the application responds with relevant output in different formats (String, Map, List, XML, JSON, Image, Video etc.). For response generation, Spring AI provides integration with most of the prominent generative AI models, including OpenAI, Azure Open AI, Bedrock (Amazon), Ollama, and Vertex AI (Google).

3.1. Client API

The following is a list of important classes/interfaces used in interacting with OpenAI LLMs:

import org.springframework.ai.chat.ChatClient;
import org.springframework.ai.image.ImageClient;
import org.springframework.ai.openai.OpenAiChatClient;
import org.springframework.ai.openai.OpenAiAudioSpeechClient;
import org.springframework.ai.openai.OpenAiImageClient;
import org.springframework.ai.openai.api.OpenAiApi;
import org.springframework.ai.openai.api.OpenAiAudioApi;
import org.springframework.ai.openai.api.OpenAiImageApi;
import org.springframework.ai.openai.audio.speech.SpeechClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AppConfiguration {

  @Bean
  SpeechClient speechClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
    return new OpenAiAudioSpeechClient(new OpenAiAudioApi(apiKey));
  }

  @Bean
  ImageClient imageClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
    return new OpenAiImageClient(new OpenAiImageApi(apiKey));
  }

  @Bean
  ChatClient chatClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
    return new OpenAiChatClient(new OpenAiApi(apiKey));
  }
}

3.2. Output Converter

We will not be interacting with LLMs in only plain text format. Sometimes, our applications will need the outputs in a very specific structure (such as fixed JSON schema). In such cases, we can use the specific converters based on the usecase.

The converters perform two main tasks:

• It instructs the prompt for the desired response format and structure. This ensures that the response is generated in a rigorous format.
• Upon receiving the response, it parses the response in the required format such as a Java bean, a list, or a map of values

3.3. Retrieval Augmented Generation (RAG)

In cases where we want the LLM to respond based on information available in a specified document, Spring AI provides the document readers. This is especially useful in Retrieval Augmented Generation (RAG).

Spring AI’s document reader API supports various formats, including simple text, JSON, Apache Tika, and PDFs. These are the main classes/interfaces part of Spring AI’s document processing capability:

• Document: text-based representation of the data and metadata.
• DocumentReader: is responsible for loading a List from a data source such as PDF, Markdown, and JSON.
• DocumentWriter: persist the documents into a vector database.
• DocumentTransformer: is responsible for processing the data in various ways (splitting, concatenation etc).
• Embedding: a representation of the data as a List that is used by the vector database to compute the ‘similarity’ of a user’s query to relevant documents.
• ContentFormatter: converts the Document text and metadata into an AI-prompt-friendly text representation.
• VectorStore: stored the embeddings to perform similarity searches.

3.4. Vector Store Integration

A vector store (or vector database) performs ‘similarity searches’, just like exact matches in traditional relational databases. A vector store enables us to organize the document data into smaller chunks. We store our documents in a vector store in RAG-related usecases using the ETL pipeline.

When given a vector as a query, a vector database returns vectors that are “similar” to the query vector. This returned vector is a sub-document that we can send to LLM instead of sending the whole document which is a costly operation.

Spring AI allows integration with several leading vector stores, including Chroma, Pinecone, Redis, Weavite, Milvus, Azure, and PostgreSQL (PG Vector).

4. Spring AI Chat Completion Example

Let us begin with the simplest example. We ask the LLM (OpenAI GPT) to tell us a Joke. The prompt is a simple string, and we do not specify anything for the response format.

The tellSimpleJoke() method utilizes the ChatClient API (discussed in section 3.1.) which sends a simple prompt “Tell me a joke” to LLM. Whatever we get in the response, we simply put it in a Map and send it as a response to the API consumer.

@RestController
public class OpenAiChatController {

  private final ChatClient chatClient;

  @Autowired
  public OpenAiChatController(ChatClient chatClient) {
    this.chatClient = chatClient;
  }

  @GetMapping("/joke-service/simple")
  public Map<String, String> tellSimpleJoke() {

    return Map.of("generation", chatClient.call("Tell me a joke"));
  }
}

Let’s test this API:

For streaming the response from Model, we can call the chatClient.stream() model if the client implementation class implements the StreamingChatClient interface. For example, OpenAiChatClient class implements both interfaces i.e. ChatClient and StreamingChatClient.

5. Spring AI PromptTemplate Example

A PromptTemplate is quite similar to template strings in Java 21. In a prompt template, we have replaceable tokens in the string, which are replaced in runtime based on the parameters. Consider a similar prompt template as defined in application.properties. This ‘subject’ and ‘language’ are template parameters that will be provided in runtime.

app.joke.simple.promptTemplate=Tell me a joke about {subject} in {language}.

Spring AI’s PromptTemplate class helps in resolving such prompts and passing it to the ChatClient.call() method.

@RestController
public class OpenAiChatController {

	private final ChatClient chatClient;
	private String promptTemplate;

	@Autowired
	public OpenAiChatController(ChatClient chatClient,
	          @Value("${app.joke.simple.promptTemplate}") String promptTemplate) {

		this.chatClient = chatClient;
		this.promptTemplate = promptTemplate;
	}

	@GetMapping("/joke-service/simple-with-prompt")
	public String tellSimpleJokeWithPrompt(@RequestParam("subject") String subject,
	                @RequestParam("language") String language) {

	  PromptTemplate pt = new PromptTemplate(promptTemplate);
	  Prompt renderedPrompt = pt.create(Map.of("subject", subject, "language", language));

	  ChatResponse response = chatClient.call(renderedPrompt);
	  return response.getResult().getOutput().getContent();
	}
}

In the above code, we have asked for the joke’s subject and language from the API user as request parameters. Let’s test it out:

6. Spring AI Structured Output Example

Simple text-based outputs are sufficient for Chatbot-type applications. But in data-intensive intelligent applications, we must request and respond in complex formats (XML, JSON etc), just like we do in typical REST APIs using Java Beans.

In the following example, we have modified the previous API to request and respond in the format in JSON with fields in the JokeResponse record type:

public record JokeResponse(String subject, String language, String joke) {
}

The API utilizes the BeanOutputConverter class that derives the JSON format from the specified Java bean. Later it uses its parse() method to convert the JSON response into JokeResponse object.

@RestController
public class OpenAiChatController {

	private final ChatClient chatClient;
	private String jsonPromptTemplate;

	@Autowired
	public OpenAiChatController(ChatClient chatClient,
	          @Value("${app.joke.formatted.promptTemplate}") String promptTemplate) {

		this.chatClient = chatClient;
		this.jsonPromptTemplate = jsonPromptTemplate;
	}

	@GetMapping("/joke-service/json-with-prompt")
  public JokeResponse tellSpecificJokeInJsonFormat(@RequestParam("subject") String subject,
                 @RequestParam("language") String language) {

    BeanOutputConverter<JokeResponse> parser = new BeanOutputConverter<>(JokeResponse.class);
    /**
     * Your response should be in JSON format.
     * Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
     * Do not include markdown code blocks in your response.
     * Remove the ```json markdown from the output.
     * Here is the JSON Schema instance your output must adhere to:
     * ```{
     *   "$schema" : "https://json-schema.org/draft/2020-12/schema",
     *   "type" : "object",
     *   "properties" : {
     *     "joke" : {
     *       "type" : "string"
     *     },
     *     "language" : {
     *       "type" : "string"
     *     },
     *     "subject" : {
     *       "type" : "string"
     *     }
     *   }
     * }```
     * */
    String format = parser.getFormat();

    PromptTemplate pt = new PromptTemplate(jsonPromptTemplate);
    Prompt renderedPrompt = pt.create(Map.of("subject", subject, "language", language, "format", format));

    ChatResponse response = chatClient.call(renderedPrompt);
    return parser.parse(response.getResult().getOutput().getContent());
  }
}

The request format is appended at the end of prompt to the LLM.

app.joke.formatted.promptTemplate=Tell me a joke about {subject} in {language}. {format}

Let’s test this API:

7. Spring AI Image Generation Example

The image generation with OpenAI uses the DALL-E models. We provide the message to ImageClient and it responds with the URL of the generated image.

@RestController
public class OpenAiImageController {

  private final ImageClient imageClient;

  public OpenAiImageController(ImageClient imageClient) {
    this.imageClient = imageClient;
  }

  @GetMapping("/image-gen")
  public String imageGen(@RequestParam String message) {
    ImageOptions options = ImageOptionsBuilder.builder()
        .withModel("dall-e-3")
        .withHeight(1024)
        .withWidth(1024)
        .build();

    ImagePrompt imagePrompt = new ImagePrompt(message, options);
    ImageResponse response = imageClient.call(imagePrompt);
    String imageUrl = response.getResult().getOutput().getUrl();

    return "redirect:" + imageUrl;
  }
}

Let us test the API:

We can download the image from the URL in our application. For testing, copy the URL in a browser which should display the generated image:

We should take care when setting the parameters in ImageOptions as invalid parameters will raise a runtime error. For example, if we request an image of size 900×600 then it will throw NonTransientAiException:

{
    "timestamp": "2024-05-09T09:42:37.369+00:00",
    "status": 500,
    "error": "Internal Server Error",
    "trace": "org.springframework.ai.retry.NonTransientAiException: 400 - {\n  \"error\": {\n    \"code\": null,\n    \"message\": \"'900x600' is not one of ['256x256', '512x512', '1024x1024', '1024x1792', '1792x1024'] - 'size'\",\n    \"param\": null,\n    \"type\": \"invalid_request_error\"\n  }
    ...
    ...,
    "message": "400 - {\n  \"error\": {\n    \"code\": null,\n    \"message\": \"'900x600' is not one of ['256x256', '512x512', '1024x1024', '1024x1792', '1792x1024'] - 'size'\",\n    \"param\": null,\n    \"type\": \"invalid_request_error\"\n  }\n}\n",
    "path": "/image-gen"
}

8. Conclusion

In this Spring AI tutorial, we learned the basics of the Spring AI module and its core terminology. We learned to interact with OpenAI’s chat and image generation services using simple text prompts as well as strictly formatted JSON prompts. Practice with these APIs to understand and take benefit of them in your next intelligent AI-enabled application.

Happy Learning !!

Source Code on Github