Java and AI with LangChain4j: Jakarta EE and SmallRye LLM

Java and AI with
LangChain4j
Code the future with AI
Ed Burns @edburns
Principal Architect
PENDING download link

Ed Burns professional biography
 Client
 NCSA Mosaic (1994)
 SGI Cosmo Web Authoring
 Sun Netscape 6 OJI
 Server
 J2EE JSF (2002)
 Oracle Java EE
 Servlet, JSF, Bean Validation, etc.
 Cloud
 Java Champion
 Microsoft Azure Cloud (2019)
 Jakarta EE 11 committer and release coordinator
 Books

Change in content from posted agenda
 Still 100% Java and AI with LangChain4j.
 Removed content:
 After a brief look at the landscape.
 How you use it to perform the most popular AI usage pattern:
Retrieval Augmented Generation (RAG).
 Added content:
 langchain4j-microprofile-jakarta.
 If you want the RAG content, I recommend Easy RAG with
LangChain4J and Docker by Julien Dubois.

Breaking Jakarta EE news
 Jakarta EE 11 Web Profile started release review ballot Monday!
 https://github.com/jakartaee/specifications/pull/802
 https://deploy-preview-802--jakartaee-
specifications.netlify.app/specifications/webprofile/11/
 Many EE 12 component specifications in Plan Review stage.
 MicroProfile is discussing whether and how to host MicroProfile in
Jakarta EE.
 The need for a unified Config solution for EE 12.
 Coincidentally timed soon after Evolving our middleware strategy blog post
from Red Hat.

Everyone I know admits: AI is moving faster than
anything that has come before
https://aka.ms/DeveloperCareerMasterplan/slides https://aka.ms/RockStarProgrammerSecrets

Scott Hanselman: Let’s all just
chill
From a trust perspective
 Very important to keep the
“intelligence” in AI in
perspective.
 LLMs cannot answer simple
questions such as “Which
episode of Gilligan’s Island was
about mind reading?”
 The question that no LLM can answer and
why it is important
https://www.mindprison.cc/p/the-question-that-no-llm-can-answer

Java meets AI
Content from Emmanuel Hugonnet http://www.ehsavoie.com/

Jakarta EE meets AI
smallrye-llm
langchain4j-
microprofile
-jakarta
Microprofile config
Microprofile Fault Tolerance Microprofile
OpenTelemetry
CDI

Components of LangChain4J

Quick look at LangChain4j examples
 https://github.com/langchain4j/langchain4j-examples

Components of SmallRye LLM/LangChain4J Jakarta
1. AI Service Interface Processing:
o Detects interfaces annotated with @RegisterAIService
o Generates proxy implementations using LangChain4J
o Makes them available as CDI beans
2. Plugin Bean Creation:
o Reads configuration from properties with smallrye.llm.plugin.* prefix
o Creates beans for LangChain4J components (models, embedding stores)
o Makes these components available for injection
3. MicroProfile Config Integration:
o Translates properties to LangChain4J configuration
o Handles environment variable substitution (e.g., ${azure.openai.api.key})
4. Fault Tolerance:
o Provides integration with MicroProfile Fault Tolerance
o Adds resilience to AI service calls
5. Telemetry:
o Provides observability for AI service calls
o Integrates with OpenTelemetry

CDI Injection
Injection.java
@Inject
@Named(value = "ollama")
ChatLanguageModel chatModel;
@Inject
@Named(value = "embedding-store-retriever")
ContentRetriever retriever;

AI Service
SimpleAIService.java
@RegisterAIService(chatLanguageModelName = "ollama", tools =
{org.wildfly.ai.websocket.Weather.class,org.wildfly.ai.websocket.Cal
culator.class}, scope = SessionScoped.class)
public interface SimpleAIService {
@SystemMessage("""
You are an AI named Bob answering general question.
Your response must be polite, use the same language
as the question, and be relevant to the question.""")
String chat(@UserMessage String question);
}

But what is MCP?
https://modelcontextprotocol.io/

But what is MCP?
 https://github.com/modelcontextprotocol/servers

Demo

MCP
Server
MCP tools workflow
MCP
Server
Web
Application
LLM
MCP
Server
1
2 3
4
MCP
Server
5
6
MCP
Server
Database
Filesystem
Message queue
JavaFX

AI Service with MCP Tools
SimpleAIService.java
@RegisterAIService(streamingChatLanguageModelName= "streaming-
ollama", toolProviderName = "mcp")
public interface ChatAiService {
@SystemMessage("""
You are an AI named Bob answering general question.
Your response must be polite, use the same language
as the question, and be relevant to the question.""")
TokenStream streamingChat(@UserMessage String question);
}

Conclusion

What we saw
External configuration instead of code
CDI Injection
AI Service support
Cross-vendors: Micronaute, OpenLiberty, Quarkus, WildFly
Enterprise ready : OpenTelemetry / Fault Tolerance / MCP
Makes LangChain4J even easier to use !

Resources
• LangChain4J:
o https://docs.langchain4j.dev/
• Model Context Protocol:
o https://spec.modelcontextprotocol.io/
• smallrye-llm: a cross vendor LangChain4J -CDI integration library
o https://github.com/smallrye/smallrye-llm
• WildFly AI Feature Pack:
o https://github.com/wildfly-extras/wildfly-ai-feature-pack
• WildFly MCP: to interact with WildFly servers through MCP
o https://github.com/wildfly-extras/wildfly-mcp
• Admin your WildFly with LLM and MCP
o https://www.youtube.com/watch?v=wg1hAdOoe2w
• Making WildFly Glow with Intelligence:
o https://www.wildfly.org/news/2025/02/10/Glowing-with-AI/
• Playing with Generative AI with WildFly:
o https://www.wildfly.org/news/2024/11/04/WildFly-playing-with-generative-ai/

Thank you
Ed Burns
@edburns on most of the things
• Please fill out this survey
• 5 winners will be chosen to win a $100 digital gift card
• https://aka.ms/edburns/survey2025

1.What is your key decision making process when it comes to app
modernization?
2.What’s your key challenge when modernizing your Java workload?
3.Are you a Java only shop or multiple language shop?
4.What is the tech stack/framework for your Java apps?
5.When migrating, what is your primary destination on cloud? What
platform are you choosing? VM? PaaS? Containers? Why?
6.How did you or your team make the above decisions?

Java and AI with LangChain4j: Jakarta EE and SmallRye LLM

More Related Content

Similar to Java and AI with LangChain4j: Jakarta EE and SmallRye LLM (20)

More from Edward Burns (20)

Recently uploaded (20)

Java and AI with LangChain4j: Jakarta EE and SmallRye LLM