Learn to read or write the records in a JSON file using Spring Batch JsonItemReader and JsonFileItemWriter implementations.
1. JSON File
Spring batch supports JSON documents that contain a single array of objects. Any other JSON document format (e.g. root element is an object) is not supported.
[
{
"firstName": "Lokesh",
"lastName": "Gupta",
"age": 41,
"active": true
},
...
...
]2. Maven
There are no restrictions on which JSON library you use for parsing the JSON to POJO, or vice-versa. In most cases, you will be either using Jackson or Gson because Spring Batch provides built-in support for these libraries.
In your application, if you are using Jackson then include the following dependencies. You may require additional Jackson modules based on requirements:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.17.1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.17.1</version>
</dependency>For using Gson-based parsing, include the Gson dependency:
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.11.0</version>
</dependency>3. JSON Item Reader Example
In Spring Batch, the main class that reads Json objects from a Resource is JsonItemReader. Internally, the JsonItemReader delegates the responsibility to JsonObjectReader interface. Spring Batch provides two built-in implementations of this interface:
- JacksonJsonObjectReader (uses Jackson library)
- GsonJsonObjectReader (uses Gson library)
If you want to use any 3rd-party library then you need to explicitly implement the JsonObjectReader interface and create a bean for it.
@Bean
@StepScope
public JsonItemReader<Person> personJsonItemReader() {
return new JsonItemReaderBuilder<Person>()
.name("personJsonItemReader")
.jsonObjectReader(new JacksonJsonObjectReader<>(Person.class))
//.jsonObjectReader(new GsonJsonObjectReader<>(Person.class)) // If Gson is used
.resource(jsonFile)
.build();
}If you want to customize the ObjectMapper bean for custom parsing rules then we can customize the ObjectMapper and pass as a constructor argument into JacksonJsonObjectReader as follows:
@Bean
@StepScope
public JsonItemReader<Person> personJsonItemReader() {
ObjectMapper objectMapper = new ObjectMapper();
//customize objectMapper as necessary
return new JsonItemReaderBuilder<Person>()
...
.jsonObjectReader(new JacksonJsonObjectReader<>(objectMapper, Person.class))
...
.build();
}4. JSON File Writer Example
The file writer configuration is pretty similar to the reader configuration. There are only two major differences:
- The resource should be an instance of WritableResource which supports writing into it.
- Provide an implementation of JsonObjectMarshaller. Spring batch provides two built-in implementations:
- JacksonJsonObjectMarshaller
- GsonJsonObjectMarshaller
@Bean
public JsonFileItemWriter<Person> personJsonFileItemWriter() {
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
WritableResource outputJsonFile = (WritableResource) resolver.getResource("file:output-person.json");
return new JsonFileItemWriterBuilder<Person>()
.name("personJsonFileItemWriter")
.jsonObjectMarshaller(new JacksonJsonObjectMarshaller<>())
//.jsonObjectMarshaller(new GsonJsonObjectMarshaller<>()) // If Gson is used
.resource(outputJsonFile)
.build();
}5. Complete Configuration
The following JsonReaderJobConfig class is a reference for creating a Job and Step that reads a JSON file, parses all the fragments, and writes them back to a new JSON file. This example demonstrates the usages of reader and writer beans.
import com.fasterxml.jackson.databind.ObjectMapper;
import com.howtodoinjava.demo.model.Person;
import com.howtodoinjava.demo.processor.LoggingPersonProcessor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.json.JacksonJsonObjectMarshaller;
import org.springframework.batch.item.json.JacksonJsonObjectReader;
import org.springframework.batch.item.json.JsonFileItemWriter;
import org.springframework.batch.item.json.JsonItemReader;
import org.springframework.batch.item.json.builder.JsonFileItemWriterBuilder;
import org.springframework.batch.item.json.builder.JsonItemReaderBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.Resource;
import org.springframework.core.io.WritableResource;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.transaction.PlatformTransactionManager;
@Configuration
@SpringBootApplication
public class JsonReaderJobConfig {
@Value("classpath:person.json")
private Resource jsonFile;
@Bean
Job job(Step step1, JobRepository jobRepository) {
var builder = new JobBuilder("job", jobRepository);
return builder
.start(step1)
.build();
}
@Bean
public Step step1(JsonItemReader<Person> reader,
JsonFileItemWriter<Person> writer,
JobRepository jobRepository,
PlatformTransactionManager txManager) {
var builder = new StepBuilder("step1", jobRepository);
return builder
.<Person, Person>chunk(1, txManager)
.reader(reader)
.processor(new LoggingPersonProcessor())
.writer(writer)
.build();
}
@Bean
@StepScope
public JsonItemReader<Person> personJsonItemReader() {
ObjectMapper objectMapper = new ObjectMapper();
//customize objectMapper if needed
JacksonJsonObjectReader<Person> jsonObjectReader
= new JacksonJsonObjectReader<>(objectMapper, Person.class);
return new JsonItemReaderBuilder<Person>()
.name("personJsonItemReader")
.jsonObjectReader(jsonObjectReader)
.resource(jsonFile)
.build();
}
@Bean
public JsonFileItemWriter<Person> personJsonFileItemWriter() {
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
WritableResource outputJsonFile = (WritableResource) resolver.getResource("file:output-person.json");
return new JsonFileItemWriterBuilder<Person>()
.name("personJsonFileItemWriter")
.jsonObjectMarshaller(new JacksonJsonObjectMarshaller<>())
.resource(outputJsonFile)
.build();
}
public static void main(String[] args) {
SpringApplication.run(JsonReaderJobConfig.class);
}
}Run the application as Spring boot application. It will start the batch job in application starup and the program will read and process the JSON file with the batch job.
2024-06-27T10:26:32.523+05:30 INFO 18868 --- [main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=job]] launched with the following parameters: [{}]
2024-06-27T10:26:32.549+05:30 INFO 18868 --- [main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
2024-06-27T10:26:32.626+05:30 INFO 18868 --- [main] c.h.d.processor.LoggingPersonProcessor : Processing person information: Person(firstName=Lokesh, lastName=Gupta, age=41, active=true)
2024-06-27T10:26:32.643+05:30 INFO 18868 --- [main] c.h.d.processor.LoggingPersonProcessor : Processing person information: Person(firstName=Brian, lastName=Schultz, age=42, active=false)
2024-06-27T10:26:32.646+05:30 INFO 18868 --- [main] c.h.d.processor.LoggingPersonProcessor : Processing person information: Person(firstName=John, lastName=Cena, age=43, active=true)
2024-06-27T10:26:32.649+05:30 INFO 18868 --- [main] c.h.d.processor.LoggingPersonProcessor : Processing person information: Person(firstName=Albert, lastName=Pinto, age=44, active=false)
2024-06-27T10:26:32.655+05:30 INFO 18868 --- [main] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 104ms
2024-06-27T10:26:32.663+05:30 INFO 18868 --- [main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=job]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 125ms6. Summary
This Spring batch tutorial discussed how to read and write JSON files using Spring Batch jobs. It demonstrated how to configure Jackson or Gson as the backend Json processor, and ran a demo to show the configuration in action.
Happy Learning !!
Comments