The Guide to Call ChatGPT 3.5 Event Stream API in Spring Boot

Posted on 2023-03-16 Edited on 2024-12-19 In Java Web , Spring Integration

Create a spring boot project

You can go to Spring Initializr to create a spring boot project

Setting your basic configurations. The following is my settings

Project: Maven
Language: Java
Spring Boot: 2.7.9
Project Metadata:
- Group: com.taogen
- Aritifact: chatgpt-demo
- Packaging: Jar
- Java: 8

You can set your own configurations according to your local environment.

After setting the project configurations, we need to add the following dependencies:

Spring Web
Lombok

Then we click the GENERATE button to download our project file.

After downloading the generated project file, we need to decompress the file and open the directory in our IDE. I prefer to use IntelliJ IDEA as my IDE.

Call the ChatGPT API in spring boot

Configuring your Open AI key in application.properties

openai.apiKey=${YOUR_OPEN_AI_KEY}

Create the RestTemplateConfig.java to build the RestTemplate bean for API calls

@Configuration
public class RestTemplateConfig {
    @Bean
    public RestTemplate restTemplate(RestTemplateBuilder builder) {
        return builder.build();
    }
}

Create a POJO ChatGpt3dot5Request.java for building ChatGPT API requests

@Data
public class ChatGpt3dot5Request {
    public static final String ENDPOINT = "https://api.openai.com/v1/chat/completions";

    public static final String STREAM_MESSAGE_PREFIX = "data: ";

    private String model = "gpt-3.5-turbo";

    private Boolean stream;
    List<Message> messages = new ArrayList<>();

    @Data
    @AllArgsConstructor
    static class Message {
        private String role;
        private String content;
    }
}

Create the controller ChatBotController.java

@RequestMapping("/chatbot")
@RestController
public class ChatBotController {

    @Autowired
    public RestTemplate restTemplate;

    @Value("${openai.apiKey}")
    private String OPEN_API_KEY;

    @PostMapping("/conversation")
    public void conversation(@RequestBody String prompt, HttpServletResponse httpServletResponse) throws IOException {
        httpServletResponse.setContentType("text/plain; charset=UTF-8");
        UriComponentsBuilder builder = UriComponentsBuilder
                .fromUriString(ChatGpt3dot5Request.ENDPOINT)
                .queryParams(null);
        HttpHeaders httpHeaders = new HttpHeaders();
        httpHeaders.setContentType(MediaType.APPLICATION_JSON);
        httpHeaders.setBearerAuth(OPEN_API_KEY);
        String json = getRequestBody(prompt);
        HttpEntity<Object> requestEntity =
                new HttpEntity<>(json, httpHeaders);
        ResponseEntity<Resource> responseEntity = restTemplate.exchange(builder.build().toUriString(), HttpMethod.POST, requestEntity, org.springframework.core.io.Resource.class);
        PrintWriter writer = httpServletResponse.getWriter();
        BufferedReader bufferedReader;
        try {
            bufferedReader = new BufferedReader(new InputStreamReader(responseEntity.getBody().getInputStream()));
            String line;
            while ((line = bufferedReader.readLine()) != null && !(ChatGpt3dot5Request.STREAM_MESSAGE_PREFIX + "[DONE]").equals(line)) {
                String message = getMessageFromLine(line, ChatGpt3dot5Request.STREAM_MESSAGE_PREFIX);
                writer.write(message);
                writer.flush();
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    private static String getMessageFromLine(String line, String prefix) throws JsonProcessingException {
        if (!StringUtils.hasLength(line.trim())) {
            return "";
        }
        String messageJsonStr = line.substring(line.indexOf(prefix) + prefix.length());
        ObjectMapper objectMapper = new ObjectMapper();
        ObjectNode objectNode = (ObjectNode) objectMapper.readTree(messageJsonStr);
        JsonNode messageNode = objectNode.get("choices").get(0).get("delta").get("content");
        if (messageNode != null) {
            return messageNode.asText();
        } else {
            return "";
        }
    }

    private String getRequestBody(String prompt) throws JsonProcessingException {
        ChatGpt3dot5Request chatGpt3dot5Request = new ChatGpt3dot5Request();
        chatGpt3dot5Request.setStream(true);
        chatGpt3dot5Request.setMessages(Arrays.asList(
                new ChatGpt3dot5Request.Message("user", prompt)));
        ObjectMapper objectMapper = new ObjectMapper();
        String json = objectMapper.writeValueAsString(chatGpt3dot5Request);
        return json;
    }
}

The ChatBotController sends an HTTP request to ChatGPT API and reads its response stream line by line. Each line of the response has a similar structure. We extract the chat content by removing the prefix data: , parsing the string to a JSON object, and retrieving the content value. This content is then written into the response of our chatbot API. The following is an example of the content of the ChatGPT API response.

Click to expand!

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"Hi"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" there"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"!"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" How"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" can"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" I"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" assist"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" you"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" today"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"?"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-6zET7QIiAHSVeqoBwPNncxtIL3L6O","object":"chat.completion.chunk","created":1680051645,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]}

data: [DONE]

Call your spring boot application API

Send a “hello” prompt to ChatGPT API by your API

curl -d 'hello' -H 'Content-Type: application/json' -X POST http://localhost:8080/chatbot/conversation

Hi there! How can I assist you today?

Note: If ChatGPT is not available in your region, you will not be able to call the ChatGPT API and the request will time out. Your application may throw the following exception:

2023-03-16 14:40:22.680 ERROR 14704 --- [nio-8080-exec-3] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://api.openai.com/v1/engines/davinci-codex/completions": Connection timed out: connect; nested exception is java.net.ConnectException: Connection timed out: connect] with root cause
java.net.ConnectException: Connection timed out: connect