Web Project Functions: Export Data

Ways of Exporting Requests

Synchronized Export

If to exported data is not large and the exporting process can be finished in a few seconds, we can just use the synchronized exporting. Send an export request, and then download the organized data file.

Asynchronized Export

If to exported data is too large, exporting will cost a lot of time. So we need to use the asynchronized exporting. Send an export request, view the schedule of the export, wait for the handling of exporting files to be finished, and download the organized data file.

Ways of Export implementations

  1. Write data into Java servlet response output stream.

  2. Write data into file and store in user temporary directory user.dir, and return download file URI. Delete file when download is finished.

  3. Upload data to OSS, and return download file URL.

The Limit of Max Size of Exported Data

Sometimes we need consider to setting the max size of data for exporting.

Fetch External Data

Fetch data from relational databases

Fetch static files with HTTP URLs

Build Exported Files

Exported File Types

  • Office Excel
  • Office Doc
  • Zip



  • SQL optimization. 1) Only query required columns. 2) Add index for query.


  • Cache rarely modified database data to Redis or memory.

Network IO

Fetch Database data

  • Fetch rows with multiple threads.
  • Each time to fetch a limited number of rows. It depends on the data size of a row. E.g. for the small size of a row, you can fetch 500 rows at a time.

Fetch Static Files by URL

  • Fetch files with multiple threads. E.g. 20 threads.
  • Caching files in the temporary directory.
  • Using blocking NIO, non-blocking NIO, or NIO2.


  • Compress images.
  • Compress text files.
  • Compress binary files.

Disk IO

  • When using traditional blocking IO, reading and writing files should use buffered Input/output wrapper class (E.g. BufferedInputStream) or read from/write to direct buffering array.

    // buffered Input/output wrapper class
    FileInputStream fis = new FileInputStream(filepath);
    BufferedInputStream bis = new BufferedInputStream(fis);
    // direct buffering array
    FileInputStream fis = new FileInputStream(filepath);
    byte buf[] = new byte[2048];
    int len;
    while ((len = fis.read(buf)) != -1) {}
  • (Optional) Using blocking NIO, non-blocking NIO, or NIO2. The Java NIO package offers the possibility to transfer bytes between two Channels without buffering them into the application memory. In single thread environment, traditional IO is better. NIO is used not because it’s faster but because it has better scalability especially there are amounts of clients.

    BufferedReader reader = null;
    BufferedWriter writer = null;
    try {
    Path inputFile = Paths.get(inputPath);
    Path outputFile = Paths.get(outputPath);
    reader = Files.newBufferedReader(inputFile, Charset.defaultCharset());
    writer = Files.newBufferedWriter(outputFile, Charset.defaultCharset());
    String lineFromFile = "";
    while ((lineFromFile = reader.readLine()) != null) {
    } catch (IOException e) {
    } finally {
    RandomAccessFile input = new RandomAccessFile(inputPath, "r");
    FileChannel inChannel = input.getChannel();
    RandomAccessFile output = new RandomAccessFile(outputPath, "rw");
    FileChannel outChannel = output.getChannel();
    int bufferSize = 1024;
    if (bufferSize > inChannel.size()) {
    bufferSize = (int) inChannel.size();
    ByteBuffer buff = ByteBuffer.allocate(bufferSize);
    while (inChannel.read(buff) != -1) {
    while (buff.hasRemaining()) {

Build Files

  • Build exported files with multiple threads. And then pack them into a zip file.


Out of memory

Be careful when loading large of data into memory.

Memory leak

Don’t forget close resources. E.g. thread pool, input/output streams and so on.


[1] Tuning Java I/O Performance - Oracle

[2] how to choose java nio vs io? - Stack Overflow

[3] NIO Performance Improvement compared to traditional IO in Java - Stack Overflow

[4] FileChannel, ByteBuffer, Memory-Mapped I/O, Locks