Web Project Functions: File Upload and Download

Content

• Upload
  • Upload files with Spring framework
  • Upload file with Java Servlet
• Download
  • Download file with spring framework
• The Example of Upload and download files in spring framework
• Read
  • Read data from disk
  • Read data from Java classpath
  • Read data from memory
  • Read data From HTTP URL
  • Read data from OSS
• Write
  • Write data to disk
  • Write data to memory
  • Write data to OSS
  • Print txt file to console
  • Package files to zip
• Performance
  • Speed up read or write
• Problems
  • Max upload file size
  • Character encoding of file name
  • URL encode of file name
• Appendixes
  • [Temporary Files and Directories](#Temporary Files and Directories)
  • Common Content-Type (MIME types) Table
• References

Upload

Upload files with Spring framework

@PostMapping(value = "/uploadFile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public String uploadFile(@RequestParam("file") MultipartFile file, String moduleUri) throws IOException {
    InputStream inputStream = file.getInputStream();
    String fileName = file.getOriginalFilename();
    // to save file
    ...
}

Upload file with Java Servlet

public class FileUploadByCommonsFileUploadServlet extends HttpServlet {
	public void doPost(HttpServletRequest request, HttpServletResponse response) throws Exception {
        DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
        diskFileItemFactory.setSizeThreshold(MAX_MEMORY_SIZE);
        diskFileItemFactory.setRepository(new File(System.getProperty("java.io.tmpdir")));
        ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
        servletFileUpload.setFileSizeMax(MAX_FILE_SIZE);
        servletFileUpload.setSizeMax(MAX_FILE_SIZE);
        List<FileItem> formItems = servletFileUpload.parseRequest(request);
        for (FileItem item : formItems) {
            if (!item.isFormField()) {
                String fileName = new File(item.getName()).getName();
                String filePath = new StringBuilder()
                    .append(UPLOAD_DIR)
                    .append(File.seperator)
                    .append(fileName)
                    .toString()
                item.write(new File(filePath));
            }
        }
    }
    ...
}

Download

Download file with spring framework

Write input streams to the OutPutStream of the HttpServletResponse

@GetMapping(value = "/download" )
public void download(HttpServletResponse response, String fileUri) {
    response.setCharacterEncoding("utf-8");
    response.setContentType("application/octet-stream");
    response.setHeader("Content-Disposition", "attachment;fileName=\"" + URLEncoder.encode(filename, "UTF-8").replace("+", "%20") + "\"");
    ServletOutputStream outputStream = response.getOutputStream();
    int bufSize = 1024;
    byte[] buffer = new byte[bufSize];
    int len;
    // get filePath by fileUri
    ...
    // get input stream by filePath
    ...
    while ((len = inputStream.read(buffer)) != -1) {
        outputStream.write(buffer, 0, len);
    }
    inputStream.close();
}

Write file Resource to ReponseBody

@GetMapping(value = "/download" )
public ResponseEntity<Resource> download(String fileUri) {
    // get filePath by fileUri
    //...
    String uploadDir = "D:\\upload";
    Path path = Paths.get(uploadDir).toAbsolutePath().normalize();
    path = path.resolve(fileUri).normalize();
    Resource resource = new UrlResource(path.toUri());
    String resultFileName = URLEncoder.encode(resource.getFilename().replace("%20", " "), "UTF-8").replace("+", "%20");
    return ResponseEntity.ok()
        .contentType(MediaType.parseMediaType("application/octet-stream"))
        .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + resultFileName + "\"")
        .body(resource);
}

Setting Response Headers

• content-type: application/octet-stream
• content-disposition: attachment; filename=”test.txt”

To encode filename when download file

original file name: “中 文.txt”

filename = resource.getFilename() // 中%20文.txt

browser download filename: -%20‡.txt

1

filename = URLEncoder.encode(resource.getFilename().replace("%20", " "), "UTF-8") // %E4%B8%AD+%E6%96%87.txt

browser download filename: 中+文.txt

2

filename = URLEncoder.encode(resource.getFilename().replace("%20", " "), "UTF-8").replace("+", "%20") // %E4%B8%AD%20%E6%96%87.txt

browser download filename: 中 文.txt

When browser resolve file name, they don’t decode “+” to “ “, so we need replace “+” to “%20”.

The Example of Upload and download files in spring framework

@RestController
@RequestMapping("files")
public class MyFileController {
    private static final Logger logger = LogManager.getLogger();

    public static final String UPLOAD_DIR = "D:\\My Desktop\\upload";

    public static final String DOWNLOAD_PREFIX = "/files/download";

    @PostMapping(value = "/uploadFile", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
    public String uploadFile(@RequestParam("file") MultipartFile file, String moduleUri) throws IOException {
        String uploadFileUri = getUploadFileUri(file, moduleUri);
        String uploadFilePath = new StringBuilder()
                .append(UPLOAD_DIR)
                .append(File.separator)
                .append(uploadFileUri)
                .toString();
        ensureDirectoryExist(uploadFilePath);
        try (
            BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(
                new FileOutputStream(uploadFilePath));
        ) {
            byte[] buf = new byte[1024];
            int len;
            InputStream inputStream = file.getInputStream();
            while ((len = inputStream.read(buf)) != -1) {
                bufferedOutputStream.write(buf, 0, len);
            }
        }
        return new StringBuilder()
                .append(DOWNLOAD_PREFIX)
                .append("/")
                .append(uploadFileUri)
                .toString();
    }

    private String getUploadFileUri(MultipartFile file, String moduleUri) {
        String uuid = UUID.randomUUID().toString().replaceAll("-", "");
        Calendar calendar = Calendar.getInstance();
        StringBuilder dateUri = new StringBuilder()
                .append(calendar.get(Calendar.YEAR))
                .append("/")
                .append(calendar.get(Calendar.MONTH) + 1)
                .append("/")
                .append(calendar.get(Calendar.DAY_OF_MONTH));
        StringBuilder resultFileUri = new StringBuilder()
                .append(moduleUri)
                .append("/")
                .append(dateUri)
                .append("/")
                .append(uuid)
                .append("/")
                .append(file.getOriginalFilename());
        return resultFileUri.toString();
    }

    private void ensureDirectoryExist(String filePath) {
        filePath = filePath.replace("/", File.separator).replace("\\", File.separator);
        String fileDir = filePath.substring(0, filePath.lastIndexOf(File.separator));
        logger.debug("fileDir: {}", fileDir);
        File dir = new File(fileDir);
        if (!dir.exists()) {
            dir.mkdirs();
        }
    }

    @GetMapping(value = "download/**")
    public void download(HttpServletRequest request, HttpServletResponse response) throws IOException {
        String servletPath = request.getServletPath();
        String fileUri = servletPath.substring(DOWNLOAD_PREFIX.length());
        String filePath = new StringBuilder()
                .append(UPLOAD_DIR)
                .append(File.separator)
                .append(fileUri)
                .toString();
        try (
                BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(filePath))
        ) {
            ServletOutputStream outputStream = response.getOutputStream();
            byte[] buf = new byte[1024];
            int len;
            while ((len = bufferedInputStream.read(buf)) != -1) {
                outputStream.write(buf, 0, len);
            }
        }
    }
}

Read

Read data from disk

java.io.FileInputStream

InputStream is = new FileInputStream("D:\\My Desktop\\test.txt");

org.springframework.core.io.UrlResource

Spring framework Resource Interface

Java’s standard java.net.URL class and standard handlers for various URL prefixes unfortunately are not quite adequate enough for all access to low-level resources. For example, there is no standardized URL implementation that may be used to access a resource that needs to be obtained from the classpath, or relative to a ServletContext. While it is possible to register new handlers for specialized URL prefixes (similar to existing handlers for prefixes such as http:), this is generally quite complicated, and the URL interface still lacks some desirable functionality, such as a method to check for the existence of the resource being pointed to.

String filepath = "D:\\My Desktop\\test.txt";
Path path = Paths.get(filepath).normalize();
//path.toUri().toString(): file:///D:/My%20Desktop/test.txt
Resource resource = new UrlResource(path.toUri()); 
InputStream is = resource.getInputStream();

java.net.URL

String filepath = "D:\\My Desktop\\test.txt";
Path path = Paths.get(filepath).normalize();
//path.toUri().toString(): file:///D:/My%20Desktop/test.txt
InputStream is = new URL(path.toUri().toString()).openStream();

Read data from Java classpath

org.springframework.core.io.ClassPathResource

// src/main/resources/application.yml
Resource resource = new ClassPathResource("application.yml");
InputStream is = resource.getInputStream();

org.springframework.core.io.ResourceLoader

@Autowired
ResourceLoader resourceLoader;

resourceLoader.getResource("classpath:application.yml");
InputStream is = resource.getInputStream();

Read data from memory

byte[] data = new String("data...").getBytes(StandardCharsets.UTF_8);
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);

Read data From HTTP URL

java.net.URL

InputStream is = new URL("http://demo.com/test.txt").openStream();

org.springframework.core.io.UrlResource

Resource resource = new UrlResource("http://demo.com/test.txt");
InputStream is = resource.getInputStream();

java.net.HttpURLConnection

URLConnection urlConnection = new URL("http://demo.com/test.txt").openConnection();
urlConnection.connect();
InputStream is = urlConnection.getInputStream();

Read data from OSS

Reference my another post: Common OSS Java SDK Usage

Write

Write data to disk

byte[] data = new String("data").getBytes(StandardCharsets.UTF_8);
String storeFilePath = "D:/My Workspace/test/test.txt";
try (
    ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
    FileOutputStream fileOutputStream = new FileOutputStream(storeFilePath)
) {
    byte[] buf = new byte[1024];
    int len = 0;
    while ((len = byteArrayInputStream.read(buf)) != -1) {
        fileOutputStream.write(buf, 0, len);
    }
} catch (IOException e) {
    e.printStackTrace();
}

Write data to memory

byte[] data = new String("data").getBytes(StandardCharsets.UTF_8);
byte[] storeDataArray;
try (
    ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
) {
    byte[] buf = new byte[1024];
    int len = 0;
    while ((len = byteArrayInputStream.read(buf)) != -1) {
        byteArrayOutputStream.write(buf, 0, len);
    }
    storeDataArray = byteArrayOutputStream.toByteArray();
    System.out.println(new String(storeDataArray, StandardCharsets.UTF_8));
} catch (IOException e) {
    e.printStackTrace();
}

Write data to OSS

Reference my another post: Common OSS Java SDK Usage

Print txt file to console

stream

InputStream is = new FileInputStream("D:\\My Desktop\\test.txt");
new BufferedReader(new InputStreamReader(is)).lines().forEach(System.out::println);

java.nio.file.Files

Path path = Paths.get("D:\\My Desktop\\test.txt");
System.out.println(new String(Files.readAllBytes(path)));

Package files to zip

basic zip

try (
    FileOutputStream fileOutputStream = new FileOutputStream("D:/My Workspace/test/test.zip");
    ZipOutputStream zipOutputStream = new ZipOutputStream(fileOutputStream)
) {
    int fileNumber = 3;
    for (int i = 1; i <= fileNumber; i++) {
        String childFilename = "file" + i + ".txt";
        zipOutputStream.putNextEntry(new ZipEntry(childFilename));
        zipOutputStream.write(("data" + i).getBytes(StandardCharsets.UTF_8));
        zipOutputStream.closeEntry();
    }
    zipOutputStream.finish();
} catch (IOException e) {
    e.printStackTrace();
}

nested zip

try (
    FileOutputStream fileOutputStream = new FileOutputStream("D:/My Workspace/test/test.zip");
    ZipOutputStream zipOutputStream = new ZipOutputStream(fileOutputStream)
) {
    int fileNumber = 3;
    for (int i = 1; i <= fileNumber; i++) {
        String childFilename = "file" + i + ".zip";
        zipOutputStream.putNextEntry(new ZipEntry(childFilename));
        try (
            ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
            ZipOutputStream nestedZipOutputStream = new ZipOutputStream(byteArrayOutputStream);
        ) {
            int nestedNumber = 3;
            for (int j = 1; j <= nestedNumber; j++) {
                String nestedFileName = new StringBuilder()
                    .append("file").append(i)
                    .append("nestFile").append(j)
                    .append(".txt").toString();
                nestedZipOutputStream.putNextEntry(new ZipEntry(nestedFileName));
                nestedZipOutputStream.write(("nest file" + j).getBytes(StandardCharsets.UTF_8));
                nestedZipOutputStream.closeEntry();
            }
            nestedZipOutputStream.finish();
            zipOutputStream.write(byteArrayOutputStream.toByteArray());
        }
        zipOutputStream.closeEntry();
    }
    zipOutputStream.finish();
} catch (IOException e) {
    e.printStackTrace();
}

Speed up read or write

Using buffer to decrease calls to the underlying runtime system

1. Direct Buffering

String inputFilePath = "C:\\Users\\Taogen\\Desktop\\input.txt";
String outputFilePath = "C:\\Users\\Taogen\\Desktop\\output.txt";
try (
    FileInputStream fileInputStream = new FileInputStream(inputFilePath);
    FileOutputStream fileOutputStream = new FileOutputStream(outputFilePath)
) {
    byte[] buf = new byte[2048];
    int len = 0;
    while ((len = fileInputStream.read(buf)) != -1) {
        fileOutputStream.write(buf, 0, len);
    }
} catch (IOException e) {
    e.printStackTrace();
}

2. Using buffered Input/output Stream

String inputFilePath = "C:\\Users\\Taogen\\Desktop\\input.txt";
String outputFilePath = "C:\\Users\\Taogen\\Desktop\\output.txt";
try (
    FileInputStream fileInputStream = new FileInputStream(inputFilePath);
    BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream);
    FileOutputStream fileOutputStream = new FileOutputStream(outputFilePath);
    BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
) {

    int b;
    while ((b=bufferedInputStream.read()) != -1) {
        bufferedOutputStream.write(b);
    }
} catch (IOException e) {
    e.printStackTrace();
}

BufferedInputStream‘s default buffer size is int DEFAULT_BUFFER_SIZE = 8192;

3. BufferedInputStream + Buffer array

String inputFilePath = "C:\\Users\\Taogen\\Desktop\\recovery_data.sql";
String outputFilePath = "C:\\Users\\Taogen\\Desktop\\recovery_data2.sql";
try (
    FileInputStream fileInputStream = new FileInputStream(inputFilePath);
    BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream);
    FileOutputStream fileOutputStream = new FileOutputStream(outputFilePath);
    BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
) {
    byte[] buf = new byte[1024];
    int len = 0;
    while ((len = bufferedInputStream.read(buf)) != -1) {
        bufferedOutputStream.write(buf, 0, len);
    }
} catch (IOException e) {
    e.printStackTrace();
}

Time cost: BufferedInputStream + Direct Buffer array < Direct Buffer array < Using a BufferedInputStream < Read Method.

Using BufferedInputStream rather than direct buffer is probably “right” for most applications. Because your used buffer size of direct buffer may be worse than BufferedInputStream default buffer size in speed.

Will making the buffer bigger make I/O go faster? Java buffers typically are by default 1024 or 2048 bytes long. A buffer larger than this may help speed I/O, but often by only a few percent, say 5 to 10%.

Problems

Max upload file size

Character encoding of file name

URL encode of file name

Appendixes

Temporary Files and Directories

java.io.tmpdir

System.getProperty("java.io.tmpdir")

Windows 10: C:\Users\{user}\AppData\Local\Temp\

Debian: /tmp

Create temporary file

// If you don't specify the file suffix, the default file suffix is ".tmp".
File file = File.createTempFile("temp", null);
System.out.println(file.getAbsolutePath());
file.deleteOnExit();

Common Content-Type (MIME types) Table

Java get file’s mimeType

// 1
String mimeType = Files.probeContentType(file.toPath());
// 2
String mimeType = URLConnection.guessContentTypeFromName(fileName);
// 3
FileNameMap fileNameMap = URLConnection.getFileNameMap();
String mimeType = fileNameMap.getContentTypeFor(file.getName());
// 4
MimetypesFileTypeMap fileTypeMap = new MimetypesFileTypeMap();
String mimeType = fileTypeMap.getContentType(file.getName());

Content-Type(Mime-Type)	Kind of document	Extension
application/octet-stream	Any kind of binary data	.bin
text/plain	Text, (generally ASCII or ISO 8859-n)	.txt
<Images>
image/bmp	Windows OS/2 Bitmap Graphics	.bmp
image/jpeg	JPEG images	.jpeg .jpg
image/png	Portable Network Graphics	.png
image/gif	Graphics Interchange Format (GIF)	.gif
image/vnd.microsoft.icon	Icon format	.ico
image/svg+xml	Scalable Vector Graphics (SVG)	.svg
<media>
audio/mpeg	MP3 audio	.mp3
video/mp4	MP4 video	.mp4
<code>
text/css	Cascading Style Sheets (CSS)	.css
text/csv	Comma-separated values (CSV)	.csv
application/json	JSON format	.json
text/javascript	JavaScript, JavaScript module	.js .mjs
text/html	HyperText Markup Language (HTML)	.htm .html
<doc>
application/msword	Microsoft Word	.doc
application/vnd.openxmlformats-officedocument.wordprocessingml.document	Microsoft Word (OpenXML)	.docx
application/pdf	Adobe Portable Document Format (PDF)	.pdf
<archive>
application/gzip	GZip Compressed Archive	.gz
application/vnd.rar	RAR archive	.rar
application/x-tar	Tape Archive (TAR)	.tar
application/zip	ZIP archive	.zip
application/x-7z-compressed	7-zip archive	.7z

Two primary MIME types are important for the role of default types:

• text/plain is the default value for textual files. A textual file should be human-readable and must not contain binary data.
• application/octet-stream is the default value for all other cases. An unknown file type should use this type. Browsers pay a particular care when manipulating these files, attempting to safeguard the user to prevent dangerous behaviors.

References

[1] Tuning Java I/O Performance

[2] Common MIME types - MDN Web Docs