Building a File Upload Service from Scratch with Vanilla JavaScript
The Magic Behind the Curtain
When it comes to building web applications, we often rely on third-party packages, compilers, and bundlers to get the job done. However, in our haste to meet deadlines and keep up with the competition, we often overlook the low-level details of how these packages work their magic. In this article, we’ll take a step back and build a file upload service from scratch using vanilla JavaScript, without relying on any external libraries.
What We’re Building
Our goal is to create a file upload service that allows users to select a file, read its contents on the frontend, and stream it to the backend in chunks. We’ll also explore how to upload multiple files and store them on the server.
Setting Up the Node.js Server
To get started, we’ll create a new folder for our project and create an index.js file that will serve as the entry point for our backend server. We’ll use the built-in HTTP package to set up our server, which will run on port 8080.
Setting Up the Frontend
Next, we’ll create a basic HTML file with a file input and an upload button. When the user clicks the upload button, we’ll attach an event listener to initiate the uploading process. We’ll also add a tiny status text to display the status of the file upload.
Reading File Contents on the Frontend
To read the file contents on the frontend, we’ll use the FileReader object. This allows us to asynchronously read the contents of files stored on the user’s computer. We’ll use the readAsArrayBuffer() method to read the file in bytes and stream it to the backend.
Divide and Stream the File in Chunks to the Backend
When dealing with large files, it’s not efficient to send the complete file at once. Instead, we’ll split the file into chunks of around 5000 bytes and send them to the backend one by one. We’ll use the fetch API to hit the backend server and make sure we send the chunks in order to avoid corrupting the file.
Receiving the Chunks and Storing Them on the Server
On the backend, we’ll listen for the file chunks and write them to the server. We’ll extract the file name from the query params of the request and use it to append the content to the respective file.
Uploading Multiple Files
To extend our implementation to support multiple file uploads, we’ll modify the file input to accept multiple files. We’ll then iterate over the array of selected files, break them into chunks, and stream them to the backend server.
The Final Result
Our implementation may not be the most efficient, but it gives us a good understanding of some core concepts in JavaScript. We can further extend it to add features like progress bars, retry chunk uploads, and more.
Get Involved
If you’re interested in exploring the GitHub repository of the codebase, you can find it here. I’m active on Twitter as @the2ndfloorguy and would love to hear your thoughts.