在日常生活中,文件上傳相關的操作隨處可見,大到處理大數(shù)據(jù)量的文件,小到頭像上傳,都離不開文件上傳操作,但是當一個文件的大小超過了某個閾值時,這個文件的上傳過程就會變得及其的慢,且會消耗大量網(wǎng)絡資源,這是我們不愿意看到的,所以,文件分片上傳孕育而生。
什么是文件分片上傳?
文件分片上傳就是將一整個文件分為幾個小塊,然后將這幾個小塊分別傳送給服務器,從而實現(xiàn)分片上傳。
上圖為文件分片的圖解,在本圖中,我們假定每一個分片都為67MB。(只是演示,實際文件分片需要考慮更多細節(jié))
如果當我們分片到最后一片的時候,我們就會直接將剩余所有空間存放到一個切片中,不管大小是否足夠我們指定的大小。
注意:這里的最后一片是指剩余的文件大小小于等于我們分片指定大小的情況。
文件分片時需要考慮什么?
在進行文件分片時,我們需要按照實際情況下文件大小來指定每一個切片的大小。并且需要在切片后將所有切片數(shù)量做記錄,具體流程將以列表形式呈現(xiàn):
前端
- 獲取文件,并規(guī)定一些常量(如切片大小,和后端約定的狀態(tài)信息等等)
- 開始文件切片,并將切片存儲到數(shù)組中
- 將切片數(shù)組中的切片轉(zhuǎn)換為二進制形式(原數(shù)組不變,只取數(shù)據(jù))并添加到緩沖區(qū)(SparkMD5庫提供的緩沖區(qū))中
- 確保所有切片全都存入緩沖區(qū)(這時候緩沖區(qū)內(nèi)的其實就是我們的整體文件,所有切片都合并了),然后計算文件hash.
- 開始對后端進行數(shù)據(jù)交互(上傳分片,提示合并,檢查是否已經(jīng)上傳文件 等)
后端
- 從前端獲取相關信息(如文件hash,文件名,切片文件等)
- 檢查是否已經(jīng)上傳過相同文件
- 等待所有切片文件存儲完成,并接收前端的合并通知(這一條看個人,也可以在后端直接計算是否拿到所有切片)
- 確保拿到所有切片文件后,開始讀取切片文件的二進制信息,并將其添加到緩沖區(qū)中
- 讀取完全部文件后,將緩沖區(qū)數(shù)據(jù)寫入指定文件中
- 將切片文件全部刪除
以上是文件分片上傳時前后端的基礎流程(可能有些地方寫的不夠嚴謹,希望各位大佬指教)
特別注意:在文件合并時要注意分片文件合并的順序問題,如果順序顛倒,那文件自然無法正常顯示。
個人建議所有分片文件命名后面跟上一個索引.
代碼實戰(zhàn)
聲明:此代碼沒有考慮過多細節(jié),只是作為一個基礎展示的案例。
前端
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport"
content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Document</title>
<style>
.msg{
font-size: 20px;
font-weight: bold;
}
</style>
</head>
<body>
<input type="file">
<p class="msg"></p>
<script src="js/axios.js"></script>
<script src="js/spark-md5.js"></script>
<script>
const statusCode = {
UPLOAD_SUCCESS: 200,
NOT_UPLOAD: 202,
ALREADY_UPLOAD: 1000,
UPLOAD_FAILED: 1004
}
let chunkSize = 2 * 1024 * 1024
let msg = document.querySelector(".msg")
let file = document.querySelector("input[type='file']")
file.addEventListener("change", async (e) => {
let fileList = e.target.files
let file = fileList[0]
let chunkArr = chunk(file, chunkSize)
let fileHash = await hash(chunkArr)
let filename = file.name
//false:沒上傳 true:上傳過了
let hasUpload = await check(fileHash, filename)
if (!hasUpload) {
let promises = []
for (let i = 0; i < chunkArr.length; i++) {
//將最后的返回結(jié)果添加到數(shù)組中
let res = await upload(fileHash, chunkArr, i, filename)
promises.push(res)
}
Promise.all(promises).then(res => {
mergeNotify(fileHash, filename, chunkArr.length)
msg.innerHTML="文件上傳成功"
msg.style.color="green"
}).catch(err => {
console.error(err)
})
} else {
//文件上傳過了,無需再次上傳
msg.innerHTML="文件已經(jīng)上傳!!"
msg.style.color="red"
}
})
/**
*
* @param file 文件File對象
* @param chunkSize 每一個切片的大小
* @return {[]} 返回切片數(shù)組
*/
const chunk = (file, chunkSize) => {
let res = []
for (let i = 0; i < file.size; i += chunkSize) {
res.push(file.slice(i, i + chunkSize))
}
return res
}
/**
*
* @param chunks 切片數(shù)組
* @return string 返回文件hash
*/
const hash = async (chunks) => {
let sparkMD5 = new SparkMD5.ArrayBuffer()
//存儲每個切片加密的任務狀態(tài),全部完成后,才會返回最終hash
let promises = []
//將切片數(shù)組所有切片轉(zhuǎn)為二進制,并將其合并為一個完整文件
for (let i = 0; i < chunks.length; i++) {
//由于hash加密耗時,所以我們采用異步
let promise = new Promise((resolve, reject) => {
let fileReader = new FileReader()//使用fileReader對象將文件切片轉(zhuǎn)為二進制
fileReader.readAsArrayBuffer(chunks[i])
fileReader.onload = (e) => {
//添加到SparkMD5中,等所有切片添加完畢后,獲取最終哈希
sparkMD5.append(e.target.result)
//每次添加成功后返回一個成功狀態(tài)
resolve()
}
fileReader.onerror = (e) => {
reject(e.target.error)
}
})
//將該promise任務添加到promise數(shù)組中
promises.push(promise)
}
//當所有加密任務全都完成后,返回加密后的完整文件hash
return await Promise.all(promises).then(res => {
return sparkMD5.end()
}).catch(err => {
console.error("Hash加密出現(xiàn)問題")
})
}
/***
*
* @param hash 文件hash
* @param chunks 切片數(shù)組
* @param currentIndex 當前切片索引
* @param filename 文件名
* @return 返回Promise,用于檢測當前切片是否上傳成功
*/
const upload = (hash, chunks, currentIndex, filename) => {
return new Promise((resolve, reject) => {
let formData = new FormData()
formData.append("hash", hash)
formData.append("chunkIndex", currentIndex)
formData.append("filename", filename)
formData.append("chunkBody", chunks[currentIndex])
axios.post("http://localhost:8080/upload", formData).then(res => {
//出現(xiàn)無法判斷是否成功的問題,推薦判斷是否成功在Promise.all中判斷
resolve("")
}).catch(err => {
reject(err)
})
})
}
/***
* 通知后端接口:可以開始合并任務了
* @param hash 文件hash
* @param filename 文件名
*/
const mergeNotify = (hash, filename, chunksLen) => {
let formData = new FormData()
formData.append("filename", filename)
formData.append("fileHash", hash)
formData.append("totalChunk", chunksLen)
axios.post("http://localhost:8080/merge", formData).then(res => {})
}
/**
* 檢查文件是否上傳
* @param hash 文件hash
* @param filename 文件名
* @return {Promise<Boolean>} 返回一個Promise對象
*/
const check = async (hash, filename) => {
let formData = new FormData()
formData.append("filename", filename)
formData.append("fileHash", hash)
let hasUpload = axios.post("http://localhost:8080/check", formData).then(res => {
let result;
//判斷是否上傳過該文件
if (res.data.code === statusCode.NOT_UPLOAD) {
result = false
} else {
result = true
}
//返回promise對象
return Promise.resolve(result)
})
return hasUpload
}
</script>
</body>
</html>
后端
entity
BaseFile
package com.cc.fileupload.entity;
/**
* @author CC
* @date Created in 2024/2/7 12:15
*/
public class BaseFile {
/**
* 文件hash
*/
private String fileHash;
public BaseFile() {
}
public BaseFile(String fileHash, String filename) {
this.fileHash = fileHash;
this.filename = filename;
}
/**
* 文件名
*/
private String filename;
@Override
public String toString() {
return "BaseFile{" +
"fileHash='" + fileHash + '\'' +
", filename='" + filename + '\'' +
'}';
}
public String getFileHash() {
return fileHash;
}
public void setFileHash(String fileHash) {
this.fileHash = fileHash;
}
public String getFilename() {
return filename;
}
public void setFilename(String filename) {
this.filename = filename;
}
}
MergeFile
package com.cc.fileupload.entity;
/**
* @author CC
* @date Created in 2024/2/7 11:27
*/
public class MergeFile {
/**
* 文件名
*/
private String filename;
/**
* 文件hash
*/
private String fileHash;
/**
* 切片總數(shù)
*/
private Integer totalChunk;
public String getFilename() {
return filename;
}
public void setFilename(String filename) {
this.filename = filename;
}
public String getFileHash() {
return fileHash;
}
public void setFileHash(String fileHash) {
this.fileHash = fileHash;
}
public Integer getTotalChunk() {
return totalChunk;
}
@Override
public String toString() {
return "MergeFile{" +
"filename='" + filename + '\'' +
", fileHash='" + fileHash + '\'' +
", totalChunk=" + totalChunk +
'}';
}
public void setTotalChunk(Integer totalChunk) {
this.totalChunk = totalChunk;
}
public MergeFile() {
}
public MergeFile(String filename, String fileHash, Integer totalChunk) {
this.filename = filename;
this.fileHash = fileHash;
this.totalChunk = totalChunk;
}
}
UploadFile
package com.cc.fileupload.entity;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.multipart.MultipartFile;
/**
* @author CC
* @date Created in 2024/2/7 10:33
*/
public class UploadFile {
/**
* 傳入的切片文件
*/
private MultipartFile chunkBody;
/**
* 文件hash
*/
private String hash;
/**
* 文件名
*/
private String filename;
/**
* 當前切片的索引號
*/
private Integer chunkIndex;
public MultipartFile getChunkBody() {
return chunkBody;
}
public void setChunkBody(MultipartFile chunkBody) {
this.chunkBody = chunkBody;
}
public String getHash() {
return hash;
}
public void setHash(String hash) {
this.hash = hash;
}
public String getFilename() {
return filename;
}
public void setFilename(String filename) {
this.filename = filename;
}
public Integer getChunkIndex() {
return chunkIndex;
}
public void setChunkIndex(Integer chunkIndex) {
this.chunkIndex = chunkIndex;
}
@Override
public String toString() {
return "UploadFile{" +
"chunkBody=" + chunkBody +
", hash='" + hash + '\'' +
", filename='" + filename + '\'' +
", chunkIndex=" + chunkIndex +
'}';
}
}
util
Helper
package com.cc.fileupload.util;
/**
* @author CC
* @date Created in 2024/2/7 10:49
*/
public class Helper {
/**
* 構(gòu)建切片文件名
*
* @param baseName 基礎文件名
* @param index 文件索引
* @return 返回切片文件名
*/
public static String buildChunkName(String baseName, Integer index) {
int i = baseName.lastIndexOf(".");
String prefix = baseName.substring(0, i).replaceAll("\\.", "_");
return prefix + "_part_" + index;
}
public static <T> ResultFormat<T> getReturnMsg(Integer code, T data, String msg) {
return new ResultFormat<T>(data, msg, code);
}
public static <T> ResultFormat<T> getReturnMsg(Integer code, T data) {
return new ResultFormat<T>(data, code);
}
public static ResultFormat<String> getReturnMsg(Integer code, String msg) {
return new ResultFormat<>(msg, code);
}
public static ResultFormat<Integer> getReturnMsg(Integer code){
return new ResultFormat<>(code);
}
//
// public static void main(String[] args) {
// String s = buildChunkName("test.xx.txt", 1);
// System.out.println(s);
// }
}
ResultFormat
package com.cc.fileupload.util;
/**
* @author CC
* @date Created in 2024/2/7 11:46
*/
public class ResultFormat<T> {
private T data;
private String msg;
private Integer code;
@Override
public String toString() {
return "{" +
"data=" + data +
", msg='" + msg + '\'' +
", code=" + code +
'}';
}
public T getData() {
return data;
}
public void setData(T data) {
this.data = data;
}
public String getMsg() {
return msg;
}
public void setMsg(String msg) {
this.msg = msg;
}
public Integer getCode() {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public ResultFormat(String msg, Integer code) {
this.msg = msg;
this.code = code;
}
public ResultFormat(Integer code) {
this.code = code;
}
public ResultFormat(T data, Integer code) {
this.data = data;
this.code = code;
}
public ResultFormat(T data, String msg, Integer code) {
this.data = data;
this.msg = msg;
this.code = code;
}
}
StatusCode?
package com.cc.fileupload.util;
/**
* @author CC
* @date Created in 2024/2/7 11:46
*/
public enum StatusCode {
UPLOAD_SUCCESS(200),
NOT_UPLOAD(202),
ALREADY_UPLOAD(1000),
UPLOAD_FAILED(1004);
private java.lang.Integer code;
StatusCode(java.lang.Integer code) {
this.code = code;
}
public java.lang.Integer getCode() {
return code;
}
public void setCode(java.lang.Integer code) {
this.code = code;
}
}
service
UploadService
package com.cc.fileupload.service;
import com.cc.fileupload.entity.BaseFile;
import com.cc.fileupload.entity.MergeFile;
import com.cc.fileupload.entity.UploadFile;
import com.cc.fileupload.util.ResultFormat;
import java.io.File;
/**
* @author CC
* @date Created in 2024/2/7 10:46
*/
public interface UploadService {
/**
* 上傳文件并保存切片的操作
*
* @param uploadFile 文件上傳實體類
* @return 返回狀態(tài)信息
*/
ResultFormat upload(UploadFile uploadFile);
/**
* 合并文件切片
*
* @param mergeFile 合并文件實體類
*/
void merge(MergeFile mergeFile);
/**
* 對文件的切片做刪除操作
* @param mergeFile 合并文件實體類
*/
void deleteChunks(MergeFile mergeFile);
/**
*
* @param baseFile 檢查文件是否已經(jīng)上傳
* @return 返回狀態(tài)信息
*/
ResultFormat<Integer> checkHasUpload(BaseFile baseFile);
}
IUploadService
package com.cc.fileupload.service.impl;
import com.cc.fileupload.entity.BaseFile;
import com.cc.fileupload.entity.MergeFile;
import com.cc.fileupload.entity.UploadFile;
import com.cc.fileupload.service.UploadService;
import com.cc.fileupload.util.Helper;
import com.cc.fileupload.util.ResultFormat;
import com.cc.fileupload.util.StatusCode;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files;
/**
* @author CC
* @date Created in 2024/2/7 10:46
*/
@Service
public class IUploadService implements UploadService {
private static final String BASE_PATH = "D:\\桌面\\圖片";
@Override
public ResultFormat<java.lang.Integer> checkHasUpload(BaseFile mergeFile) {
String fileHash = mergeFile.getFileHash();
String filename = mergeFile.getFilename();
File folder = new File(BASE_PATH, fileHash);
if (folder.exists()) {
File file = new File(folder, filename);
if (file.exists()) {
return Helper.getReturnMsg(StatusCode.ALREADY_UPLOAD.getCode());
}
}
return Helper.getReturnMsg(StatusCode.NOT_UPLOAD.getCode());
}
@Override
public ResultFormat upload(UploadFile uploadFile) {
String filename = uploadFile.getFilename();
String hash = uploadFile.getHash();
java.lang.Integer currentChunkIndex = uploadFile.getChunkIndex();
MultipartFile chunkBody = uploadFile.getChunkBody();
//根據(jù)hash來創(chuàng)建文件夾,有助于檢測是否上傳
File folder = new File(BASE_PATH, hash);
if (!folder.exists()) {
folder.mkdirs();
}
//這里獲取需要寫入的文件路徑和文件名
File file1 = new File(folder, Helper.buildChunkName(filename, currentChunkIndex));
try {
//文件寫入
chunkBody.transferTo(file1);
return Helper.getReturnMsg(StatusCode.UPLOAD_SUCCESS.getCode(), "上傳成功");
} catch (IOException e) {
System.out.println("出現(xiàn)錯誤");
e.printStackTrace();
}
//對文件進行寫入
return Helper.getReturnMsg(StatusCode.UPLOAD_FAILED.getCode(), "上傳失敗");
}
@Override
public void deleteChunks(MergeFile mergeFile) {
File hashFolder = new File(BASE_PATH, mergeFile.getFileHash());
java.lang.Integer totalChunk = mergeFile.getTotalChunk();
String filename = mergeFile.getFilename();
for (int i = 0; i < totalChunk; i++) {
//獲取切片
File tmpChunkFile = new File(hashFolder, Helper.buildChunkName(filename, i));
tmpChunkFile.delete();
}
}
@Override
public void merge(MergeFile mergeFile) {
String hash = mergeFile.getFileHash();
String filename = mergeFile.getFilename();
java.lang.Integer totalChunk = mergeFile.getTotalChunk();
//文件hash的Folder
File hashFolder = new File(BASE_PATH, hash);
OutputStream os = null;
//檢查是否有該hash目錄
try {
if (hashFolder.exists()) {
//指定最后輸出的文件名
os = new FileOutputStream(new File(hashFolder, filename));
for (int i = 0; i < totalChunk; i++) {
//獲取切片
File tmpChunkFile = new File(hashFolder, Helper.buildChunkName(filename, i));
//數(shù)據(jù)讀取并寫入緩存區(qū)
byte[] bytes = Files.readAllBytes(tmpChunkFile.toPath());
//將每一個切片數(shù)據(jù)讀取寫入緩存區(qū)
os.write(bytes);
}
//在將每一個切片的字節(jié)全都寫入緩沖區(qū)后,最后合并輸出文件
os.flush();
//輸出后清理臨時文件
deleteChunks(mergeFile);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
//資源關閉
if (os != null) {
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
controller
UploadController
package com.cc.fileupload.controller;
import com.cc.fileupload.entity.BaseFile;
import com.cc.fileupload.entity.MergeFile;
import com.cc.fileupload.entity.UploadFile;
import com.cc.fileupload.service.UploadService;
import com.cc.fileupload.util.ResultFormat;
import org.springframework.web.bind.annotation.*;
import javax.annotation.Resource;
/**
* @author CC
* @date Created in 2024/2/7 9:46
*/
@RestController
@CrossOrigin
public class UploadController {
@Resource
private UploadService uploadService;
@RequestMapping("/upload")
public ResultFormat upload(@ModelAttribute UploadFile uploadFile) {
System.out.println("上傳");
return uploadService.upload(uploadFile);
}
@RequestMapping("/merge")
public void merge(@ModelAttribute MergeFile mergeFile) {
uploadService.merge(mergeFile);
}
@RequestMapping("/check")
public ResultFormat check(@ModelAttribute BaseFile file) {
System.out.println("檢查");
return uploadService.checkHasUpload(file);
}
}
github鏈接
前端:GitHub - wewCc/fileUpload_frontend: 文件上傳前端文件上傳前端. Contribute to wewCc/fileUpload_frontend development by creating an account on GitHub.https://github.com/wewCc/fileUpload_frontend
后端:https://github.com/wewCc/fileUploadhttps://github.com/wewCc/fileUpload文章來源:http://www.zghlxwxcb.cn/news/detail-827381.html
?文章來源地址http://www.zghlxwxcb.cn/news/detail-827381.html
到了這里,關于SpringBoot+前端文件分片上傳的文章就介紹完了。如果您還想了解更多內(nèi)容,請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關文章,希望大家以后多多支持TOY模板網(wǎng)!