背景
公司最近的业务大量涉及安可项目,要求避免使用第三方组件,原有开发框架支持本地文件存储/Minio/各类云存储,现在要求文件独立存储且文件服务需要自研,经调研评估后决定基于SpringBoot开发文件存储服务,使用s3协议标准,这样可以直接使用aws-sdk接入无需再开发客户端,且安全安全性方面可以得到足够的保证(签名验证部分参考我的博文《[Java实现AWS S3 V4 Authorization自定义验证]》)
项目地址:https://gitee.com/code2roc/local-s3
运行jar包,默认信息如下
api地址:http: //localhost:8001/s3
用户名:admin
密码:abcd@1234
概述
s3协议无标准说明文档,为rest风格,创建/删除/详情方法通过PUT/DELETE/HEAD表述
很多方法共用一个路由,通过head参数区分(例如putObject和copyObject)
参考aws的最新api文档:https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html
实现以下基础功能
Bucket创建
@PutMapping("/{bucketName}") public ResponseEntity<String> createBucket(@PathVariable String bucketName)
Bucket删除
@DeleteMapping("/{bucketName}") public ResponseEntity<String> deleteBucket(@PathVariable String bucketName)
文件上传
@PutMapping("/{bucketName}/**") public ResponseEntity<String> putObject(@PathVariable String bucketName, HttpServletRequest request)
文件删除
@DeleteMapping("/{bucketName}/**") public ResponseEntity<String> deleteObject(@PathVariable String bucketName, HttpServletRequest request)
文件下载
@GetMapping("/{bucketName}/**") public void getObject(@PathVariable String bucketName, HttpServletRequest request, HttpServletResponse response)
文件分片操作(初始化/分片上传/合并)
@RequestMapping(value = "/{bucketName}/**", method = RequestMethod.POST, params = "uploads") public ResponseEntity<Object> createMultipartUpload(@PathVariable String bucketName, HttpServletRequest request) @RequestMapping(value = "/{bucketName}/**", method = RequestMethod.PUT, params = {"partNumber", "uploadId"}) public ResponseEntity<String> uploadPart(@PathVariable String bucketName, HttpServletRequest request, HttpServletResponse response) @RequestMapping(value = "/{bucketName}/**", method = RequestMethod.POST, params = "uploadId") public ResponseEntity<String> completeMultipartUpload(@PathVariable String bucketName, HttpServletRequest request)
实现以下扩展功能(兼容s3 browser使用)
Bucket详情
@RequestMapping(value = "/{bucketName}", method = RequestMethod.HEAD) public ResponseEntity<Object> headBucket(@PathVariable(value = "bucketName") String bucketName)
Buckent列表
@GetMapping("/") public ResponseEntity<String> listBuckets()
文件详情
@RequestMapping(value = "/{bucketName}/**", method = RequestMethod.HEAD) public ResponseEntity<Object> headObject(@PathVariable String bucketName, HttpServletRequest request, HttpServletResponse response)
文件列表
@GetMapping("/{bucketName}") public ResponseEntity<String> listObjects(@PathVariable String bucketName, HttpServletRequest request)
项目接入
maven引用
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>2.20.45</version>
</dependency>
客户端连接
private S3Client getClient() {
S3Client s3 = S3Client.builder()
.credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(systemConfig.getUsername(), systemConfig.getPassword())))
.endpointOverride(URI.create(CommonUtil.getApiPath() + "s3/"))
.serviceConfiguration(S3Configuration.builder().pathStyleAccessEnabled(true).chunkedEncodingEnabled(false).build())
.region(Region.US_EAST_1)
.build();
return s3;
}
文件操作
public void upload(String bucketName, String key, InputStream inputStream) throws Exception {
S3Client s3Client = getClient();
PutObjectRequest request = PutObjectRequest.builder().bucket(bucketName).key(key).build();
RequestBody requestBody = RequestBody.fromBytes(FileUtil.convertStreamToByte(inputStream));
s3Client.putObject(request, requestBody);
s3Client.close();
}
工具使用
下载工具
链接:https://pan.baidu.com/s/1HnB3KUOQx4_QELkDTXyG2Q?pwd=nnio 提取码:nnio
配置连接
Account type:选择S3 Compatible Storage
EndPoint填写部署服务后的地址:http://ip:port/s3
Access Key ID:填写配置文件中的username
Secret Access Key:填写配置文件中的password
去除SSL选项
配置签名
在编辑连接页面点击左下角Advanced S3 Compatible Storage Setting
选择签名版本为V4
支持功能
支持创建桶/删除桶/上传文件/删除文件/下载文件/创建文件夹功能