背景
Minio是就地环境下比较好用的对象存储工具,适合在CI/CD流程中使用。主要是因为GIT里用LFS来放大文件不妥,把部署流程中需要的中间文件放minio上,通过SDK去存取文件非常方便。
Minio的上传文件fput_object有progress参数,但是下载文件fget_object默认没有 progress 参数,所以我们需要自己用get_object对代码稍加改造,把流下载和进度条结合在一起。先获取一下总文件大小,然后每次读取内容1MB,根据下载速率预估剩余时间
涉及到的库
https://github.com/verigak/progress
用于提供进度条
pip install progress pip install minio
代码
from minio import Minio from progress.bar import Bar def get_object_with_progress(client, bucket_name, object_name): try: data = client.get_object(bucket_name, object_name) total_length = int(data.headers.get('content-length')) bar = Bar(object_name, max=total_length / 1024 / 1024, fill='*', check_tty=False, suffix='%(percent).1f%% - %(eta_td)s') with open('./' + object_name, 'wb') as file_data: for d in data.stream(1024 * 1024): bar.next(1) file_data.write(d) bar.finish() except Exception as err: print(err) if __name__ == '__main__': client = Minio( "play.min.io", access_key="Q3AM3UQ867SPQQA43P2F", secret_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", ) bucket_name = "downlaod" object_name = "eiop-timescaledb-offline.zip" get_object_with_progress(client, bucket_name, object_name)
实现效果