![个人头像照片](https://ucc.alicdn.com/avatar/avatar3.jpg)
暂无个人介绍
经过进一步的代码调试,发现 模型文件下载基本都会进入parallel_download下载逻辑,也就是并行下载,默认的并行度(线程数是4)。
parallel_download源码的基本实现逻辑就是构造了一个线程池,然后调用
download_part_with_retry方法去下载文件。
而download_part_with_retry方法里面也有一个无限循环(类似于http_get_file方法),代码片段如下:
retry = Retry(
total=API_FILE_DOWNLOAD_RETRY_TIMES,
backoff_factor=1,
allowed_methods=['GET'])
什么垃圾代码 ???
while True:
try:
with open(file_name, 'rb+') as f:
f.seek(start)
print("URL:",url)
r = requests.get(
url,
stream=True,
headers=get_headers,
cookies=cookies,
timeout=API_FILE_DOWNLOAD_TIMEOUT)
for chunk in r.iter_content(
chunk_size=API_FILE_DOWNLOAD_CHUNK_SIZE):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
progress.update(end - start)
break
except (Exception) as e: # no matter what exception, we will retry.
retry = retry.increment('GET', url, error=e)
logger.warning('Downloading: %s failed, reason: %s will retry' %
(model_file_name, e))
retry.sleep()