利用函数计算对oss压缩文件做自动解压处理

前言

一个应用可能已经使用对象存储（Object Storage Service，简称OSS）来存放用户上传的图片，用户可以实现一个函数去下载图片进行处理，并将结果存入OSS或者其他服务。如果OSS能够帮助我们关注新上传的图片，并且自动的去调用执行相应的处理函数，用户就不需要再去自己调用函数了，从而简化了开发和使用流程。
函数计算提供了一种事件驱动的计算模型。函数的执行是由事件驱动的，函数计算触发器描述了一组规则，当某个事件满足这些规则，事件源就会触发相应的函数。

repo 工程： fc-decompress-oss

示例内容

在本教程示例中，一个被存入OSS的文件，如果它是以source/为前缀，并且后缀是.gz .tar .zip的文件，那么OSS就会自动的调用相应的函数进行解压处理。

# -*- coding: utf-8 -*-
import oss2, json
import gzip
import tarfile 
import zipfile
import os
import cStringIO


# This template code can decompress the following three types of compression files.
#.gz .tar  .zip 
# if you want uncompress rar or for more info
# please refer to https://yq.aliyun.com/articles/468050?spm=a2c4e.11153959.teamhomeleft.1.1534d3c20HYMx5

def handler(event, context):
    """
    The object from OSS will be decompressed automatically .
    param: event:   The OSS event json string. Including oss object uri and other information.
    
    param: context: The function context, including credential and runtime info.

            For detail info, please refer to https://help.aliyun.com/document_detail/56316.html#using-context
    """
    evt_lst = json.loads(event)
    creds = context.credentials
    # Required by OSS sdk
    auth=oss2.StsAuth(
        creds.access_key_id,
        creds.access_key_secret,
        creds.security_token)
    evt = evt_lst['events'][0]
    bucket_name = evt['oss']['bucket']['name']
    endpoint = 'oss-' +  evt['region'] + '.aliyuncs.com'
    bucket = oss2.Bucket(auth, endpoint, bucket_name)
    object_name = evt['oss']['object']['key']

    """
    When a source/ prefix object is placed in an OSS, it is hoped that the object will be decompressed and then stored in the OSS as processed/ prefixed.
    For example, source/a.zip will be processed as processed/a/... 
    "Source /", "processed/" can be changed according to the user's requirements.
    """

    newKey = object_name.replace("source/", "processed/")
    remote_stream = bucket.get_object(object_name)
    if not remote_stream:
        raise RuntimeError('failed to get oss object. bucket: %s. object: %s' % (bucket_name, object_name))

    print 'download object from oss success: {}'.format(object_name)

    file_type = os.path.splitext(object_name)[1]


    if file_type == ".gz":
        data = cStringIO.StringIO(remote_stream.read())
        newKey = newKey.strip()[:-3]
        with gzip.GzipFile(mode = 'rb', fileobj = data) as f:
            r_data = f.read()  
            bucket.put_object(newKey, r_data)
            
    elif file_type == ".tar":
        data = cStringIO.StringIO(remote_stream.read())
        with tarfile.TarFile(mode = 'r', fileobj = data) as tar:
            newKey.replace(".tar", "")
            names = tar.getnames()
            for name in names:
                r = tar.extractfile(name)
                if r: # filter folder
                    bucket.put_object(newKey +  name, r.read())
                    r.close()
                    
    elif file_type == ".zip":
        data = cStringIO.StringIO(remote_stream.read())
        with zipfile.ZipFile(data,"r") as zip_file:
            newKey.replace(".zip", "")
            for name in zip_file.namelist():
                file = zip_file.open(name)
                r_data = file.read()
                if r_data: # filter folder
                    bucket.put_object(newKey +  name, r_data)
                file.close()

FAQ

如果是文件是后缀是.rar ，怎么解？

rar 这种格式如果用python，需要依赖第三方module和二进制文件，附件提供代码下载