Question

我正在尝试将S3存储桶文件夹中500k文件的ACL从“私有”更改为“公开读取”

有什么方法可以加快速度吗？

我正在使用以下代码段。

from boto3.session import Session
from multiprocessing.pool import ThreadPool
pool = ThreadPool(processes=100)

BUCKET_NAME = ""
aws_access_key_id = ""
aws_secret_access_key = ""
Prefix='pics/'

session = Session(aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)
_s3 = session.resource("s3")
_bucket = _s3.Bucket(BUCKET_NAME)

def upload(eachObject):
    eachObject.Acl().put(ACL='public-read')

counter = 0
filenames = []
for eachObject in _bucket.objects.filter(Prefix=Prefix):
    counter += 1
    filenames.append(eachObject)
    if counter % 100 == 0:
        pool.map(upload, filenames)
        print(counter)
if filenames:
    pool.map(upload, filenames)

Answer 1

据我所知，没有将ACL应用于整个存储桶，就无法简单地将ACL应用于包含相同前缀的所有项目，而无需遍历每个项目，如下所示：

bucketName='YOUR_BUCKET_NAME'
prefix="YOUR_FOLDER_PREFIX"

s3 = boto3.resource('s3')
bucket = s3.Bucket(bucketName)

[obj.Acl().put(ACL='public-read') for obj in bucket.objects.filter(Prefix=prefix).all()]

S3 Boto3 Python-将所有文件ACL更改为公共读取

1 个答案: