从Amazon S3向Django / Heroku提供媒体文件和静态文件

时间:2020-02-07 15:31:27

标签: django amazon-web-services heroku amazon-s3

我有一个有关在S3存储桶中提供媒体文件和静态文件的问题。如果有人能提出解决问题的建议,我将不胜感激。

背景

  • 我已经完成了许多教程,以基本了解Django的工作方式。为了巩固我所学的知识,我决定将许多概念重新定位到一个简单的Web应用程序中,然后将其在线部署。

https://tbdcl-allconf-production.herokuapp.com/

https://www.allconf.io/

  • 该网站显示了会议的列表,这些列表与会议系列有一对多的关系(例如 DjangoCon 父母,将 DjangoCon 2019 DjangoCon 2020 作为孩子)。
  • 该会议系列的徽标图像是其模型的一部分。该图像在所有会议列表中显示为卡的一部分:

https://tbdcl-allconf-production.herokuapp.com/conferences/

https://www.allconf.io/conferences/

  • 所有这些都是通过管理员添加的,因为它不希望用户能够更新会议列表。

问题

  • 在本地工作时一切正常,但是当我部署到Heroku时徽标不会显示(即使文件在已部署应用程序的管理员视图中可见)。
  • 快速搜索显示,Heroku无法提供媒体文件,因此我选择了(看似)最常见的方法-将 boto3 django-storages 添加到我的项目中,并且将其指向Amazon S3存储桶。
  • 我遵循了许多流行的教程来进行设置。
  • 但是,看起来该应用仍在寻找媒体和静态文件的“本身”,而不是S3存储桶。我可以这样说是因为:

  • 简单来说,即使我已经按照教程将AWS凭证添加到Django中,媒体文件和静态文件也不在S3存储桶中。

  • 当我在Chrome中打开开发人员视图时,图像+ CSS文件的源看起来像本地路径(每个文件的路径中都没有引用“ aws”或“ s3”)。

可能的解决方案

  • 经过更多的挖掘和基本的故障排除之后,我认为可能是两件事:
    1. Whitenoise仍在我的项目中设置(从我在本地仅使用静态文件时开始)。应该删除吗?是否与boto3和django-storages冲突?
    2. 我是否需要以某种方式(例如,使用def __save__方法)更新模型?
    3. 我是否需要迁移数据库(或任何其他数据库任务)以更新每个模型的图像文件的URL?
    4. 应用程序是否在Docker容器中运行是事实导致与“输出”到S3存储桶的通信问题?

尽管说实话,我完全迷路了!我在网上找不到任何资源可以解释此问题以及如何解决该问题。

我将非常感谢您提供任何建议。

如果您需要查看 settings.py conferences / models.py 文件,我将其粘贴在下面。

非常感谢!

settings.py

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

ENVIRONMENT = os.environ.get('ENVIRONMENT', default='production')

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.environ.get('SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = int(os.environ.get('DEBUG', default=0))

ALLOWED_HOSTS = ['tbdcl-allconf-production.herokuapp.com', '.allconf.io', 'localhost', '127.0.0.1']


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'whitenoise.runserver_nostatic',
    'django.contrib.staticfiles',

    # Third party
    'debug_toolbar',
    'storages',

    # Local
    'users.apps.UsersConfig',
    'pages.apps.PagesConfig',
    'conferences.apps.ConferencesConfig',
]

MIDDLEWARE = [
    'django.middleware.cache.UpdateCacheMiddleware',
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'debug_toolbar.middleware.DebugToolbarMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',
]

CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_SECONDS = 600
CACHE_MIDDLEWARE_KEY_PREFIX = ''

ROOT_URLCONF = 'allconf_project.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'allconf_project.wsgi.application'


# Database
# https://docs.djangoproject.com/en/2.2/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'SQL_NAME',
        'USER': 'SQL_USER',
        'PASSWORD': 'SQL_PASSWORD',
        'HOST': 'db',
        'PORT': 5432
    }
}

# Password validation
# https://docs.djangoproject.com/en/2.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/2.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'America/New_York'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.2/howto/static-files/

USE_S3 = os.environ.get('USE_S3') == 'TRUE'

if USE_S3:
    # aws settings
    AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
    AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
    AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
    AWS_S3_FILE_OVERWRITE = False
    AWS_DEFAULT_ACL = None
    AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
    AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
    # s3 static settings
    STATIC_LOCATION = 'static'
    STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{STATIC_LOCATION}/'
    STATICFILES_STORAGE = 'hello_django.storage_backends.StaticStorage'
    # s3 public media settings
    PUBLIC_MEDIA_LOCATION = 'media'
    MEDIA_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{PUBLIC_MEDIA_LOCATION}/'
    DEFAULT_FILE_STORAGE = 'hello_django.storage_backends.PublicMediaStorage'
    # s3 private media settings
    PRIVATE_MEDIA_LOCATION = 'private'
    PRIVATE_FILE_STORAGE = 'hello_django.storage_backends.PrivateMediaStorage'
else:
    STATIC_URL = '/staticfiles/'
    STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
    MEDIA_URL = '/mediafiles/'
    MEDIA_ROOT = os.path.join(BASE_DIR, 'mediafiles')

STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
STATICFILES_FINDERS = [
    "django.contrib.staticfiles.finders.FileSystemFinder",
    "django.contrib.staticfiles.finders.AppDirectoriesFinder",
]


# User authentication

AUTH_USER_MODEL = 'users.CustomUser'


# django-debug-toolbar

import socket
hostname, _, ips = socket.gethostbyname_ex(socket.gethostname())
INTERNAL_IPS = [ip[:-1] + "1" for ip in ips]


# production

if ENVIRONMENT == 'production':
    SECURE_BROWSER_XSS_FILTER = True
    X_FRAME_OPTIONS = 'DENY'
    SECURE_SSL_REDIRECT = True
    SECURE_HSTS_SECONDS = 3600
    SECURE_HSTS_INCLUDE_SUBDOMAINS = True
    SECURE_HSTS_PRELOAD = True
    SECURE_CONTENT_TYPE_NOSNIFF = True
    SESSION_COOKIE_SECURE = True
    CSRF_COOKIE_SECURE = True
    SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')


# heroku

import dj_database_url
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)

storage_backends.py

from storages.backends.s3boto3 import S3Boto3Storage
from django.conf import settings


class StaticStorage(S3Boto3Storage):
    location = 'static'
    default_acl = 'public-read'


class PublicMediaStorage(S3Boto3Storage):
    location = 'media'
    default_acl = 'public-read'
    file_overwrite = False


class PrivateMediaStorage(S3Boto3Storage):
    location = 'private'
    default_acl = 'private'
    file_overwrite = False
    custom_domain = False

conferences / models.py

import uuid
from django.conf import settings
from django.contrib.auth import get_user_model
from django.db import models
from django.urls import reverse


class Series(models.Model):
    slug = models.SlugField(null=False, unique=True)
    title = models.CharField(max_length=200)
    description = models.CharField(max_length=300)
    series_logo = models.ImageField(upload_to='series_logos/', blank=True)

    class Meta: 
        indexes = [
            models.Index(fields=['slug'], name='slug_index'),
        ]

    def __str__(self):
        return self.title

    def get_absolute_url(self):
        return reverse('series_detail', kwargs={'slug': self.slug})


class Conference(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False,
    )
    title = models.CharField(max_length=255)
    start_date = models.DateField()
    end_date = models.DateField()
    location = models.CharField(max_length=255)
    conf_url = models.CharField(max_length=255)
    conf_series = models.ForeignKey(
        Series,
        null=True,
        on_delete=models.CASCADE,
        related_name='conferences',
    )

    def __str__(self):
        return self.title

    def get_absolute_url(self):
        return reverse('conference_detail', args=[str(self.id)])

1 个答案:

答案 0 :(得分:0)

看起来中间件中仍然存在白噪声,因此请尝试将其删除。我还将AWS_S3_CUSTOM_DOMAIN相对于AWS中显示的内容加倍。我认为该地区也应包括在内。

对我来说,我遇到这个问题似乎是因为django-heroku。我必须使用以下命令更新settings.py文件:

django_heroku.settings(locals(), staticfiles=False)