Elastic Beanstalk数据库池问题

时间:2017-03-10 03:53:05

标签: ruby-on-rails ruby activerecord sinatra puma

我最近决定从Heroku切换到通过他们的Elastic Beanstalk服务直接部署到AWS - 希望我的术语仍然可以找出AWS。我有这个应用程序在EB上运行,这里是Heroku(这是一个免费的Heroku应用程序,因此可能需要10秒钟才能启动)。该应用程序在Heroku上运行完美,但在EB上它有时只会加载。也许像4次中的1次,我得到Internal Server Error。我的数据库池设置为5,如果我将它扩展到25,它可以工作,但我是唯一一个使用该网站的人,而且从我听到的应该是充足的。

这是我的database.yml:

development:
  adapter: sqlite3
  database: db/website.db
  pool: 5
  timeout: 5000
test:
  adapter: sqlite3
  database: db/cucumber-tests.db
  pool: 5
  timeout: 5000
production:
  adapter: postgresql
  encoding: utf8
  database: <%= ENV['RDS_DB_NAME'] %>
  username: <%= ENV['RDS_USERNAME'] %>
  password: <%= ENV['RDS_PASSWORD'] %>
  pool: 5
  host: <%= ENV['RDS_HOSTNAME'] %>
  port: <%= ENV['RDS_PORT'] %>

旁注,在我弄清楚为什么第5池不够之后,如何让池大小可扩展?据我所知,没有RDS_POOL变量。有没有办法根据用户数量来扩展它?

这是我的puma错误日志:

-------------------------------------
/var/log/puma/puma.log
-------------------------------------
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:225:in `context'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:220:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/nulllogger.rb:9:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/head.rb:13:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:182:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:2013:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `block in call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1787:in `synchronize'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/deflater.rb:35:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/configuration.rb:232:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/server.rb:578:in `handle_request'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/server.rb:415:in `process_client'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/server.rb:275:in `block in run'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/thread_pool.rb:120:in `block in spawn_thread'
2017-03-10 03:43:14 - ActiveRecord::ConnectionTimeoutError - could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were i
n use:
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:202:in `block in wait_poll'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:193:in `loop'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:193:in `wait_poll'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:154:in `internal_poll'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:278:in `internal_poll'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:148:in `block in poll'
        /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:158:in `synchronize'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:148:in `poll'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:717:in `acquire_connection'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:490:in `checkout'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:364:in `connection'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:883:in `retrieve_connection'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_handling.rb:128:in `retrieve_connection'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_handling.rb:91:in `connection'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/associations/join_dependency.rb:96:in `initialize'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/finder_methods.rb:399:in `new'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/finder_methods.rb:399:in `construct_join_dependency'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/finder_methods.rb:321:in `exists?'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.2/lib/active_record/querying.rb:3:in `exists?'
        app/views/index.erb:66:in `block in singleton class'
app/views/index.erb:-6:in `instance_eval'
        app/views/index.erb:-6:in `singleton class'
        app/views/index.erb:-8:in `__tilt_47116404404940'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/tilt-2.0.6/lib/tilt/template.rb:161:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/tilt-2.0.6/lib/tilt/template.rb:161:in `evaluate'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/tilt-2.0.6/lib/tilt/template.rb:100:in `render'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:823:in `render'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:667:in `erb'
        /var/app/current/app/controllers/application_controller.rb:60:in `block in <class:ApplicationController>'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1611:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1611:in `block in compile!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `block (3 levels) in route!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:994:in `route_eval'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `block (2 levels) in route!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1015:in `block in process_route'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1013:in `catch'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1013:in `process_route'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:973:in `block in route!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:972:in `each'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:972:in `route!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1085:in `block in dispatch!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `block in invoke'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `catch'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `invoke'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1082:in `dispatch!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:907:in `block in call!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `block in invoke'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `catch'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `invoke'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:907:in `call!'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:895:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:225:in `context'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:220:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/nulllogger.rb:9:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/head.rb:13:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:182:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:2013:in `call'
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `block in call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1787:in `synchronize'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/rack-1.6.5/lib/rack/deflater.rb:35:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/configuration.rb:232:in `call'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/server.rb:578:in `handle_request'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/server.rb:415:in `process_client'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/server.rb:275:in `block in run'
        /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/puma-3.7.1/lib/puma/thread_pool.rb:120:in `block in spawn_thread'

- 所有代码都是开源的,可以找到here -

谢谢。非常感谢任何帮助。

更新:我不能让这个应用程序停止运行。我现在将游泳池设置为25。我还更改了DSN,以便old app can be found herethe new app here

更新2:我正在添加ruby-on-rails标记以获得更多牵引力,因为此问题是相关的,但项目使用的是Sinatra而不是Rails。

1 个答案:

答案 0 :(得分:0)

看起来你正在使用Puma--一个线程化的Web服务器。 Puma在一个进程中启动多个ruby线程以同时处理请求。美洲狮正在创建的线程数可能高于池中的数据库连接数(在database.yml中定义)。

ActiveRecord自然会打开一个数据库连接池并懒惰地创建新连接。您的database.yml似乎将最大池大小设置为5.这意味着ActiveRecord不会在池中创建超过5个连接,并且只在池为空并且要求连接时才创建新连接。

我在你的存储库中找不到puma配置,所以我假设puma正在使用默认值。 The default max threads for Puma is 16。这可能表示您的Web服务器有16个线程正在运行(最坏情况),并且第5个之后的每个线程都在等待连接返回到池中使用。这是你的16个线程共享的5个连接。作为最佳实践,我始终确保数据库池大小为n+1,其中n是将在我的应用程序中运行的最大线程数。

我无法猜测为什么这个问题在heroku中也不存在。