uWSGI产生太多线程

时间：2018-07-05 15:30:45

标签： python uwsgi systemd

类似于uwsgi spawning more processes than expected，但由于没有人回答，而且我没有足够的声誉向其提问，因此请再次讨论。 （UWSGI killing workers too fast）也可能相关。

我的情况：我有一个flask应用程序，uWSGI和nginx。 uWSGI配置的非系统执行是一种魅力。当我使用皇帝启动uWSGI时，我看到一个进程正在运行。当我将一个.ini文件添加到附庸国位置时，该事件会爆炸，启动和终止进程，以至于CPU使用率达到100％，而我正在使用的VM崩溃了。 .ini文件的内容似乎无关紧要（尝试过一个空文件）。

/ some / nfs / share / workers / conf 

PYTHONPATH=/some/nfs/share/production/repo
PATH=/some/nfs/share/production/bin:/sbin:/bin:/usr/sbin:/usr/bin
UWSGI_BIN=/some/nfs/share/production/bin/uwsgi
UWSGI_INI=/some/nfs/share/workers/emperor.ini

emperor.service 

[Unit]
Description=uWSGI Emperor
After=syslog.target

[Service]
User=myuser
Group=mygroup

EnvironmentFile=/some/nfs/share/workers/conf

ExecStart=/bin/sh -c '${UWSGI_BIN} --ini ${UWSGI_INI}'

RuntimeDirectory=uwsgi

RestartSec=5
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all

[Install]
WantedBy=multi-user.target

emperor.ini 

[uwsgi]
emperor = /some/nfs/share/uwsgi/

uid = myuser
gid = mygroup

实际服务.ini 

[uwsgi]
wsgi-file = /some/nfs/share/production/repo/app/main.py
callable = create_app

# load some variables
for-readline = /some/nfs/share/workers/conf
  env = %(_)
endfor =

plugins = python

wsgi-disable-file-wrapper = true

socket = /some/nfs/share/workers/uwsgi/%n.sock
pidfile = /some/nfs/share/workers/uwsgi/%n.pid

chmod-socket = 664
vacuum = true
die-on-term = true
close-on-exec = true

我尝试过的另一个虚拟服务.ini文件导致相同的问题（是的，它是空的）：

[uwsgi]

如果将日志记录添加到emepror.ini，则会得到：

emperor.ini 

[uwsgi]
emperor = /some/nfs/share/uwsgi/

uid = myuser
gid = evfold

logger = file:/some/nfs/share/uwsgi/emperor.log

这将产生两个进程，而不是一个：

● emperor.service - uWSGI Emperor
   Loaded: loaded (/etc/systemd/system/emperor.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2018-07-05 11:09:40 EDT; 2min 7s ago
 Main PID: 19777 (uwsgi)
   Status: "The Emperor is governing 0 vassals"
   CGroup: /system.slice/emperor.service
           ├─19777 /some/nfs/share/production/bin/uwsgi --ini /some/nfs/share/workers/emperor.ini
           └─19778 /some/nfs/share/production/bin/uwsgi --ini /some/nfs/share/workers/emperor.ini

Jul 05 11:09:40 backend systemd[1]: Starting uWSGI Emperor...
Jul 05 11:09:40 backend sh[19777]: [uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
Jul 05 11:09:40 backend sh[19777]: [uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
Jul 05 11:09:40 backend systemd[1]: Started uWSGI Emperor.

此日志文件到目前为止（请注意，所有内容都是重复的，很可能是因为记录器实例是单独的：

*** Starting uWSGI 2.0.16 (64bit) on [Thu Jul  5 11:09:40 2018] ***
*** Starting uWSGI 2.0.16 (64bit) on [Thu Jul  5 11:09:40 2018] ***
compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-15) on 12 February 2018 22:16:17
compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-15) on 12 February 2018 22:16:17
os: Linux-3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015
os: Linux-3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015
nodename: backend
nodename: backend
machine: x86_64
machine: x86_64
clock source: unix
clock source: unix
pcre jit disabled
pcre jit disabled
detected number of CPU cores: 4
detected number of CPU cores: 4
current working directory: /
current working directory: /
detected binary path: /some/nfs/share/production/bin/uwsgi
detected binary path: /some/nfs/share/production/bin/uwsgi
dropping root privileges as early as possible
dropping root privileges as early as possible
your processes number limit is 31212
your processes number limit is 31212
your memory page size is 4096 bytes
your memory page size is 4096 bytes
detected max file descriptor number: 1024
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
thunder lock: disabled (you can enable it with --thunder-lock)
*** starting uWSGI Emperor ***
*** starting uWSGI Emperor ***
dropping root privileges after socket binding
dropping root privileges after socket binding
Python version: 3.5.4 |Continuum Analytics, Inc.| (default, Aug 14 2017, 13:26:58)  [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Python version: 3.5.4 |Continuum Analytics, Inc.| (default, Aug 14 2017, 13:26:58)  [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1cf2180
Python main interpreter initialized at 0x1cf2180
dropping root privileges after plugin initialization
dropping root privileges after plugin initialization
your mercy for graceful operations on workers is 60 seconds
your mercy for graceful operations on workers is 60 seconds
*** Operational MODE: no-workers ***
*** Operational MODE: no-workers ***
dropping root privileges after application loading
dropping root privileges after application loading
spawned uWSGI master process (pid: 19777)
spawned uWSGI master process (pid: 19777)

如果我随后将production.ini添加到vassals文件夹中，则会收到以下日志：

*** has_emperor mode detected (fd: 10) ***
*** has_emperor mode detected (fd: 10) ***
[uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
[uWSGI] getting INI configuration from /some/nfs/share/workers/emperor.ini
[uWSGI] getting INI configuration from production.ini
[uWSGI] getting INI configuration from production.ini
open("./python_plugin.so"): No such file or directory [core/utils.c line 3713]
open("./python_plugin.so"): No such file or directory [core/utils.c line 3713]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
Thu Jul  5 11:22:29 2018 - [emperor] vassal production.ini is now loyal
Thu Jul  5 11:22:29 2018 - [emperor] vassal production.ini is now loyal

python_plugin错误似乎不会以任何特定方式影响执行，我可以从plugins = python文件中删除.ini，并且不会有太大变化。

在此阶段，上面有50多名与uWSGI工作者相关联的工作者，并且计算机完全没有响应。

我们将不胜感激：）

编辑：我能够隔离一次emperor.service，而无需进行日志记录，而在vassals中使用production.ini文件，它看起来像这样（我认为，这些容器应该是嵌套在主要PID里面吗？）

● emperor.service - uWSGI Emperor
   Loaded: loaded (/etc/systemd/system/emperor.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2018-07-05 12:01:57 EDT; 12min ago
 Main PID: 25432 (uwsgi)
   Status: "The Emperor is governing 1 vassals"
   CGroup: /system.slice/emperor.service
           ├─25432 /some/nfs/share/production/bin/uwsgi --ini /some/nfs/share/workers/emperor.ini
           ├─25960 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25961 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25962 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25963 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25964 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25965 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25966 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25967 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25968 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25969 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25970 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25971 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25972 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25973 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25974 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25975 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25976 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25977 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25978 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25979 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25980 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25981 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25982 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25983 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25984 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25985 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25986 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25987 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25988 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25989 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25990 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25991 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25992 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25993 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25994 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25995 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25996 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25997 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25998 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─25999 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26000 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26001 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26002 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26003 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26004 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26005 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26006 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26007 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26008 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26014 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26015 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26030 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26035 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26037 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26038 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26039 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26040 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26044 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26045 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26048 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26049 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26051 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26052 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26054 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26055 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26056 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26057 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26059 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26062 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26063 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26064 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26065 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26066 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26068 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26069 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26070 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26071 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26074 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26075 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26076 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26078 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26080 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26081 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26083 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26084 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26086 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26088 /some/nfs/share/production/bin/uwsgi --ini production.ini
           ├─26090 /some/nfs/share/production/bin/uwsgi --ini production.ini
           └─26091 /some/nfs/share/production/bin/uwsgi --ini production.ini

Jul 05 12:14:29 backend uwsgi[25432]: *** Starting uWSGI 2.0.16 (64bit) on [Thu Jul  5 12:14:29 2018] ***
Jul 05 12:14:29 backend uwsgi[25432]: compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-15) on 12 February 2018 22:16:17
Jul 05 12:14:29 backend uwsgi[25432]: os: Linux-3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015
Jul 05 12:14:29 backend uwsgi[25432]: nodename: backend
Jul 05 12:14:29 backend uwsgi[25432]: machine: x86_64
Jul 05 12:14:29 backend uwsgi[25432]: clock source: unix
Jul 05 12:14:29 backend uwsgi[25432]: pcre jit disabled
Jul 05 12:14:29 backend uwsgi[25432]: detected number of CPU cores: 4
Jul 05 12:14:29 backend uwsgi[25432]: current working directory: /some/nfs/share/production/bin/workers/uwsgi
Jul 05 12:14:29 backend uwsgi[25432]: writing pidfile to /some/nfs/share/production/bin/workers/uwsgi/production.pid

1 个答案:

答案 0 :(得分：0)

想通了：

如果您通过systemctl运行uWSGI，则使用--ini指令包括皇帝文件将被解释为ini实际上是服务。将命令放入ExecStart的正确方法是：

ExecStart=/bin/sh -c '${UWSGI_BIN} --emperor ${UWSGI_INI}'

其中UWSGI_BIN是二进制文件的位置，而UWSGI_INI是表示服务的.ini文件的位置。

我认为，如果要设置任何皇帝配置标志，这些标志应放在ExecStart中。有点不幸，但显然是这样。

相关问题
最新问题