不使用gunicorn加载的Tensorflow模型

我正在尝试将机器学习工作负载转移到生产环境中，现在工作负载正在Flask中运行，但它不是为生产而设计的，我还面临一些性能问题。 我尝试将gunicorn与Flask一起使用，但在加载tensorflow模型时遇到了一些问题，gunicorn服务器在加载模型之前启动。下面是代码

wsgi.py

from ml_service import service_api as application
import time
if __name__ == "__main__":
    # initialize the session
    ml_service.init()
    # load model
    ml_service.load_models()
    application.run()

ml_service.py

def init(num_gpu=1, num_cpu=1, num_cores=4):

    global session

    init_op = tf.global_variables_initializer()

    configuration = tf.ConfigProto(
        intra_op_parallelism_threads=num_cores,
        inter_op_parallelism_threads=num_cores, allow_soft_placement=True,
        # gpu_options=gpu_options,
        device_count={'CPU': num_cpu, 'GPU': num_gpu}
    )
    session = tf.Session(config=configuration)
    session.run(init_op)

def load_models():
    try:
        #Do Stuff
    except:
        #Do Stuff



@service_api.route("/serviceendpoint", methods=["POST"])
def get_prediction():
    #Do Stuff
    return



if __name__ == "__main__":
    service_api.run(host='0.0.0.0')

运行gunicorn的命令：

gunicorn --bind 0.0.0.0:5000 wsgi

输出：

[2019-12-17 14:16:18 +0000] [7] [INFO] Starting gunicorn 20.0.4
[2019-12-17 14:16:18 +0000] [7] [INFO] Listening at: http://0.0.0.0:5000 (7)
[2019-12-17 14:16:18 +0000] [7] [INFO] Using worker: sync
[2019-12-17 14:16:18 +0000] [10] [INFO] Booting worker with pid: 10
Using TensorFlow backend.

预期产出：

Using TensorFlow backend.
2019-12-17 13:17:00.497866: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-17 13:17:02.556270: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-17 13:17:02.556854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties: 
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2019-12-17 13:17:02.556902: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-12-17 13:17:02.970751: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-17 13:17:02.970851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0 
2019-12-17 13:17:02.970879: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N 
2019-12-17 13:17:02.971248: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10759 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
2019-12-17 13:17:03.197540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-12-17 13:17:03.197611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-17 13:17:03.197626: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0 
2019-12-17 13:17:03.197636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N 
2019-12-17 13:17:03.197822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10759 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
../posenet/converter/config.py:9: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  cfg = yaml.load(cfg_f)
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)