Python 从.pb文件加载TensorRT优化的TensorFlow图的时间非常长（超过10分钟）

对于使用TensorRT优化的TensorFlow图，我经历了非常长的加载时间。非优化的加载速度很快，但加载优化的加载需要10分钟以上，代码相同：

trt_graph_def = tf.GraphDef()
with tf.gfile.GFile(pb_path, 'rb') as pf:
   trt_graph_def.ParseFromString(pf.read())

我使用的是NVIDIA Drive PX 2设备（如果需要的话），TensorFlow 1.12.0是从源代码CUDA 9.2和TensorRT 4.1.1构建的。 由于它在ParseFromString（）上卡住了，我怀疑protobuf，所以下面是它的配置：

$ dpkg -l | grep protobuf
ii libmirprotobuf3:arm64 0.26.3+16.04.20170605-0ubuntu1.1 arm64 Display server for Ubuntu - RPC definitions
ii libprotobuf-dev:arm64 2.6.1-1.3 arm64 protocol buffers C++ library (development files)
ii libprotobuf-lite9v5:arm64 2.6.1-1.3 arm64 protocol buffers C++ library (lite version)
ii libprotobuf9v5:arm64 2.6.1-1.3 arm64 protocol buffers C++ library
ii protobuf-compiler 2.6.1-1.3 arm64 compiler for protocol buffer definition files

$ pip3 freeze | grep protobuf
protobuf==3.6.1

以下是我将非优化模型转换为TRT模型的方法：

def get_frozen_graph(graph_file):
  """Read Frozen Graph file from disk."""
  with tf.gfile.FastGFile(graph_file, "rb") as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())
  return graph_def

print("Load frozen graph from disk")

frozen_graph = get_frozen_graph(DATA_DIR + MODEL + '.pb')

print("Optimize the model with TensorRT")

trt_graph = trt.create_inference_graph(
    input_graph_def=frozen_graph,
    outputs=output_names,
    max_batch_size=1,
    max_workspace_size_bytes=1 << 26,
    precision_mode='FP16',
    minimum_segment_size=2
)

print("Write optimized model to the file")
with open(DATA_DIR + MODEL + '_fp16_trt.pb', 'wb') as f:
    f.write(trt_graph.SerializeToString())

def get_freezed_图形（图形文件）：
“”“从磁盘读取冻结的图形文件。”“”
将tf.gfile.FastGFile（图形文件“rb”）作为f：
graph_def=tf.GraphDef（）
graph_def.ParseFromString（f.read（））
返回图
打印（“从磁盘加载冻结的图形”）
冻结图=获取冻结图（数据目录+模型+'.pb'）
打印（“使用TensorRT优化模型”）
trt\u图=trt.create\u推断图(
输入图定义=冻结图，
输出=输出名称，
最大批量大小=1，
max_workspace_size_bytes=1好的，我想我已经把它整理好了。我让protobuf 2.6.1几乎不动，只是从源代码中安装了3.6.1，旁边有cpp实现，并以3.6.1是默认的方式设置符号链接。现在：

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp

所有模型在几分之一秒内加载

以下是我所做的具体步骤，以供参考：

# Check current version
$ protoc --version
libprotoc 2.6.1

# Create a backup of the current config, just in case
mkdir protobuf
cd protobuf/
mkdir backup_originals
mkdir backup_originals/protoc
cp /usr/bin/protoc backup_originals/protoc/
tar cvzf backup_originals/libprotobuf.tgz /usr/lib/aarch64-linux-gnu/libprotobuf*
# Original include files located at: /usr/include/google/protobuf/
# I did not backed them up

# Original configuration of the libraries
$ ls -l /usr/lib/aarch64-linux-gnu/libprotobuf*
-rw-r--r-- 1 root root 2464506 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf.a
-rw-r--r-- 1 root root  430372 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf-lite.a
lrwxrwxrwx 1 root root      25 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf-lite.so -> libprotobuf-lite.so.9.0.1
lrwxrwxrwx 1 root root      25 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf-lite.so.9 -> libprotobuf-lite.so.9.0.1
-rw-r--r-- 1 root root  199096 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf-lite.so.9.0.1
lrwxrwxrwx 1 root root      20 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf.so -> libprotobuf.so.9.0.1
lrwxrwxrwx 1 root root      20 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf.so.9 -> libprotobuf.so.9.0.1
-rw-r--r-- 1 root root 1153872 Oct 24  2015 /usr/lib/aarch64-linux-gnu/libprotobuf.so.9.0.1

# Fetch and upack the sources of version 3.6.1
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.6.1/protobuf-python-3.6.1.zip
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.6.1/protoc-3.6.1-linux-aarch_64.zip
unzip protoc-3.6.1-linux-aarch_64.zip -d protoc-3.6.1
unzip protobuf-python-3.6.1.zip

# Update the protoc
sudo cp protoc-3.6.1/bin/protoc /usr/bin/protoc

$ protoc --version
libprotoc 3.6.1

# BUILD AND INSTALL THE LIBRARIES
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
cd protobuf-3.6.1/
./autogen.sh
./configure
make
make check
sudo make install

# Remove unnecessary links to the old version
sudo rm /usr/lib/aarch64-linux-gnu/libprotobuf.a
sudo rm /usr/lib/aarch64-linux-gnu/libprotobuf-lite.a
sudo rm /usr/lib/aarch64-linux-gnu/libprotobuf-lite.so
sudo rm /usr/lib/aarch64-linux-gnu/libprotobuf.so

# Move old version of the libraries to the same folder where the new ones have been installed, for clarity
sudo cp -d /usr/lib/aarch64-linux-gnu/libproto* /usr/local/lib/
sudo rm /usr/lib/aarch64-linux-gnu/libproto*


sudo ldconfig # Refresh shared library cache   


# Check the updated version
$ protoc --version
libprotoc 3.6.1


# Final configuration of the libraries after the update
$ ls -l /usr/local/lib/libproto*
-rw-r--r-- 1 root root 77064022 Feb  9 11:07 /usr/local/lib/libprotobuf.a
-rwxr-xr-x 1 root root      978 Feb  9 11:07 /usr/local/lib/libprotobuf.la
-rw-r--r-- 1 root root  9396522 Feb  9 11:07 /usr/local/lib/libprotobuf-lite.a
-rwxr-xr-x 1 root root     1013 Feb  9 11:07 /usr/local/lib/libprotobuf-lite.la
lrwxrwxrwx 1 root root       26 Feb  9 11:07 /usr/local/lib/libprotobuf-lite.so -> libprotobuf-lite.so.17.0.0
lrwxrwxrwx 1 root root       26 Feb  9 11:07 /usr/local/lib/libprotobuf-lite.so.17 -> libprotobuf-lite.so.17.0.0
-rwxr-xr-x 1 root root  3722376 Feb  9 11:07 /usr/local/lib/libprotobuf-lite.so.17.0.0
lrwxrwxrwx 1 root root       25 Feb  9 11:19 /usr/local/lib/libprotobuf-lite.so.9 -> libprotobuf-lite.so.9.0.1
-rw-r--r-- 1 root root   199096 Feb  9 11:19 /usr/local/lib/libprotobuf-lite.so.9.0.1
lrwxrwxrwx 1 root root       21 Feb  9 11:07 /usr/local/lib/libprotobuf.so -> libprotobuf.so.17.0.0
lrwxrwxrwx 1 root root       21 Feb  9 11:07 /usr/local/lib/libprotobuf.so.17 -> libprotobuf.so.17.0.0
-rwxr-xr-x 1 root root 30029352 Feb  9 11:07 /usr/local/lib/libprotobuf.so.17.0.0
lrwxrwxrwx 1 root root       20 Feb  9 11:19 /usr/local/lib/libprotobuf.so.9 -> libprotobuf.so.9.0.1
-rw-r--r-- 1 root root  1153872 Feb  9 11:19 /usr/local/lib/libprotobuf.so.9.0.1
-rw-r--r-- 1 root root 99883696 Feb  9 11:07 /usr/local/lib/libprotoc.a
-rwxr-xr-x 1 root root      994 Feb  9 11:07 /usr/local/lib/libprotoc.la
lrwxrwxrwx 1 root root       19 Feb  9 11:07 /usr/local/lib/libprotoc.so -> libprotoc.so.17.0.0
lrwxrwxrwx 1 root root       19 Feb  9 11:07 /usr/local/lib/libprotoc.so.17 -> libprotoc.so.17.0.0
-rwxr-xr-x 1 root root 32645760 Feb  9 11:07 /usr/local/lib/libprotoc.so.17.0.0
lrwxrwxrwx 1 root root       18 Feb  9 11:19 /usr/local/lib/libprotoc.so.9 -> libprotoc.so.9.0.1
-rw-r--r-- 1 root root   991440 Feb  9 11:19 /usr/local/lib/libprotoc.so.9.0.1

# Reboot, just in case :)
sudo reboot

# BUILD AND INSTALL THE PYTHON-PROTOBUF MODULE
cd protobuf-3.6.1/python/
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp


# Fix setup.py to force compilation with c++11 standard
vim setup.py

$ diff setup.py setup.py~
205,208c205,208
<     #if v:
<     #  extra_compile_args.append('-std=c++11')
<     #elif os.getenv('KOKORO_BUILD_NUMBER') or os.getenv('KOKORO_BUILD_ID'):
<     extra_compile_args.append('-std=c++11')
---
>     if v:
>       extra_compile_args.append('-std=c++11')
>     elif os.getenv('KOKORO_BUILD_NUMBER') or os.getenv('KOKORO_BUILD_ID'):
>       extra_compile_args.append('-std=c++11')

# Build, test and install
python3 setup.py build --cpp_implementation
python3 setup.py test --cpp_implementation
sudo python3 setup.py install --cpp_implementation

# Make the cpp backend a default one when user logs in
sudo sh -c "echo 'export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp' >> /etc/profile.d/protobuf.sh"

文件有多大？例如，对于model zoo的ssd_mobilenet_v2_coco，这两个文件的大小非常相似：67MB。精确的字节大小：69688296-原始型号，69219036-优化型号嗯，好吧，我不知道。文件读取效率有点低，但67MB文件的读取时间不到一秒钟。
wget http://se.archive.ubuntu.com/ubuntu/pool/universe/p/python-pip/python3-pip_9.0.1-2_all.deb
wget http://se.archive.ubuntu.com/ubuntu/pool/universe/p/python-pip/python-pip-whl_9.0.1-2_all.deb
sudo dpkg -i *.deb