Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/276.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/tensorflow/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 在CPU上提供模型时出现InvalidArgumentError,在GPU上没有问题_Python_Tensorflow_Keras - Fatal编程技术网

Python 在CPU上提供模型时出现InvalidArgumentError,在GPU上没有问题

Python 在CPU上提供模型时出现InvalidArgumentError,在GPU上没有问题,python,tensorflow,keras,Python,Tensorflow,Keras,我正在尝试将我在GPU ec2实例上训练的模型导入到一个ec2实例,该实例没有用于推断的GPU,并且在调用model.predict()函数时遇到一些问题 我曾尝试将设备设置为仅使用实例上的可用CPU,但没有任何帮助,我很确定这就是问题所在,因为我已使用GPU在另一个ec2实例上重新创建了我的pipenv环境,运行了用于推断的相同代码,没有任何问题 以下是我的模型推断代码: import keras from keras import backend as K import tensorflow

我正在尝试将我在GPU ec2实例上训练的模型导入到一个ec2实例,该实例没有用于推断的GPU,并且在调用
model.predict()函数时遇到一些问题

我曾尝试将设备设置为仅使用实例上的可用CPU,但没有任何帮助,我很确定这就是问题所在,因为我已使用GPU在另一个ec2实例上重新创建了我的pipenv环境,运行了用于推断的相同代码,没有任何问题

以下是我的模型推断代码:

import keras
from keras import backend as K
import tensorflow as tf
import pandas as pd
import numpy as np

# A preprocessed pandas dataframe I read in
f_listing_data = pd.read_csv("f_listing_data.csv")

def df_to_array(df):
    """Converts a pandas dataframe to a list of numpy arrays"""
    array_list = []
    for col in df.columns:
        array_list.append(df[col].values)
    return array_list

def r2(y_true, y_pred):
    SS_res =  K.sum(K.square(y_true - y_pred)) 
    SS_tot = K.sum(K.square(y_true - K.mean(y_true))) 
    return ( 1 - SS_res/(SS_tot + K.epsilon()) )

def rmse(y_true, y_pred):
        return K.sqrt(K.mean(K.square(y_pred - y_true)))

# loading in my pretrained model
inf_model = keras.models.load_model('keras_instant_price_no_upload.h5', custom_objects={'rmse':rmse, 'r2':r2})

# dropping target label and getting predictions 
preds = inf_model.predict(df_to_array(f_listing_data.drop("price", axis=1)))
以下是我的环境设置的pipenv Pipfile的外观(我使用的是从私有Pypi服务器下载的软件包):

My Pipenv图形:

ipython==7.7.0
  - backcall [required: Any, installed: 0.1.0]
  - decorator [required: Any, installed: 4.4.0]
  - jedi [required: >=0.10, installed: 0.15.1]
    - parso [required: >=0.5.0, installed: 0.5.1]
  - pexpect [required: Any, installed: 4.7.0]
    - ptyprocess [required: >=0.5, installed: 0.6.0]
  - pickleshare [required: Any, installed: 0.7.5]
  - prompt-toolkit [required: >=2.0.0,<2.1.0, installed: 2.0.9]
    - six [required: >=1.9.0, installed: 1.12.0]
    - wcwidth [required: Any, installed: 0.1.7]
  - pygments [required: Any, installed: 2.4.2]
  - setuptools [required: >=18.5, installed: 41.2.0]
  - traitlets [required: >=4.2, installed: 4.3.2]
    - decorator [required: Any, installed: 4.4.0]
    - ipython-genutils [required: Any, installed: 0.2.0]
    - six [required: Any, installed: 1.12.0]
Keras==2.2.5
  - h5py [required: Any, installed: 2.9.0]
    - numpy [required: >=1.7, installed: 1.17.0]
    - six [required: Any, installed: 1.12.0]
  - keras-applications [required: >=1.0.8, installed: 1.0.8]
    - h5py [required: Any, installed: 2.9.0]
      - numpy [required: >=1.7, installed: 1.17.0]
      - six [required: Any, installed: 1.12.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
  - keras-preprocessing [required: >=1.1.0, installed: 1.1.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
    - six [required: >=1.9.0, installed: 1.12.0]
  - numpy [required: >=1.9.1, installed: 1.17.0]
  - pyyaml [required: Any, installed: 5.1.2]
  - scipy [required: >=0.14, installed: 1.3.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
  - six [required: >=1.9.0, installed: 1.12.0]
loftydatafetcher==0.2.3
  - boto3 [required: Any, installed: 1.9.216]
    - botocore [required: >=1.12.216,<1.13.0, installed: 1.12.216]
      - docutils [required: >=0.10,<0.16, installed: 0.15.2]
      - jmespath [required: >=0.7.1,<1.0.0, installed: 0.9.4]
      - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.0]
        - six [required: >=1.5, installed: 1.12.0]
      - urllib3 [required: >=1.20,<1.26, installed: 1.25.3]
    - jmespath [required: >=0.7.1,<1.0.0, installed: 0.9.4]
    - s3transfer [required: >=0.2.0,<0.3.0, installed: 0.2.1]
      - botocore [required: >=1.12.36,<2.0.0, installed: 1.12.216]
        - docutils [required: >=0.10,<0.16, installed: 0.15.2]
        - jmespath [required: >=0.7.1,<1.0.0, installed: 0.9.4]
        - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.0]
          - six [required: >=1.5, installed: 1.12.0]
        - urllib3 [required: >=1.20,<1.26, installed: 1.25.3]
  - mysql-connector [required: Any, installed: 2.2.9]
  - pandas [required: Any, installed: 0.25.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
    - python-dateutil [required: >=2.6.1, installed: 2.8.0]
      - six [required: >=1.5, installed: 1.12.0]
    - pytz [required: >=2017.2, installed: 2019.2]
  - sqlalchemy [required: Any, installed: 1.3.7]
scikit-learn==0.21.3
  - joblib [required: >=0.11, installed: 0.13.2]
  - numpy [required: >=1.11.0, installed: 1.17.0]
  - scipy [required: >=0.17.0, installed: 1.3.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
tensorflow==1.14.0
  - absl-py [required: >=0.7.0, installed: 0.7.1]
    - six [required: Any, installed: 1.12.0]
  - astor [required: >=0.6.0, installed: 0.8.0]
  - gast [required: >=0.2.0, installed: 0.2.2]
  - google-pasta [required: >=0.1.6, installed: 0.1.7]
  - grpcio [required: >=1.8.6, installed: 1.23.0]
    - six [required: >=1.5.2, installed: 1.12.0]
  - keras-applications [required: >=1.0.6, installed: 1.0.8]
    - h5py [required: Any, installed: 2.9.0]
      - numpy [required: >=1.7, installed: 1.17.0]
      - six [required: Any, installed: 1.12.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
  - keras-preprocessing [required: >=1.0.5, installed: 1.1.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
    - six [required: >=1.9.0, installed: 1.12.0]
  - numpy [required: >=1.14.5,<2.0, installed: 1.17.0]
  - protobuf [required: >=3.6.1, installed: 3.9.1]
    - setuptools [required: Any, installed: 41.2.0]
    - six [required: >=1.9, installed: 1.12.0]
  - six [required: >=1.10.0, installed: 1.12.0]
  - tensorboard [required: >=1.14.0,<1.15.0, installed: 1.14.0]
    - absl-py [required: >=0.4, installed: 0.7.1]
      - six [required: Any, installed: 1.12.0]
    - grpcio [required: >=1.6.3, installed: 1.23.0]
      - six [required: >=1.5.2, installed: 1.12.0]
    - markdown [required: >=2.6.8, installed: 3.1.1]
      - setuptools [required: >=36, installed: 41.2.0]
    - numpy [required: >=1.12.0, installed: 1.17.0]
    - protobuf [required: >=3.6.0, installed: 3.9.1]
      - setuptools [required: Any, installed: 41.2.0]
      - six [required: >=1.9, installed: 1.12.0]
    - setuptools [required: >=41.0.0, installed: 41.2.0]
    - six [required: >=1.10.0, installed: 1.12.0]
    - werkzeug [required: >=0.11.15, installed: 0.15.5]
    - wheel [required: >=0.26, installed: 0.33.6]
  - tensorflow-estimator [required: >=1.14.0rc0,<1.15.0rc0, installed: 1.14.0]
  - termcolor [required: >=1.1.0, installed: 1.1.0]
  - wheel [required: >=0.26, installed: 0.33.6]
  - wrapt [required: >=1.11.1, installed: 1.11.2]
xgboost==0.90
  - numpy [required: Any, installed: 1.17.0]
  - scipy [required: Any, installed: 1.3.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
这是我在CPU实例(在ipython中运行)上运行同一行代码时得到的错误堆栈跟踪:

---------------------------------------------------------------------------
InvalidArgumentError回溯(最后一次最近调用)
在里面
27
28#放弃目标标签并获得预测
--->29 preds=inf_model.predict(df_到_数组(f_listing_data.drop(“price”,axis=1)))
预测中的~/.local/share/virtualenvs/xgb\u列表\u pred-eH2Cah04/lib/python3.6/site-packages/keras/engine/training.py(self、x、批处理大小、详细程度、步骤、回调、最大队列大小、工人、使用多处理)
1399 verbose=详细,
1400步=步,
->1401回调=回调)
1402
1403 def系列在批次上(自身、x、y、,
预测循环中的~/.local/share/virtualenvs/xgb\u listing\u pred-eH2Cah04/lib/python3.6/site-packages/keras/engine/training\u arrays.py(模型、f、ins、批量大小、详细信息、步骤、回调)
330批处理日志={'batch':批处理索引,'size':len(批处理ID)}
331回调。调用批处理钩子(“预测”、“开始”、批处理索引、批处理日志)
-->332批次输出=f(批次输入)
333批处理=待处理列表(批处理)
334如果批次索引=0:
~/.local/share/virtualenvs/xgb\u listing\u pred-eH2Cah04/lib/python3.6/site-packages/keras/backend/tensorflow\u backend.py in\uuuuuuu调用(self,输入)
2977返回自我。\u传统\u调用(输入)
2978
->2979返回自调用(输入)
2980其他:
2981如果py_有(输入中x的张量为x):
~/.local/share/virtualenvs/xgb\u listing\u pred-eH2Cah04/lib/python3.6/site-packages/keras/backend/tensorflow\u backend.py in\u调用(自我,输入)
2935 fetched=self.\u callable\u fn(*array\u vals,run\u metadata=self.run\u metadata)
2936其他:
->2937 fetched=self.\u callable\u fn(*array\u vals)
2938获取的返回[:len(自输出)]
2939
~/.local/share/virtualenvs/xgb_listing_pred-eH2Cah04/lib/python3.6/site-packages/tensorflow/python/client/session.py in uuuu调用(self,*args,**kwargs)
1456 ret=tf_session.tf_SessionRunCallable(self._session._session,
1457自动控制手柄,args,
->1458运行(元数据)
1459如果运行\u元数据:
1460 proto_data=tf_session.tf_GetBuffer(run_metadata_ptr)
InvalidArgumentError:索引[0,0]=1490不在[0,38]中
[{{节点嵌入\u 22/嵌入\u查找}]]

非常感谢您提供的任何帮助/建议,感谢您抽出时间。

取消订阅Apache直升机并重试。取消订阅Apache直升机并重试。
ipython==7.7.0
  - backcall [required: Any, installed: 0.1.0]
  - decorator [required: Any, installed: 4.4.0]
  - jedi [required: >=0.10, installed: 0.15.1]
    - parso [required: >=0.5.0, installed: 0.5.1]
  - pexpect [required: Any, installed: 4.7.0]
    - ptyprocess [required: >=0.5, installed: 0.6.0]
  - pickleshare [required: Any, installed: 0.7.5]
  - prompt-toolkit [required: >=2.0.0,<2.1.0, installed: 2.0.9]
    - six [required: >=1.9.0, installed: 1.12.0]
    - wcwidth [required: Any, installed: 0.1.7]
  - pygments [required: Any, installed: 2.4.2]
  - setuptools [required: >=18.5, installed: 41.2.0]
  - traitlets [required: >=4.2, installed: 4.3.2]
    - decorator [required: Any, installed: 4.4.0]
    - ipython-genutils [required: Any, installed: 0.2.0]
    - six [required: Any, installed: 1.12.0]
Keras==2.2.5
  - h5py [required: Any, installed: 2.9.0]
    - numpy [required: >=1.7, installed: 1.17.0]
    - six [required: Any, installed: 1.12.0]
  - keras-applications [required: >=1.0.8, installed: 1.0.8]
    - h5py [required: Any, installed: 2.9.0]
      - numpy [required: >=1.7, installed: 1.17.0]
      - six [required: Any, installed: 1.12.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
  - keras-preprocessing [required: >=1.1.0, installed: 1.1.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
    - six [required: >=1.9.0, installed: 1.12.0]
  - numpy [required: >=1.9.1, installed: 1.17.0]
  - pyyaml [required: Any, installed: 5.1.2]
  - scipy [required: >=0.14, installed: 1.3.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
  - six [required: >=1.9.0, installed: 1.12.0]
loftydatafetcher==0.2.3
  - boto3 [required: Any, installed: 1.9.216]
    - botocore [required: >=1.12.216,<1.13.0, installed: 1.12.216]
      - docutils [required: >=0.10,<0.16, installed: 0.15.2]
      - jmespath [required: >=0.7.1,<1.0.0, installed: 0.9.4]
      - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.0]
        - six [required: >=1.5, installed: 1.12.0]
      - urllib3 [required: >=1.20,<1.26, installed: 1.25.3]
    - jmespath [required: >=0.7.1,<1.0.0, installed: 0.9.4]
    - s3transfer [required: >=0.2.0,<0.3.0, installed: 0.2.1]
      - botocore [required: >=1.12.36,<2.0.0, installed: 1.12.216]
        - docutils [required: >=0.10,<0.16, installed: 0.15.2]
        - jmespath [required: >=0.7.1,<1.0.0, installed: 0.9.4]
        - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.0]
          - six [required: >=1.5, installed: 1.12.0]
        - urllib3 [required: >=1.20,<1.26, installed: 1.25.3]
  - mysql-connector [required: Any, installed: 2.2.9]
  - pandas [required: Any, installed: 0.25.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
    - python-dateutil [required: >=2.6.1, installed: 2.8.0]
      - six [required: >=1.5, installed: 1.12.0]
    - pytz [required: >=2017.2, installed: 2019.2]
  - sqlalchemy [required: Any, installed: 1.3.7]
scikit-learn==0.21.3
  - joblib [required: >=0.11, installed: 0.13.2]
  - numpy [required: >=1.11.0, installed: 1.17.0]
  - scipy [required: >=0.17.0, installed: 1.3.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
tensorflow==1.14.0
  - absl-py [required: >=0.7.0, installed: 0.7.1]
    - six [required: Any, installed: 1.12.0]
  - astor [required: >=0.6.0, installed: 0.8.0]
  - gast [required: >=0.2.0, installed: 0.2.2]
  - google-pasta [required: >=0.1.6, installed: 0.1.7]
  - grpcio [required: >=1.8.6, installed: 1.23.0]
    - six [required: >=1.5.2, installed: 1.12.0]
  - keras-applications [required: >=1.0.6, installed: 1.0.8]
    - h5py [required: Any, installed: 2.9.0]
      - numpy [required: >=1.7, installed: 1.17.0]
      - six [required: Any, installed: 1.12.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
  - keras-preprocessing [required: >=1.0.5, installed: 1.1.0]
    - numpy [required: >=1.9.1, installed: 1.17.0]
    - six [required: >=1.9.0, installed: 1.12.0]
  - numpy [required: >=1.14.5,<2.0, installed: 1.17.0]
  - protobuf [required: >=3.6.1, installed: 3.9.1]
    - setuptools [required: Any, installed: 41.2.0]
    - six [required: >=1.9, installed: 1.12.0]
  - six [required: >=1.10.0, installed: 1.12.0]
  - tensorboard [required: >=1.14.0,<1.15.0, installed: 1.14.0]
    - absl-py [required: >=0.4, installed: 0.7.1]
      - six [required: Any, installed: 1.12.0]
    - grpcio [required: >=1.6.3, installed: 1.23.0]
      - six [required: >=1.5.2, installed: 1.12.0]
    - markdown [required: >=2.6.8, installed: 3.1.1]
      - setuptools [required: >=36, installed: 41.2.0]
    - numpy [required: >=1.12.0, installed: 1.17.0]
    - protobuf [required: >=3.6.0, installed: 3.9.1]
      - setuptools [required: Any, installed: 41.2.0]
      - six [required: >=1.9, installed: 1.12.0]
    - setuptools [required: >=41.0.0, installed: 41.2.0]
    - six [required: >=1.10.0, installed: 1.12.0]
    - werkzeug [required: >=0.11.15, installed: 0.15.5]
    - wheel [required: >=0.26, installed: 0.33.6]
  - tensorflow-estimator [required: >=1.14.0rc0,<1.15.0rc0, installed: 1.14.0]
  - termcolor [required: >=1.1.0, installed: 1.1.0]
  - wheel [required: >=0.26, installed: 0.33.6]
  - wrapt [required: >=1.11.1, installed: 1.11.2]
xgboost==0.90
  - numpy [required: Any, installed: 1.17.0]
  - scipy [required: Any, installed: 1.3.1]
    - numpy [required: >=1.13.3, installed: 1.17.0]
>>> preds = inf_model.predict(df_to_array(f_listing_data.drop("price", axis=1)))
>>> preds[:5], len(preds)

(array([[0.18931788],
        [0.08384943],
        [0.48033524],
        [0.19470096],
        [0.16680539]], dtype=float32), 340)
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-3-6559eecde46a> in <module>
     27 
     28 # dropping target label and getting predictions
---> 29 preds = inf_model.predict(df_to_array(f_listing_data.drop("price", axis=1)))

~/.local/share/virtualenvs/xgb_listing_pred-eH2Cah04/lib/python3.6/site-packages/keras/engine/training.py in predict(self, x, batch_size, verbose, steps, callbacks, max_queue_size, workers, use_multiprocessing)
   1399                                             verbose=verbose,
   1400                                             steps=steps,
-> 1401                                             callbacks=callbacks)
   1402 
   1403     def train_on_batch(self, x, y,

~/.local/share/virtualenvs/xgb_listing_pred-eH2Cah04/lib/python3.6/site-packages/keras/engine/training_arrays.py in predict_loop(model, f, ins, batch_size, verbose, steps, callbacks)
    330             batch_logs = {'batch': batch_index, 'size': len(batch_ids)}
    331             callbacks._call_batch_hook('predict', 'begin', batch_index, batch_logs)
--> 332             batch_outs = f(ins_batch)
    333             batch_outs = to_list(batch_outs)
    334             if batch_index == 0:

~/.local/share/virtualenvs/xgb_listing_pred-eH2Cah04/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
   2977                     return self._legacy_call(inputs)
   2978 
-> 2979             return self._call(inputs)
   2980         else:
   2981             if py_any(is_tensor(x) for x in inputs):

~/.local/share/virtualenvs/xgb_listing_pred-eH2Cah04/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in _call(self, inputs)
   2935             fetched = self._callable_fn(*array_vals, run_metadata=self.run_metadata)
   2936         else:
-> 2937             fetched = self._callable_fn(*array_vals)
   2938         return fetched[:len(self.outputs)]
   2939 

~/.local/share/virtualenvs/xgb_listing_pred-eH2Cah04/lib/python3.6/site-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1456         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1457                                                self._handle, args,
-> 1458                                                run_metadata_ptr)
   1459         if run_metadata:
   1460           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

InvalidArgumentError: indices[0,0] = 1490 is not in [0, 38)
         [[{{node embedding_22/embedding_lookup}}]]