Python 无法重建错误:对象丢失(用户收回或删除LRU),无法重建

Python 无法重建错误:对象丢失(用户收回或删除LRU),无法重建,python,python-3.x,multiprocessing,ray,Python,Python 3.x,Multiprocessing,Ray,我的代码如下所示: # Features' construction - Multiprocessing # import pandas as pd import time import ray import multiprocessing ray.shutdown() num_cpus = multiprocessing.cpu_count() print(num_cpus) print() ray.init(num_cpus=num_cpus) start = time.time(

我的代码如下所示:

# Features' construction - Multiprocessing #
import pandas as pd
import time
import ray
import multiprocessing


ray.shutdown()

num_cpus = multiprocessing.cpu_count()
print(num_cpus)
print()
ray.init(num_cpus=num_cpus)


start = time.time()


@ray.remote
def features_construct(index, row):

    dict_features ={}

    ...

    return(dict_features)


data = []
for index, row in enumerate(raw_data):

    data.append(features_construct.remote(index, row))


df_data = pd.DataFrame.from_records(ray.get(data))

end = time.time()
print(round(end - start, 2), 'seconds')
但是,我收到以下错误:

2019-10-21 11:30:06,751 WARNING worker.py:416 -- Local object store memory usage:
num clients with quota: 0
quota map size: 0
pinned quota map size: 0
allocated bytes: 5933914928
allocation limit: 7055781888
pinned bytes: 0
(global lru) capacity: 7055781888
(global lru) used: 84.1%
(global lru) num objects: 375838
(global lru) num evictions: 504699
(global lru) bytes evicted: 8467163584

---------------------------------------------------------------------------
UnreconstructableError                    Traceback (most recent call last)
<ipython-input-27-0f27da969dd5> in <module>
    155 print()
    156 
--> 157 df_data = pd.DataFrame.from_records(ray.get(data))
    158 
    159 

/storage2/user/anaconda/lib/python3.7/site-packages/ray/worker.py in get(object_ids)
   2347                 if isinstance(value, ray.exceptions.UnreconstructableError):
   2348                     worker.dump_object_store_memory_usage()
-> 2349                 raise value
   2350 
   2351         # Run post processors.

UnreconstructableError: Object 54158c91583effffffff0100000000c001000000 is lost (either LRU evicted or deleted by user) and cannot be reconstructed. Try increasing the object store memory available with ray.init(object_store_memory=<bytes>) or setting object store limits with ray.remote(object_store_memory=<bytes>). See also: https://ray.readthedocs.io/en/latest/memory-management.html

32
作为输出。

您是否找到了导致此
不可恢复错误的原因?@mimoralea,老实说,我现在记不起来了:)谢谢您的跟进。出于追踪的目的,我在这里想办法:@mimoralea,当然,谢谢;)
num_cpus = multiprocessing.cpu_count()
print(num_cpus)