Python 如何在光线中从对象存储中清除对象？
我正在试用有前途的多处理软件包Python 如何在光线中从对象存储中清除对象？,python,python-3.x,windows,ray,Python,Python 3.x,Windows,Ray,我正在试用有前途的多处理软件包ray。我有一个我似乎无法解决的问题。我的程序第一次运行正常，但第二次运行时，ray.put（）行出现此异常： ObjectStoreFullError: Failed to put object ffffffffffffffffffffffffffffffffffffffff010000000c000000 in object store because it is full. Object size is 2151680255 bytes. The local
rayray.put（）ObjectStoreFullError: Failed to put object ffffffffffffffffffffffffffffffffffffffff010000000c000000 in object store because it is full. Object size is 2151680255 bytes.
The local object store is full of objects that are still in scope and cannot be evicted. Tip: Use the `ray memory` command to list active objects in the cluster.
大数据对象。我想一次在内存中保存一个大数据对象
，并对大数据对象进行几次繁重（独立）计算。我想并行执行这些计算。完成这些操作后，我必须将对象存储中的这些大数据对象
替换为新对象，然后再次开始计算（并行）
使用我的测试脚本，我通过再次启动脚本而不使用ray.shutdown（）。如果我使用ray.shutdown（）
关闭ray
，则对象存储将被清除，但重新初始化需要很长时间，并且我无法按我想要的顺序处理多个大数据对象
我研究了哪些信息来源：
我研究了这个文档，并研究了“反模式：大型/不可分解对象的闭包捕获”一节，以及如何创建正确的模式。我还学习了入门指南，该指南产生了以下测试脚本
重现问题的最低示例：
我创建了一个测试脚本来测试这个。这是：
#%% Imports
import ray
import time
import psutil
import numpy as np
#%% Testing ray
# Start Ray
num_cpus = psutil.cpu_count(logical=False)
if not ray.is_initialized():
ray.init(num_cpus=num_cpus, include_dashboard=False)
# Define function to do work in parallel
@ray.remote
def my_function(x): # Later I will have multiple (different) my_functions to extract different feature from my big_data_object
time.sleep(1)
data_item = ray.get(big_data_object_ref)
return data_item[0,0]+x
# Define large data
big_data_object = np.random.rand(16400,16400) # Define an object of approx 2 GB. Works on my machine (16 GB RAM)
# big_data_object = np.random.rand(11600,11600) # Define an object of approx 1 GB.
# big_data_object = np.random.rand(8100,8100) # Define an object of approx 500 MB.
# big_data_object = np.random.rand(5000,5000) # Define an object of approx 190 MB.
big_data_object_ref = ray.put(big_data_object)
# Start 4 tasks in parallel.
result_refs = []
# for item in data:
for item in range(4):
result_refs.append(my_function.remote(item))
# Wait for the tasks to complete and retrieve the results.
# With at least 4 cores, this will take 1 second.
results = ray.get(result_refs)
print("Results: {}".format(results))
#%% Clean-up object store data - Still their is a (huge) memory leak in the object store.
for index in range(4):
del result_refs[0]
del big_data_object_ref
我认为哪里出了问题：
我想我删除了脚本末尾对对象存储的所有引用。因此，应将对象从对象存储中清除（如上所述）。显然，有些地方出了问题，因为大数据对象仍保留在对象存储中。但是，结果
从对象存储中删除很好
一些调试信息：
我使用ray memory
命令检查了对象存储，结果如下：
(c:\python\cenv38rl) PS C:\WINDOWS\system32> ray memory
---------------------------------------------------------------------------------------------------------------------
Object ID Reference Type Object Size Reference Creation Site
=====================================================================================================================
; worker pid=20952
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\serialization.py:object_ref_deserializer:45 | c:\python\cenv38rl\lib\site-packages\ray\function_manager.py:fetch_and_register_remote_function:180 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_process_key:140 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_run:87
; worker pid=29368
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\serialization.py:object_ref_deserializer:45 | c:\python\cenv38rl\lib\site-packages\ray\function_manager.py:fetch_and_register_remote_function:180 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_process_key:140 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_run:87
; worker pid=17388
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\serialization.py:object_ref_deserializer:45 | c:\python\cenv38rl\lib\site-packages\ray\function_manager.py:fetch_and_register_remote_function:180 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_process_key:140 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_run:87
; worker pid=24208
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\serialization.py:object_ref_deserializer:45 | c:\python\cenv38rl\lib\site-packages\ray\function_manager.py:fetch_and_register_remote_function:180 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_process_key:140 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_run:87
; worker pid=27684
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\serialization.py:object_ref_deserializer:45 | c:\python\cenv38rl\lib\site-packages\ray\function_manager.py:fetch_and_register_remote_function:180 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_process_key:140 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_run:87
; worker pid=6860
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\serialization.py:object_ref_deserializer:45 | c:\python\cenv38rl\lib\site-packages\ray\function_manager.py:fetch_and_register_remote_function:180 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_process_key:140 | c:\python\cenv38rl\lib\site-packages\ray\import_thread.py:_run:87
; driver pid=28684
ffffffffffffffffffffffffffffffffffffffff010000000b000000 LOCAL_REFERENCE 2151680261 c:\python\cenv38rl\lib\site-packages\ray\worker.py:put_object:277 | c:\python\cenv38rl\lib\site-packages\ray\worker.py:put:1489 | c:\python\cenv38rl\lib\site-packages\ray\_private\client_mode_hook.py:wrapper:47 | C:\Users\Stefan\Documents\Python examples\Multiprocess_Ray3_SO.py:<module>:42
---------------------------------------------------------------------------------------------------------------------
--- Aggregate object store stats across all nodes ---
Plasma memory usage 2052 MiB, 1 objects, 77.41% full
然后脚本成功地清除了对象存储，但是my_函数
无法使用我需要的big_data_对象
我的问题是：如何修复我的代码，以便在脚本结束时从对象存储中删除大数据对象
，而不关闭ray？
注意：我使用pip install ray
安装了ray，它给了我现在使用的版本ray==1.2.0
。我在Windows
上使用ray，并在Spyder v4.2.5
中开发conda
（实际上是miniconda）环境，以防相关
编辑：
我也在一台有8GB内存的Ubuntu机器上进行了测试。为此，我使用了1GB的大数据对象。
我可以确认该问题也发生在这台机器上
光线存储器
输出：
(SO_ray) stefan@stefan-HP-ZBook-15:~/Documents/Ray_test_scripts$ ray memory
---------------------------------------------------------------------------------------------------------------------
Object ID Reference Type Object Size Reference Creation Site
=====================================================================================================================
; worker pid=18593
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
; worker pid=18591
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
; worker pid=18590
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
; driver pid=17712
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 (put object) | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/_private/client_mode_hook.py:wrapper:47 | /home/stefan/Documents/Ray_test_scripts/Multiprocess_Ray3_SO.py:<module>:43 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/spyder_kernels/customize/spydercustomize.py:exec_code:453
; worker pid=18592
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
---------------------------------------------------------------------------------------------------------------------
--- Aggregate object store stats across all nodes ---
Plasma memory usage 1026 MiB, 1 objects, 99.69% full
（sou-ray）stefan@stefan-HP-ZBook-15:~/Documents/Ray\u test\u scripts$Ray内存
---------------------------------------------------------------------------------------------------------------------
对象ID引用类型对象大小引用创建站点
=====================================================================================================================
; 工人pid=18593
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0100000001000000本地\u参考1076480259/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/function\u-manager.py:fetch_和_-register\u远程\u功能：180 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import\u-thread.py:|:/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import_-thread.py:_-run:87 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/threading.py:run:870
; 工人pid=18591
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0100000001000000本地\u参考1076480259/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/function\u-manager.py:fetch_和_-register\u远程\u功能：180 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import\u-thread.py:|:/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import_-thread.py:_-run:87 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/threading.py:run:870
; 工人pid=18590
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0100000001000000本地\u参考1076480259/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/function\u-manager.py:fetch_和_-register\u远程\u功能：180 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import\u-thread.py:|:/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import_-thread.py:_-run:87 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/threading.py:run:870
; 驱动器pid=17712
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0100000001000000本地\u参考1076480259（put对象）|/home/stefan/miniconda3/envs/SO|u-ray/lib/python3.8/site-packages/ray/private/client_-mode_-hook.py:wrapper:47 |/home/stefan/Documents/ray|u-test_-scripts/multi-process|()\u-Ray3.py:：43 |/home/stefan/miniconda3/envs/SO|-ray/lib/python3.8/site-packages/spyder/spyder/u-kernels/spyder/sp
; 工人pid=18592
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0100000001000000本地\u参考1076480259/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/function\u-manager.py:fetch_和_-register\u远程\u功能：180 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import\u-thread.py:|:/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/site-packages/ray/import_-thread.py:_-run:87 |/home/stefan/miniconda3/envs/SO_-ray/lib/python3.8/threading.py:run:870
---------------------------------------------------------------------------------------------------------------------
---阿格雷格
(SO_ray) stefan@stefan-HP-ZBook-15:~/Documents/Ray_test_scripts$ ray memory
---------------------------------------------------------------------------------------------------------------------
Object ID Reference Type Object Size Reference Creation Site
=====================================================================================================================
; worker pid=18593
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
; worker pid=18591
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
; worker pid=18590
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
; driver pid=17712
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 (put object) | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/_private/client_mode_hook.py:wrapper:47 | /home/stefan/Documents/Ray_test_scripts/Multiprocess_Ray3_SO.py:<module>:43 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/spyder_kernels/customize/spydercustomize.py:exec_code:453
; worker pid=18592
ffffffffffffffffffffffffffffffffffffffff0100000001000000 LOCAL_REFERENCE 1076480259 /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/function_manager.py:fetch_and_register_remote_function:180 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_process_key:140 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/site-packages/ray/import_thread.py:_run:87 | /home/stefan/miniconda3/envs/SO_ray/lib/python3.8/threading.py:run:870
---------------------------------------------------------------------------------------------------------------------
--- Aggregate object store stats across all nodes ---
Plasma memory usage 1026 MiB, 1 objects, 99.69% full
# Remote function
@ray.remote
def my_function(big_data_object_ref_list, x):
time.sleep(1)
big_data_object = ray.get(big_data_object_ref_list[0])
return big_data_object[0,0]+x
# Calling the remote function
my_function.remote([big_data_object_ref], item)
#%% Testing ray
# ... my ray testing code
#%% Clean-up object store data
print("Wait 10 sec BEFORE deletion")
time.sleep(10) # Now quickly use the 'ray memory' command to inspect the contents of the object store.
del result_refs
del big_data_object_ref
print("Wait 10 sec AFTER deletion")
time.sleep(10) # Now again use the 'ray memory' command to inspect the contents of the object store.