Python 如何在远程Linux机器上设置MLFlow工件sftp存储?
我的目标是将ML流配置为在远程Linux服务器上运行,日志存储在PostgreSQL数据库中,工件存储在/home/aw/mlfow/mllogs中,其中“aw”是具有root权限的用户名 当我运行简单的python代码时(请参阅下面的部分):Python 如何在远程Linux机器上设置MLFlow工件sftp存储?,python,mlflow,Python,Mlflow,我的目标是将ML流配置为在远程Linux服务器上运行,日志存储在PostgreSQL数据库中,工件存储在/home/aw/mlfow/mllogs中,其中“aw”是具有root权限的用户名 当我运行简单的python代码时(请参阅下面的部分): 当最后一行(“mlflow.log_artifact(“features.txt”))被注释掉时:Python中没有错误,但是当我通过浏览器输入日志详细信息时,在artifact部分我得到警告:无法列出存储的工件undersftp://mlflow_us
无法列出存储的工件undersftp://mlflow_user@194.39.141.27:~/mlflow/mlruns/0/d1eb9ce83b6b4ede96a9ea5203c097da/artifacts
ValueError:Port无法转换为整数值为“~”
我尝试了从CLI以多种不同的方式运行服务器,每次都更改--default artifact root参数(请参见下面的部分)。有趣的是,Python代码中mlflow.get_artifact_uri()变量的打印输出保持不变,尽管CLI服务器参数发生了更改:它始终显示为
sftp://mlflow_user@94.39.141.27:~/mlflow/mlruns/0/c613c110839946a3adc198377cc82c0c/工件
mlflow服务器——后端存储uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db——默认工件根sftp://mlflow_user@194.39.141.27:~/mlflow/mlruns-h0.0.0-p8000&
选项2.1和2.2(aw是我在计算机上的用户名,具有root权限):
mlflow服务器——后端存储uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db——默认工件根sftp://aw:aw_pass@194.39.141.27:~/mlflow/mlruns-h0.0.0-p8000&
mlflow服务器——后端存储uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db——默认工件根sftp://aw:@194.39.141.27:~/home/aw/mlflow/mlruns-h0.0.0-p8000&
备选案文3:
mlflow服务器——后端存储uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db——默认工件根sftp://mlflow_user:mlflow_pass#@194.39.141.27:~/mlflow/mlruns-h0.0.0-p8000&
Python代码:
import mlflow
if __name__ == "__main__":
mlflow.set_tracking_uri("http://194.39.141.27:8000") #hostname IP here is just an example
features = "rooms, zipcode, median_price, school_rating, transport"
with open("features.txt", 'w') as f:
f.write(features)
with mlflow.start_run():
tracking_uri = mlflow.get_tracking_uri()
artifact_uri = mlflow.get_artifact_uri()
print("Tracking uri: {}".format(tracking_uri))
print("Artifact uri: {}".format(artifact_uri))
mlflow.log_artifact("features.txt")
其他信息/日志
Python编译器日志:
File "1.py", line 18, in
mlflow.log_artifact("features.txt")
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/fluent.py", line 544, in log_artifact
MlflowClient().log_artifact(run_id, local_path, artifact_path)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/client.py", line 903, in log_artifact
self._tracking_client.log_artifact(run_id, local_path, artifact_path)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 271, in log_artifact
artifact_repo = self._get_artifact_repo(run_id)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 262, in _get_artifact_repo
return get_artifact_repository(artifact_uri)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 102, in get_artifact_repository
return _artifact_repository_registry.get_artifact_repository(artifact_uri)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 71, in get_artifact_repository
return repository(artifact_uri)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/store/artifact/sftp_artifact_repo.py", line 32, in init
"port": parsed.port,
File "/home/aw/anaconda3/lib/python3.8/urllib/parse.py", line 174, in port
raise ValueError(message) from None
ValueError: Port could not be cast to integer value as '~'
总而言之:我的主要目标是在远程Linux服务器上存储MLFLow工件。我非常感谢您对如何实现这一点的帮助