Python 3.x 从文件路径导入模块的速度非常慢有什么解决方案吗?
我有一个模块列表,这些模块应该以动态方式自动导入。 以下是我的代码片段:Python 3.x 从文件路径导入模块的速度非常慢有什么解决方案吗?,python-3.x,reflection,python-importlib,Python 3.x,Reflection,Python Importlib,我有一个模块列表,这些模块应该以动态方式自动导入。 以下是我的代码片段: for m in modules_to_import: module_name = dirname(__file__)+ "/" +m spec = importlib.util.spec_from_file_location("package", module_name) imported_module = importlib.util.module_from_spec(spec) spe
for m in modules_to_import:
module_name = dirname(__file__)+ "/" +m
spec = importlib.util.spec_from_file_location("package", module_name)
imported_module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(imported_module)
我测量了时间,每次导入后它都会变得越来越慢。有什么办法解决这个问题吗?或者为什么速度会变慢?非常感谢 我没有计时,但是为什么不简化代码呢。查看代码,您希望导入与该文件位于同一目录中的模块。默认情况下,当您导入模块时,该模块是它要查找的第一个位置 首先,让我们创建一些文件,以便在同一目录中导入所有文件: First.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
Second.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
Third.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
因此,一种方法是将模块放入dict中,以便以后使用。我在这里用一个口述理解来构建这个口述:
解决方案1.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
或者,如果您确实想使用虚线符号访问模块的内容,您可以使用来帮助:
解决方案2.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
我没有计时,但是为什么不简化代码呢。查看代码,您希望导入与该文件位于同一目录中的模块。默认情况下,当您导入模块时,该模块是它要查找的第一个位置 首先,让我们创建一些文件,以便在同一目录中导入所有文件: First.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
Second.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
Third.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
因此,一种方法是将模块放入dict中,以便以后使用。我在这里用一个口述理解来构建这个口述:
解决方案1.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
或者,如果您确实想使用虚线符号访问模块的内容,您可以使用来帮助:
解决方案2.py
def display_first():
print("I'm first")
def display_second():
print("I'm second")
def display_third():
print("I'm third")
import importlib
modules_to_import = ["First", "Second", "Third"]
modules_imported = {x: importlib.import_module(x) for x in modules_to_import}
modules_imported["First"].display_first()
modules_imported["Second"].display_second()
modules_imported["Third"].display_third()
import importlib
import collections
modules_to_import = ["First", "Second", "Third"]
modules_imported = collections.namedtuple("imported_modules", modules_to_import)
for next_module in modules_to_import:
setattr(modules_imported, next_module, importlib.import_module(next_module))
modules_imported.First.display_first()
modules_imported.Second.display_second()
modules_imported.Third.display_third()
我的问题是,每次调用importlib.import_模块(x)后,速度都会变慢。实际上什么都试过了。现在的解决方案是非常丑陋的,包括打开一个子流程并使用该模块进行操作。似乎是importlib中的错误/问题。您需要导入多少个模块?这些模块是否有可能导入其他模块,这将花费更多的时间?我的问题是,每次调用importlib.import_模块(x)后,它都会变慢。实际上什么都试过了。现在的解决方案是非常丑陋的,包括打开一个子流程并使用该模块进行操作。似乎是importlib中的错误/问题。您需要导入多少个模块?这些模块是否有可能导入其他模块,这将花费更多的时间?你是如何计算时间的?你找到解决这个问题的方法了吗?我有大约20个模块要导入,在我的系统和Google Colab上大约需要1小时!不知怎么的,这是一个问题。又不知道怎么了。但是1h肯定太长了。我有100多个模块,用了不到一个小时。嗯,有趣的是,我在一个月前做了同样的程序,而且速度很快。但是现在它太慢了。毫无疑问,您为什么使用
importlib.import\u模块(项)
而不是脚本?有什么不同吗?你找到解决这个问题的办法了吗?我有大约20个模块要导入,在我的系统和Google Colab上大约需要1小时!不知怎么的,这是一个问题。又不知道怎么了。但是1h肯定太长了。我有100多个模块,用了不到一个小时。嗯,有趣的是,我在一个月前做了同样的程序,而且速度很快。但是现在它太慢了。毫无疑问,您为什么使用importlib.import\u模块(项)
而不是脚本?有什么区别吗?