Python 运行多处理代码时,数据帧不会更新

Python 运行多处理代码时,数据帧不会更新,python,pandas,dataframe,multiprocessing,screen-scraping,Python,Pandas,Dataframe,Multiprocessing,Screen Scraping,我试图用下面的类用temp-df(self.df_temp['linkedin_profile'])中的一列更新数据帧(self.df),但它似乎没有更新任何内容。 守则: class NameToSocialURLScraper: def __init__(self, csv_file_name, person_name_column, organization_name_column): self.proxy_list = PROXY_LIST pool = Pool()

我试图用下面的类用temp-df(self.df_temp['linkedin_profile'])中的一列更新数据帧(self.df),但它似乎没有更新任何内容。 守则:

class NameToSocialURLScraper:


def __init__(self, csv_file_name, person_name_column, organization_name_column):
    self.proxy_list = PROXY_LIST
    pool = Pool()
    self.csv_file_name = csv_file_name
    self.person_name_column = person_name_column
    self.organization_name_column = organization_name_column
    self.df = pd.read_csv(csv_file_name)
    self.df_temp = pd.DataFrame()

def internal_linkedin_job(self):
    self.df['linkedin_profile'] = np.nan
    self.df_temp['linkedin_profile'] = np.nan
    self.df_temp['linkedin_profile'] = self.df.apply(
        lambda row: term_scraper(
            str(row[self.person_name_column]) + " " + str(row[self.organization_name_column]), self.proxy_list,
            'link', output_generic=False), axis=1)
    self.df['linkedin_profile'] = self.df_temp['linkedin_profile']
    print(self.df.values)

...

def multiprocess_job(self):
    multiprocessing.log_to_stderr(logging.DEBUG)
    linkedin_profile_proc = Process(target=self.internal_linkedin_job, args=())
    jobs = [linkedin_profile_proc]
    # Start the processes (i.e. calculate the random number lists)
    for j in jobs:
        j.start()

    # Ensure all of the processes have finished
    for j in jobs:
        j.join()

在内部linkedin作业中打印时,它会显示带有新列“linkedin\u profile”的df,但在j.join()之后打印时,该列不存在。

在执行多处理时,每个进程都在其自己的内存空间中运行。您需要重构代码,以便内部linkedin作业返回数据帧。

明白了!借助以下答案,使用Queue解决了此问题: