Python 获取干净的数据：漂亮的汤就足够了，还是我也必须使用正则表达式？_Python_Regex_Beautifulsoup_Data Cleaning

Python 获取干净的数据：漂亮的汤就足够了，还是我也必须使用正则表达式？

python regex

Python 获取干净的数据：漂亮的汤就足够了，还是我也必须使用正则表达式？,python,regex,beautifulsoup,data-cleaning,Python,Regex,Beautifulsoup,Data Cleaning,我正在用Python学习美丽的汤和字典。下面是斯坦福大学《美丽的汤》的简短教程：由于禁止访问网站，我将教程中的文本存储为字符串，然后将字符串soup转换为soup对象。打印输出如下所示： print(soup_string) <html><body><div class="ec_statements"><div id="legalert_title"><a href="/Legislation-and-Po

我正在用Python学习美丽的汤和字典。下面是斯坦福大学《美丽的汤》的简短教程：

由于禁止访问网站，我将教程中的文本存储为字符串，然后将字符串soup转换为soup对象。打印输出如下所示：

    print(soup_string)

    <html><body><div class="ec_statements"><div id="legalert_title"><a    
    href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-Senators-
    Urging-Them-to-Support-Cloture-and-Final-Passage-of-the-Paycheck-
    Fairness-Act-S.2199">'Letter to Senators Urging Them to Support Cloture     
    and Final Passage of the Paycheck Fairness Act (S.2199)
    </a>
    </div>
    <div id="legalert_date">
    September 10, 2014
    </div>
    </div>
    <div class="ec_statements">
    <div id="legalert_title">
    <a href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-  
    Representatives-Urging-Them-to-Vote-on-the-Highway-Trust-Fund-Bill">
    Letter to Representatives Urging Them to Vote on the Highway Trust Fund Bill
    </a>
    </div>
    <div id="legalert_date">
            July 30, 2014
           </div>
    </div>
    <div class="ec_statements">
    <div id="legalert_title">
    <a href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-Representatives-Urging-Them-to-Vote-No-on-the-Legislation-Providing-Supplemental-Appropriations-for-the-Fiscal-Year-Ending-Sept.-30-2014">
             Letter to Representatives Urging Them to Vote No on the Legislation Providing Supplemental Appropriations for the Fiscal Year Ending Sept. 30, 2014
            </a>
    </div>
    <div id="legalert_date">
            July 30, 2014
           </div>
    </div>
 </body></html>

lobbying_1 = []
lobbying_2 = []
lobbying_3 = []
for element in letters:
    lobbying_1.append(element.a.get_text())
    lobbying_2.append(element.a.attrs.get('href'))
    lobbying_3.append(element.find(id="legalert_date").get_text())
df =pd.DataFrame([])
df = pd.DataFrame(lobbying_1, columns = ['Name'] )
df['href'] = lobbying_2
df['Date'] = lobbying_3

print(df)

                                                Name  \
0  \n        'Letter to Senators Urging Them to S...   
1  \n         Letter to Representatives Urging Th...   
2  \n         Letter to Representatives Urging Th...   

                                                href  \
0  /Legislation-and-Politics/Legislative-Alerts/L...   
1  /Legislation-and-Politics/Legislative-Alerts/L...   
2  /Legislation-and-Politics/Legislative-Alerts/L...   

                                    Date  
0  \n        September 10, 2014\n         
1       \n        July 30, 2014\n         
2       \n        July 30, 2014\n

然后导师说：

“我们将检查信件集合中的所有项目，对于每个项目，提取名称并将其作为dict中的键。该值将是另一个dict，但我们尚未找到其他项目的内容，因此我们将创建并分配一个空dict对象。”

在这一点上，我采取了一种不同的方法，我决定先将数据存储在列表中，然后再存储在数据帧中。代码如下：

    print(soup_string)

    <html><body><div class="ec_statements"><div id="legalert_title"><a    
    href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-Senators-
    Urging-Them-to-Support-Cloture-and-Final-Passage-of-the-Paycheck-
    Fairness-Act-S.2199">'Letter to Senators Urging Them to Support Cloture     
    and Final Passage of the Paycheck Fairness Act (S.2199)
    </a>
    </div>
    <div id="legalert_date">
    September 10, 2014
    </div>
    </div>
    <div class="ec_statements">
    <div id="legalert_title">
    <a href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-  
    Representatives-Urging-Them-to-Vote-on-the-Highway-Trust-Fund-Bill">
    Letter to Representatives Urging Them to Vote on the Highway Trust Fund Bill
    </a>
    </div>
    <div id="legalert_date">
            July 30, 2014
           </div>
    </div>
    <div class="ec_statements">
    <div id="legalert_title">
    <a href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-Representatives-Urging-Them-to-Vote-No-on-the-Legislation-Providing-Supplemental-Appropriations-for-the-Fiscal-Year-Ending-Sept.-30-2014">
             Letter to Representatives Urging Them to Vote No on the Legislation Providing Supplemental Appropriations for the Fiscal Year Ending Sept. 30, 2014
            </a>
    </div>
    <div id="legalert_date">
            July 30, 2014
           </div>
    </div>
 </body></html>

lobbying_1 = []
lobbying_2 = []
lobbying_3 = []
for element in letters:
    lobbying_1.append(element.a.get_text())
    lobbying_2.append(element.a.attrs.get('href'))
    lobbying_3.append(element.find(id="legalert_date").get_text())
df =pd.DataFrame([])
df = pd.DataFrame(lobbying_1, columns = ['Name'] )
df['href'] = lobbying_2
df['Date'] = lobbying_3

print(df)

                                                Name  \
0  \n        'Letter to Senators Urging Them to S...   
1  \n         Letter to Representatives Urging Th...   
2  \n         Letter to Representatives Urging Th...   

                                                href  \
0  /Legislation-and-Politics/Legislative-Alerts/L...   
1  /Legislation-and-Politics/Legislative-Alerts/L...   
2  /Legislation-and-Politics/Legislative-Alerts/L...   

                                    Date  
0  \n        September 10, 2014\n         
1       \n        July 30, 2014\n         
2       \n        July 30, 2014\n

输出如下：

    print(soup_string)

    <html><body><div class="ec_statements"><div id="legalert_title"><a    
    href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-Senators-
    Urging-Them-to-Support-Cloture-and-Final-Passage-of-the-Paycheck-
    Fairness-Act-S.2199">'Letter to Senators Urging Them to Support Cloture     
    and Final Passage of the Paycheck Fairness Act (S.2199)
    </a>
    </div>
    <div id="legalert_date">
    September 10, 2014
    </div>
    </div>
    <div class="ec_statements">
    <div id="legalert_title">
    <a href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-  
    Representatives-Urging-Them-to-Vote-on-the-Highway-Trust-Fund-Bill">
    Letter to Representatives Urging Them to Vote on the Highway Trust Fund Bill
    </a>
    </div>
    <div id="legalert_date">
            July 30, 2014
           </div>
    </div>
    <div class="ec_statements">
    <div id="legalert_title">
    <a href="/Legislation-and-Politics/Legislative-Alerts/Letter-to-Representatives-Urging-Them-to-Vote-No-on-the-Legislation-Providing-Supplemental-Appropriations-for-the-Fiscal-Year-Ending-Sept.-30-2014">
             Letter to Representatives Urging Them to Vote No on the Legislation Providing Supplemental Appropriations for the Fiscal Year Ending Sept. 30, 2014
            </a>
    </div>
    <div id="legalert_date">
            July 30, 2014
           </div>
    </div>
 </body></html>

lobbying_1 = []
lobbying_2 = []
lobbying_3 = []
for element in letters:
    lobbying_1.append(element.a.get_text())
    lobbying_2.append(element.a.attrs.get('href'))
    lobbying_3.append(element.find(id="legalert_date").get_text())
df =pd.DataFrame([])
df = pd.DataFrame(lobbying_1, columns = ['Name'] )
df['href'] = lobbying_2
df['Date'] = lobbying_3

print(df)

                                                Name  \
0  \n        'Letter to Senators Urging Them to S...   
1  \n         Letter to Representatives Urging Th...   
2  \n         Letter to Representatives Urging Th...   

                                                href  \
0  /Legislation-and-Politics/Legislative-Alerts/L...   
1  /Legislation-and-Politics/Legislative-Alerts/L...   
2  /Legislation-and-Politics/Legislative-Alerts/L...   

                                    Date  
0  \n        September 10, 2014\n         
1       \n        July 30, 2014\n         
2       \n        July 30, 2014\n

我的问题是：是否有一种方法可以获得更干净的数据，即不带\n和空格的字符串，只通过Beautiful Soup获得真实值？或者我必须使用正则表达式对数据进行后期处理

谢谢你的建议

要删除文本中的换行符，请在调用

get_text（）

时传递

strip=True

：

当然，这是假设您仍然希望数据采用

数据帧的形式