Python BeautifulSoup-从开始关键字到结束关键字刮取文本

我有下面的HTML源代码，需要刮。数据包括未配置

标记的表。我不能使用标记来标识元素，因为整个HTML代码中都使用相同的标记。如何刮取数据以接收以下输出？标题行始终保持不变，表中的数据会有所不同

代码

<p>
    <div class="my-test-class" style="white-space: pre-wrap; font-size: small; font-family: &quot;Courier New&quot;">
        <div class="my-test-class">Random text goes on.........</div>
        <div class="my-test-class"><br></div>
        <div class="my-test-class">Header1 Header2 Header3 Header4 Header5
        </div>
        <div class="my-test-class">--------------------------------------
        </div>
        <div class="my-test-class">A1 B1 C1 D1 E1</div>
        <div class="my-test-class">A2 B2 C2 D2 E2</div>
        <div class="my-test-class">A3 B3 C3 D3 E3</div>
        <div class="my-test-class">--------------------------------------
        </div>
    </div>
</p>

到目前为止，正在删除代码：

from bs4 import BeautifulSoup
import re
soup = BeautifulSoup('''\
<p>
    <div class="my-test-class" style="white-space: pre-wrap; font-size: small; font-family: &quot;Courier New&quot;">
        <div class="my-test-class">Random text goes on.........</div>
        <div class="my-test-class"><br></div>
        <div class="my-test-class">Header1 Header2 Header3 Header4 Header5
        </div>
        <div class="my-test-class">--------------------------------------
        </div>
        <div class="my-test-class">A1 B1 C1 D1 E1</div>
        <div class="my-test-class">A2 B2 C2 D2 E2</div>
        <div class="my-test-class">A3 B3 C3 D3 E3</div>
        <div class="my-test-class">--------------------------------------
        </div>
    </div>
</p>
''')

h = soup.find_all(text=re.compile('Header1*'))
print(h)

从bs4导入美化组
进口稀土
汤=美汤（“”）\

随机文本继续。。。。。。。。。


校长1校长2校长3校长4校长5
--------------------------------------
A1 B1 C1 D1 E1
A2 B2 C2 D2 E2
A3 B3 C3 D3 E3
--------------------------------------

''')
h=soup.find_all（text=re.compile（'Header1*'））
打印（h）

这将帮助您：

from bs4 import BeautifulSoup
import re
soup = BeautifulSoup('''\
<p>
    <div class="my-test-class" style="white-space: pre-wrap; font-size: small; font-family: &quot;Courier New&quot;">
        <div class="my-test-class">Random text goes on.........</div>
        <div class="my-test-class"><br></div>
        <div class="my-test-class">Header1 Header2 Header3 Header4 Header5
        </div>
        <div class="my-test-class">--------------------------------------
        </div>
        <div class="my-test-class">A1 B1 C1 D1 E1</div>
        <div class="my-test-class">A2 B2 C2 D2 E2</div>
        <div class="my-test-class">A3 B3 C3 D3 E3</div>
        <div class="my-test-class">--------------------------------------
        </div>
    </div>
</p>
''', 'html5lib')

txt = soup.find_all('div', class_ = "my-test-class", text=True)
txt = [elem.text.strip() for elem in txt]
pattern = re.compile('[A-Z][0-9]')

[print(elem) for elem in txt if 'Header' in elem or '-' in elem or pattern.match(elem)]

---

您可以找到页眉，然后找到

。查找下一个兄弟姐妹（）

是您的表：

from bs4 import BeautifulSoup

txt = '''<p>
    <div class="my-test-class" style="white-space: pre-wrap; font-size: small; font-family: &quot;Courier New&quot;">
        <div class="my-test-class">Random text goes on.........</div>
        <div class="my-test-class"><br></div>
        <div class="my-test-class">Header1 Header2 Header3 Header4 Header5
        </div>
        <div class="my-test-class">--------------------------------------
        </div>
        <div class="my-test-class">A1 B1 C1 D1 E1</div>
        <div class="my-test-class">A2 B2 C2 D2 E2</div>
        <div class="my-test-class">A3 B3 C3 D3 E3</div>
        <div class="my-test-class">--------------------------------------
        </div>
    </div>
</p>'''

soup = BeautifulSoup(txt, 'html.parser')

header = soup.find(text=lambda t: '----' in t).parent.find_previous()

print(header.text)
print(*[tag.get_text(strip=True) for tag in header.find_next_siblings()], sep='\n')

---

表中有哪些值？只是带空格的数字或字符串？

Header1 Header2 Header3 Header4 Header5
--------------------------------------     
A1 B1 C1 D1 E1
A2 B2 C2 D2 E2
A3 B3 C3 D3 E3
--------------------------------------

from bs4 import BeautifulSoup

txt = '''<p>
    <div class="my-test-class" style="white-space: pre-wrap; font-size: small; font-family: &quot;Courier New&quot;">
        <div class="my-test-class">Random text goes on.........</div>
        <div class="my-test-class"><br></div>
        <div class="my-test-class">Header1 Header2 Header3 Header4 Header5
        </div>
        <div class="my-test-class">--------------------------------------
        </div>
        <div class="my-test-class">A1 B1 C1 D1 E1</div>
        <div class="my-test-class">A2 B2 C2 D2 E2</div>
        <div class="my-test-class">A3 B3 C3 D3 E3</div>
        <div class="my-test-class">--------------------------------------
        </div>
    </div>
</p>'''

soup = BeautifulSoup(txt, 'html.parser')

header = soup.find(text=lambda t: '----' in t).parent.find_previous()

print(header.text)
print(*[tag.get_text(strip=True) for tag in header.find_next_siblings()], sep='\n')

Header1 Header2 Header3 Header4 Header5
        
--------------------------------------
A1 B1 C1 D1 E1
A2 B2 C2 D2 E2
A3 B3 C3 D3 E3
--------------------------------------