Python scraping doesn't work.

Asked 1 years ago, Updated 1 years ago, 109 views

https://ntp.msn.com/edge/ntp?locale=ja

Microsoft Edge ↑ I took the news from this URL and tried to display it, but an error occurred to see if BeautifulSoup was working properly.

import requests
from bs4 import BeautifulSoup

load_url="https://ntp.msn.com/edge/ntp?locale=ja"
html=requests.get(load_url)
soup = BeautifulSoup(html.content, "html.parser")
print(soup)

topic=soup.find(class_="news-list")
for element intopic.find_all("a"):
    print(element.text)

This code

topic=soup.find(class_="news-list")

in the line of
An exception has occurred: AttributeError
'NoneType' object has no attribute 'find_all'
  File "Hidden Path", line 10, in <module>
    for element intopic.find_all("a"):

The error
occurs. If anyone knows how to solve it, please reply.

python beautifulsoup python-requests

2022-09-30 19:29

1 Answers

You probably don't understand where the error occurred.

The question states that an error occurred on the following line:

topic=soup.find(class_="news-list")

The error message indicates that an error occurred on the following line:

for element intopic.find_all("a"):

Since it is actually 'NoneType' object has no attribute 'find_all', and the object topic is NoneType (=None), the first problem is presented because it does not have the attribute find_all (in this case, method) _group>.

If there is still a gap in perception around here, it is more likely that the investigation will never progress and will not be resolved by making mistakes in what or what needs to be investigated and addressed.

Has it been confirmed that there is an element with the class name "news-list" in the web page contents that can be retrieved manually by using the MS-Edge browser instead of the Python program?

I think it would be better to work step by step so that you can gradually move from those prerequisites to the program.


2022-09-30 19:29

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.