import requests
url = 'https://finance.naver.com/item/board.nhn?code=000020&page=1'
req = requests.get(url)
html = req.text
print(html)
Url is included, but there is no page requested in the HTML variable.
I think this code used to work before, but it doesn't work when I rewind it today. I wonder if Naver blocked the crawl or if my code is wrong.
python crawler crawling
When I tested it with Postman, Naver made me respond with a page that does not exist unless it is the User-Agent
corresponding to the browser.
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3)
Add this header and turn it around.
© 2024 OneMinuteCode. All rights reserved.