I'm practicing web crawling with Python, but there's something I don't know

I'm practicing using beautiful soup I want to know how to find tags and classes on web pages, and when I use soup.find to find what I want, I wonder if I can only find tags and classes.

And soup.find_all. When several information came out of one tag, What should I do if I want to get the fourth one? What I want is in the fourth one, but I don't know how to use the find properly even if I try to specify it. I specified it as a tag and class, but I got several.

This is the code I'm writing

import requests
from bs4 import BeautifulSoup

url ="https://search.naver.com/search.naver?where=nexearch&sm=top_hty&fbm=1&ie=utf8&query=%EC%86%8D%EC%B4%88%EB%82%A0%EC%94%A8"
res = requests.get(url)
res.raise_for_status()

soup = BeautifulSoup(res.text, "lxml")
temperture = soup.find_all("div", attrs={"class":"todaytemp"})

dust = soup.find_all("span", attrs={"class","num"})
print(dust)

python

2022-09-20 15:15

1 Answers

How to find tags and classes well on web pages

It is usually convenient to open the developer tool window by pressing [F12] on Google Chrome, and then find the corresponding task or class by using the Control+F key or [Find]. Search [Google Developer Tools] for a lot of learning materials.

soup.find : Can only find tags and classes

Find is to find the tag you want. Tags are made up of names (name), attributes (attributes), and attribute values (value), so you can find the tags by specifying Name, Attribute, and Attribute Value as find. The class here is just a type of property.

soup.find_all : find the 4th one

The find_all retrieves all of the tags and creates them in the form of a list. If you saved the results in the myList variable, you can write myList[3] like the fourth one in the list.

2022-09-20 15:15

If you have any answers or tips

Popular Tags

python x 4647

android x 1593

java x 1494

javascript x 1427

c x 927

c++ x 878

ruby-on-rails x 696

php x 692

python3 x 685

html x 656