I have tried web scraping for the first time by referring to the site below.
http://blog.aidemy.net/entry/2017/12/17/214715
The following error occurred when I tried and obtained only API_KEY and CUSTOM_SEARCH_ENGINE and used the source code of the image collection as it was.
Traceback (most recent call last):
File "gazou_shutoku.py", line 52, in<module>
img_list = get_image_url (keywords[j], 100)
File "gazou_shutoku.py", line 26, inget_image_url
res=urllib.request.urlopen(query_img)
File"C:\Users\Username\Anaconda3\envs\gazou_atsume\lib\urllib\request.
py", line 223, in urlopen
return opener.open(url,data,timeout)
File"C:\Users\Username\Anaconda3\envs\gazou_atsume\lib\urllib\request.
py", line 532, in open
response=meth(req, response)
File"C:\Users\Username\Anaconda3\envs\gazou_atsume\lib\urllib\request.
py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File"C:\Users\Username\Anaconda3\envs\gazou_atsume\lib\urllib\request.
py", line570, in error
return self._call_chain(*args)
File"C:\Users\Username\Anaconda3\envs\gazou_atsume\lib\urllib\request.
py", line 504, in_call_chain
result=func(*args)
File"C:\Users\Username\Anaconda3\envs\gazou_atsume\lib\urllib\request.
py", line 650, in http_error_default
raise HTTPError(req.full_url,code,msg,hdrs,fp)
urllib.error.HTTPError:HTTPError403:Forbidden
What is the problem?Thank you for your cooperation.
python web-scraping
From the error message, the error occurred in the following places:
res=urllib.request.urlopen(query_img)
Therefore, I think there is a problem with the query_image content.
query_img="https://www.googleapis.com/customsearch/v1?key="+API_KEY+"&cx="+CUSTOM_SEARCH_ENGINE+"&num="+str(10 if(total_num-i)>10 else(total_num-i))+"&start="+str(i+1)+quear="&quarech"+"+"
First, to see if the Google customsearch API is enabled, enter the following URL directly in the browser URL field:For API_KEY and CUSTOM_SEARCH_ENGINE, enter your number.
https://www.googleapis.com/customsearch/v1?key=API_KEY&cx=CUSTOM_SEARCH_ENGINE&q=Erika Ikuta & searchType=image
So, if you get an error, please check the API_KEY and CUSTOM_SEARCH_ENGINE settings again.The error is marked as Forbidden, so there is probably a problem with the configuration.Google API misconfiguration is common.
If you can see it correctly, print the query_img value, etc., and check if the query_img value is correct because Custom Search APIs Explorer is located below.
https://developers.google.com/apis-explorer/?hl=ja#p/customsearch/v1/search.cse.list
© 2024 OneMinuteCode. All rights reserved.