How to Retrieve http Requests in Python

Asked 2 years ago, Updated 2 years ago, 74 views

I am currently trying to develop a simple client browser using python. Is there any way I can do this with python?I would appreciate it if you could let me know the Python library I use.
1. You can render HTML just like a modern browser.
2. You can interpret javascript just like a modern browser.
3. Communicate like a modern browser.
4. Support for SSL communication.
5. You can get the URL (or http request) when you click on the page and transition to the linked page (link to href on HTML).ここThis is the point
6. Even if it is not href, the URL (or http request) can be obtained if you want to transition to a new page by clicking the button defined by javascript on HTML.ここThis is the point
7. http requests can also be obtained for SLL communication.In other words, it should be obtained before encryption.
8. Do not use PyQt.It's because the license is strict and the fee is expensive.
9. You don't need anything other than the above features, such as browser back/forward buttons, favorites, or access history.

Especially, 5 and 6 are the points.Other than that, I think I can do it with pywebview or webbrowser, but I can't do 5 or 6.If you can do this, you can use python webbrowser, pywebview, or picurl for browser functionality.However, as mentioned above, PyQt is not allowed.
I would appreciate it if you could tell me the best combination and how to achieve it.I look forward to your kind cooperation.

python ssl webview browser

2022-09-30 17:45

1 Answers

Can I use selenium-wire-PyPI?

Here's a question about the old similar request, but in the answer, "As of 2021, the answer is yes.You can use the new library selenium-wire."
Is there any way to log http requests/responses using Selenium Webdriver (firefox)?

The PyPI and GitHub commentary says that you can get a response as well as a request.
It would be much better to use your existing browser as it is than to start reinventing the grand wheel of creating a browser.

Accessing Requests

Selenium Wirecaptures all HTTP/HTTPS traffic made by the browser.

The Selenium Wire captures all HTTP/HTTPS traffic created by the browser.

Intercepting Requests and Responses

As well as capturing requests and responses, Selenium Wire allows you to modify them on the fly using interceptors.An interceptor is a function that gets invoked with requests and responses as they pass through Selenium Wire.Within an interceptor you request.

In addition to capturing requests and responses, Selenium Wire allows you to change them on the spot using an interceptor.An interceptor is a function called when a request and response passes through SeleniumWire.You can modify requests and responses as needed within the interceptor.

Note:
The question seems to have changed from http request to URL (or http request), but every part of the commentary article says request.url, URL, driver.requests, driver.last_request, Occode>.

Here is an article on how to use it easily.
How to capture HTTP requests using Selenium

This article says that you can get AJAX requests.
Selenium-wire does not intercept requests when connecting remote

This is an example of how to delete an ad.
Block ads and load them faster with Selenium [selenium-wire]


2022-09-30 17:45

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.