I would like to automatically dump the contents of the web app's ajax communication to a local file.
If tcpdump dumps a packet from the source into a file, it may not be impossible, but
I have to separate HTTP headers, so it's hard.
with Chrome's development tool
Network >XHR>Name is a specific endpoint
Only the response body for
Is there an easy way to keep printing to files?
HTML5 (JavaScript) communicates, so it's easy to interrupt
You can use chrome extension or bookmarklet.
Whether it's a browser plug-in or a packet capture tool, it doesn't matter what you do
I'd like to write a program that triggers a specific response while browsing, so I want to be able to read the response from a local program without any manual intervention or give you a response.
javascript html5 google-chrome
I want to write a program that triggers a specific response while browsing.
If that's the case, I recommend that you use puppeter to directly hit Fetch domain of Chrome DevTools Protocol, etc. to interrupt all communications in your browser and crush headers and bodies.
In the following example using node.js, the body binary is output to the console as a UTF-8 string, but of course it can also be saved to a file.
For the above purpose, I think it would be better not to save it to a file.
const supplier=require('puppeter');
(async() = > {
const browser = wait supplier.launch({headless:false, defaultViewport:null});
const page=(wait browser.pages())[0];
try{
const client=wait page.target().createCDPSession();
wait client.send('Fetch.enable', {'patterns':[{'urlPattern':'*', 'requestStage':'Response'}] });
client.on('Fetch.requestPaused', async(requestEvent)=>{
const {requestId, responseStatusCode, responseHeaders} = requestEvent;
try{
if(!responseStatusCode)throw`responseStatusCode:${responseStatusCode}`;
const response = wait client.send('Fetch.getResponseBody', {requestId});
// dump response body
if(response.base64Encoded){
console.log(new Buffer(response.body, 'base64').toString('utf-8'));
} else{
console.log (response.body);
}
wait client.send('Fetch.fullRequest', {requestId, responseCode:responseStatusCode, responseHeaders, 'body':response.body});
} catch{
wait client.send('Fetch.continueRequest', {requestId});
}
});
// wait for close
wait new Promise((resolve, reject)=>page.on('close',resolve));
} US>finally
(await browser.pages()) .forEach(p=>p.close());
browser.close();
}
})();
client.send('Fetch.enable',...); allows you to handle only certain endpoints by setting urlPattern appropriately.
In the example above, we used a supplier bundled with Chrome, but for example, if you want to use Chrome installed separately, you have the option of using puppeter-core or using bindings in other languages.
Chrome DevTools Protocol and Remote Debugging Protocol are standardized specifications, so they are available from other tools, such as Selenium v4, as well as papeteers.
DevTools can export and import HAR files.
I can't output it endlessly, but if it's enough to capture and export it for a certain period of time, please try it.
Generate HAR files for troubleshooting
chrome://net-export/
How about using this?
With Windows, how about a tool called Fiddler?
https://www.telerik.com/fiddler/fiddler-classic
The add-on feature in mitmproxy will enable it.
Example)
save_body.py:
import mitmproxy
def response (flow):
if flow.request.pretty_url.startswith("https://www3.nhk.or.jp/news/special/coronavirus/data/") and flow.response.headers ["content-type"].startswith("application/json"):
with open("responses.txt", "ab") asf:
f.write(flow.response.content)
Verification Steps:
Launch the proxy server with the mitmdump command and
mitmdump-save_body.py
Browse to https://www3.nhk.or.jp/news/special/coronavirus/data/ in a web browser with mitmproxy set to proxy.
*I don't know Python grammar, so I used the following as a reference:
© 2024 OneMinuteCode. All rights reserved.