Python javascript Crawling without web page selenium rendered

There are a lot of crawling questions these days

You can think at a common sense level.

You can treat html as a simple string, but javascript needs to be run to get the results.

You must have a JavaScript engine (VM) to perform javascript sources.

The reason for using selenium is that you can use the javascript engine and DOM, which are the basic features of your browser.

Except selenium, javascript source should be performed by javascript source with javascript engine such as spider monkey and parsed the result, and if HTML is being processed within javascript, DOM should be handled as well.

I recommend using selenium for mental health.