beautifulsoup selenium scrapy playwright web scraping crawling

 https://scrapfly.io/blog/web-scraping-with-playwright-and-python/

 https://research.aimultiple.com/python-web-scraping-libraries/

 https://serpdog.io/blog/web-scraping-with-python/

https://blog.apify.com/what-are-the-best-python-web-scraping-libraries/ 

 Beautiful Soup  Requests Scrapy Selenium Playwright Lxml Urllib3 MechanicalSoup

web scraping crawling spider 

https://github.com/fredwu/crawler

https://en.wikipedia.org/wiki/Web_crawler

https://en.wikipedia.org/wiki/Spider_trap

https://proxiesapi-com.medium.com/how-to-scrape-wikipedia-using-python-scrapy-8a867efae2ab