https://kxue43.github.io/notes-and-blogs/blogs/Recent content in Blogs on Ke's Notes and BlogsHugoen-usSat, 05 Feb 2022 17:31:35 -0500https://kxue43.github.io/notes-and-blogs/blogs/selenium-vs-playwright/Sat, 05 Feb 2022 17:31:35 -0500https://kxue43.github.io/notes-and-blogs/blogs/selenium-vs-playwright/<p>Recently I developed a Python program that scrapes a Single-Page Application with <a href="https://selenium-python.readthedocs.io/">Selenium</a>, and then reimplemented it with <a href="https://playwright.dev/python/">Playwright</a>. This gave me firsthand knowledge of the difference between the two. In this article, I shall explain my preference of Playwright over Selenium, and share some general experience of scraping single-page applications.</p> <h2 id="the-reason-and-result-of-the-reimplementation" class="heading ">The reason and result of the reimplementation<a href="#the-reason-and-result-of-the-reimplementation" aria-labelledby="the-reason-and-result-of-the-reimplementation"> <!-- <i class="fas fa-link anchor"></i> --> <svg class="svg-inline--fa fas fa-link anchor" fill="currentColor" aria-hidden="true" role="img" viewBox="0 0 576 512"><use href="#fas-link"></use></svg>&nbsp; </a> </h2> <p>My scraping program needs to click an anchor element on the target webpage to trigger file download by the browser, wait for the file download to finish, and then do something with the downloaded file. With Selenium, as surprising as it may seem, waiting for file download to finish is not straightforward. According to its <a href="https://www.selenium.dev/documentation/test_practices/discouraged/file_downloads/">official documentation</a>:</p>