Questions tagged [selenium]
Selenium is a tools suite to automate web browsers, and web application testing
165 questions
3votes
1answer
83views
Multi-Page Web Scraping Code Using Selenium with Multithreading
I have written a web scraping script using Selenium to crawl blog content from multiple URLs. The script processes URLs in batches of 1000 and uses multithreading with the ThreadPoolExecutor to ...
- 31
3votes
1answer
86views
Scraping website with Python and Selenium to collect data from dynamic website
Summary: The code scrapes the website and collects the data to store it in CSV. It also downloads selected information that is available for download in PDF format. The details and the entire code are ...
- 269
2votes
1answer
66views
A selenium web scraper to package NBA data
I'm building a selenium web scraper for basketball-reference.com that takes a player name and returns data in either a JSON format or Pandas DataFrame object. The class in question is one of many that ...
0votes
2answers
60views
Application install code- Need to remove Nested Loops and make code more compact and readable. Code installs application and its version as per req
This code installs the application, according to the type of application selected like alpha, beta, and released, also it checks for latest version in each type. ...
- 29
2votes
0answers
89views
Python script that refreshes network connection at random intervals
This is a followup to Python script that reboots the router every 600 seconds. Recently censorship in China has increased because of the meeting of heads of states, and my previous script has been ...
- 3,406
11votes
4answers
2kviews
Python script that reboots the router every 600 seconds
This is a Python script I wrote to reboot the router every 600 seconds. Why did I do it? Well, in case you don't know, I live in China, and I use VPNs to browse the international internet. My ISP ...
- 3,406
0votes
1answer
110views
Poetry Web Scraping in Python [closed]
I have a script that obtains urls that lead to a specific poem. This code current works and uses multiprocessing pools. I currently am getting restricted or blocked by some way from the website that I ...
5votes
1answer
78views
Waits for one of two xpaths to be present, one inside of an iframe, one outside of it
I'm using selenium for testing, and I've written code that waits for one of two locations before it continues. This is really easy when both locations are within the same frame, because then I can ...
- 175
1vote
1answer
56views
Python script for get minimum product price having maximum reviews
Technical task. user visits amazon.com website user fills out a search field with the product name and activates search ==> a page with search results is displayed. user looks for the product ...
1vote
1answer
223views
Selenium/Beautifulsoup web scrape of Grainger
This is my first time writing a web scrape using Selenium and Beautifulsoup. The website I'm scraping is https://www.grainger.com/ and I have it pulling a specific set of SKUs stored in an Excel file. ...
1vote
3answers
2kviews
To automate scheduling at the Italian Consulate
I'm a beginner developer in Python and I need some tips to improve my code, in performance (speed) and hardware usage. Is it possible to improve? And I have a question: is it possible to keep the ...
user267072
2votes
1answer
57views
scraping pollution data tables from government website
i made this python program to scrape and save the daily maximum pollution values in mexico city, the data will be used in machine learning but i wonder what could be improved: ...
3votes
1answer
200views
Scrape PokeDex and display in tkinter
Hi I am new here and I just completed my first working version of a pokedex app with a GUI using tkinter. I used selenium to scrape the data from pokemondb.net, and then used pandas to clean up the ...
3votes
1answer
185views
Crawler/scraper for soccer match results
I wrote this code some time ago as part of the web-scraping learning. Every now and then I find mistakes in it, as well as I have doubts. Feedback please, is this code compliant with the common best ...
1vote
1answer
73views
Volunteer registration website Selenium testing
I have a volunteer registration website that I wrote some end-to-end tests for in Selenium. Selenium is of course software that lets you automate clicking around a browser, which makes it great for ...
- 1,405
