Error running Selenium Webdriver with a proxy in Python

Question

I tested a script with Selenium Python to retrieve information from a web page. It works, at least until my IP is found. I would like to try using a proxy. I have tried the following two alternatives.

First way:

PROXY = "176.31.68.252:20213" webdriver.DesiredCapabilities.CHROME['proxy'] = { "httpProxy": PROXY, "ftpProxy": PROXY, "sslProxy": PROXY, "noProxy": None, "proxyType": "MANUAL", "class": "org.openqa.selenium.Proxy", "autodetect": False } url = 'https://www.ufficiocamerale.it/' driver = webdriver.Chrome(desired_capabilities=webdriver.DesiredCapabilities.CHROME) wait = WebDriverWait(driver, 20) driver.get(url) # rest of the code 

Second way:

PROXY = "176.31.68.252:20213" chrome_options = webdriver.ChromeOptions() chrome_options.add_argument('--proxy-server=https://%s' % PROXY) url = 'https://www.ufficiocamerale.it/' driver = webdriver.Chrome(chrome_options=chrome_options) wait = WebDriverWait(driver, 20) driver.get(url) # rest of the code 

But in the first case the webpage returns the error ERR_EMPTY_RESPONSE and in the second case it returns the error ERR_TIMED_OUT.

I found the proxy in an online free proxy list.

Is this a problem with my chosen proxy? Or is it a code problem? What is the standard way to proceed in these cases?

Answer 1

Google Restriction

You are getting an error message because of the proxy,

Google blocks all free proxy's

---

Proxy Spider

I also coded a proxy spider that tests all proxy's on: https://free-proxy-list.net/

Code: https://github.com/xtekky/proxy-spider

---

Setup Own Proxy

Here is an article that tells you how to create your own proxy server.

---

Setup Selenium with proxy

• Try this script:

Github Code: https://github.com/xtekky/selenium-tutorials/tree/main/selenium%20proxy

from selenium import webdriver from selenium.webdriver.common.proxy import ProxyType, Proxy #proxy module import time proxy_ip = 'ip:port' #get a free proxy from the websites in the description #setting up proxy proxy =Proxy() proxy.proxy_type = ProxyType.MANUAL proxy.http_proxy = proxy_ip proxy.ssl_proxy = proxy_ip #linking proxy and setting up driver capabilities = webdriver.DesiredCapabilities.CHROME proxy.add_to_capabilities(capabilities) driver = webdriver.Chrome('CHROMEDRIVER PATH', desired_capabilities=capabilities) # replace the chromedriver path #loading test page driver.get('https://httpbin.org/ip') time.sleep(8) driver.quit()