Robot framework, Python - handle lazy load on page with dynamic content

Question

I have a page with dynamically generated table (rows are dynamically generated). In my test I do NOT know how many rows to expect. I just create a dictionary from all the rows and then compare to another dictionary. But the table has lazy load, so when the number of rows is higher, some of the rows which should be there are not visible "on the first sight" but it needs to be scrolled down to get them. So those rows are not included into my dictionary and then it fails... But again, I do not know when to scroll (if the table has small amount of rows, there is nowhere to scroll) and where to scroll (I do not expect any particular element), how many rows to expect etc.

Does anyone has an idea how to handle situation like this? Because I don't. :-( Thank you!

Answer 1

You can try to solve this problem as below:

from selenium.webdriver.common.keys import Keys from selenium.webdriver.support.ui import WebDriverWait as wait from selenium.common.exceptions import TimeoutException # Get the current number of rows current_rows_number = len(driver.find_elements_by_xpath('//tr')) while True: # Scroll down to make new XHR (request more table rows) driver.find_element_by_tag_name('body').send_keys(Keys.END) try: # Wait until number of rows increased wait(driver, 5).until(lambda: len(driver.find_elements_by_xpath('//tr')) > current_rows_number) # Update variable with current rows number current_rows_number = len(driver.find_elements_by_xpath('//tr')) # If number of rows remains the same after 5 seconds passed, break the loop # as there no more rows to receive except TimeoutException: break # Now you can scrape the entire table 

P.S. Unfortunately, I'm not familiar with RobotFramework, so above code is on pure Selenium + Python. I hope it can be easily interpreted :)