Home » Java » Java HTMLUnit getByFirstXPath not working

Java HTMLUnit getByFirstXPath not working

Posted by: admin July 23, 2018 Leave a comment


I am trying to load a page and click a link using HTMLUnit 2.4 in Java. I am attempting to find the calendar on a page by xPath. It is returning null. The xPath was copied directly out of Chrome DevTools. I used Chrome DevTools Network tab to see that the very first resource loaded to the page contains the data I want. I don’t think this is AJAX related.

import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlDivision;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

import java.io.IOException;

public class App {
    public static void main( String[] args ) throws IOException {
        final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3);
        HtmlPage homePage = webClient.getPage("http://bay.realtaxdeed.com");
        HtmlDivision calendarButtonDiv = homePage.getFirstByXPath("//*[@id=\"splashMenuBottom\"]");
        HtmlPage currentMonthPage = calendarButtonDiv.click();

        HtmlElement element = currentMonthPage.getElementById("MAIN_TBL_CONTENT");
        HtmlDivision calendarDivision = currentMonthPage.getFirstByXPath("//*[@id=\"MAIN_TBL_CONTENT\"]/div[2]/div/div[11]");

        System.out.println( "Run complete." );

I had to disable the javascript to avoid getting an error on the landing page. I re-enable to try and ensure the WebClient performs like a normal browser. I successfully get the calendarButtonDiv by xpath. I click that to get the page with the calendar. My goal is to click the days with hyperlinks in them.

Why can’t I find the calendar (calendarDivision) by xPath? What is the proper way to find elements using HTMLUnit?