Most part of what I did is about the testing.
First, I rewrote my test class and separate tests that one test method test one of the target's method. But there is one exception. I will talk about it later.
Second, I modified the method that return void. They are retrivePage(String) and getPages(int).
Both of them are modified to return an int value. The return of retrivePage(String) is the number of links found on this link. The return of getPages(int) is the number of pages that actually retrieved. After that, I found it easier to test them, but also better to know them that I did not know how much pages were actually retrieved before.
When I check for the pmd, it tell me I have to add a assert in testWebSpider() where I test the main method by only call it. In order to past PMD, I added a test of another method with a assert. So this is the only test method that test 2 methods.
I amitted that to do this in order to past PMD test is not a good way. But in the view of PMD, I think the proper way is remove that whole test method because it actually has nothing to assert. However, I would like to keep the test. I think keeping the test is more useful than satisfying PMD.