I have written a program that scraped source code twice, and created A CSV with specific information from the retrieved data. My issue is, when I go to save the second bit of data, instead of adding to the created CSV, it overwrites it with the new information. I have referred to this link, but it is using a different class. My code is currently:
public static void scrapeWebsite() throws IOException {
final WebClient webClient = new WebClient();
final HtmlPage page = webClient.getPage(s);
originalHtml = page.getWebResponse().getContentAsString();
obtainInformation();
originalHtml = "";
final HtmlForm form = page.getForms().get(0);
final HtmlSubmitInput button = form.getInputByValue(">");
final HtmlPage page2 = button.click();
try {
synchronized (page2) {
page2.wait(1000);
}
}
catch(InterruptedException e)
{
System.out.println("error");
}
originalHtml = originalHtml + page2.refresh().getWebResponse().getContentAsString();
obtainInformation();
}
public static void obtainInformation() throws IOException {
PrintWriter docketFile = new PrintWriter(new FileWriter("tester3.csv", true));
// creates the csv file. (name must be changed, override deletes file) originalHtml = originalHtml.replace('"','*'); int i = 0;
//While loop runs through all the data in the source code. There is (14) entries per page.
while(i<14) {
String plaintiffAtty = "PlaintiffAtty_"+i+"*>"; //creates the search string for the plaintiffatty
Pattern plaintiffPattern = Pattern.compile("(?<="+Pattern.quote(plaintiffAtty)+").*?(?=</span>)");//creates the pattern for the atty
Matcher plaintiffMatcher = plaintiffPattern.matcher(originalHtml); // looks for a match for the atty
while (plaintiffMatcher.find()) {
docketFile.write(plaintiffMatcher.group().toString()+", "); //writes the found atty to the file
}
i++;
}
docketFile.close(); //closes the file
}
}
I believe the change will have to be made in the second method.