Idris Kadri
Front & Back End Developer & Animator

Bulk Webpage Cacher

Github

Languages & Tools: Java

2 Programs, The first visits a list of websites from a text document automatically. The second creates html files from php based websites such as wordpress, which in turn creates cached pages. This is mainly used to speed up caching webpages on large websites. Recently used to create cached pages for 1.4 million product pages on http://electronikz.com.


Sample Code



	
count = thread -1;

while (count < urls.length) {



	try {

		document = Jsoup.connect(urls[count]).timeout(60000).get(); //get websitelink

		completed ++;

		System.out.println(count + " : " + "cached : " + urls[count] + " (" + completed + ")");
		writer.append("success : " + urls[count] + "n");

		count+=threadAmount;
		attempts = 0;

	}catch(Exception e){

		System.out.println(count + " : " + "failed : " + urls[count] + " : Failed to cache post...");
		writer.append("failed : " + urls[count] + "n");
		count+=threadAmount;

	}


}

System.out.println("Thread " + thread + " completed...");
Back