Call Google search API in Java

This post shows how to call google search API in a standard Java program.

1. A Naive Approach

The method below does not require any third-party library, but the result format is not readable, and we can not find what we need through the result text.

public static void main(String[] args) throws IOException {
 
	String address = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=";
	String query = "java tutorial";
	String charset = "UTF-8";
 
	URL url = new URL(address + URLEncoder.encode(query, charset));
 
	BufferedReader in = new BufferedReader(new InputStreamReader(
			url.openStream()));
	String str;
 
	while ((str = in.readLine()) != null) {
		System.out.println(str);
	}
 
	in.close();
}

2. A Better Approach

Luckily, someone already wrote a library for us to convert the Json format to Java class. The library is called "Google Gson", it can be download here: http://code.google.com/p/google-gson/downloads/list. We need to use gson-2.2.1.jar inside of the zip file.

The following code now can return the urls for us.

package google.api.search;
 
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
import java.net.URLEncoder;
import java.util.List;
import com.google.gson.Gson;
 
public class TestGoogleSea {
 
	public static void main(String[] args) throws IOException {
 
		String address = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=";
		String query = "programcreek";
		String charset = "UTF-8";
 
		URL url = new URL(address + URLEncoder.encode(query, charset));
		Reader reader = new InputStreamReader(url.openStream(), charset);
		GoogleResults results = new Gson().fromJson(reader, GoogleResults.class);
 
		int total = results.getResponseData().getResults().size();
		System.out.println("total: "+total);
 
		// Show title and URL of each results
		for(int i=0; i<=total-1; i++){
			System.out.println("Title: " + results.getResponseData().getResults().get(i).getTitle());
			System.out.println("URL: " + results.getResponseData().getResults().get(i).getUrl() + "\n");
 
		}
	}
}
 
 
class GoogleResults{
 
    private ResponseData responseData;
    public ResponseData getResponseData() { return responseData; }
    public void setResponseData(ResponseData responseData) { this.responseData = responseData; }
    public String toString() { return "ResponseData[" + responseData + "]"; }
 
    static class ResponseData {
        private List<Result> results;
        public List<Result> getResults() { return results; }
        public void setResults(List<Result> results) { this.results = results; }
        public String toString() { return "Results[" + results + "]"; }
    }
 
    static class Result {
        private String url;
        private String title;
        public String getUrl() { return url; }
        public String getTitle() { return title; }
        public void setUrl(String url) { this.url = url; }
        public void setTitle(String title) { this.title = title; }
        public String toString() { return "Result[url:" + url +",title:" + title + "]"; }
    }
}

Output:

total: 4

Title: ProgramCreek.com
URL: http://www.programcreek.com/

Title: Top 8 Diagrams for Understanding Java - ProgramCreek.com
URL: http://www.programcreek.com/2013/09/top-8-diagrams-for-understanding-java/

Title: Top 10 Methods for Java Arrays - ProgramCreek.com
URL: http://www.programcreek.com/2013/09/top-10-methods-for-java-arrays/

Title: Eclipse JDT Tutorials - ProgramCreek.com
URL: http://www.programcreek.com/2011/01/best-java-development-tooling-jdt-and-astparser-tutorials/

Now you have got a list of readable Google search result by using search API and Gson, but you may quickly notice that the result only contains 4 results, but we need more.

This is not a bug, it is designed to be this way. What you can do is to add a parameter to the url "start=#" like the FOLLOWING:

String address = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&start=4&q=";

If start is set to be 4, we have 5th-8th results; if the number is 100, we have 101st-104th results, so on and so forth. The following is results when start is 4.

Title: Junior Ranger Program – Creek Critters | Friends of Tryon Creek
URL: http://www.tryonfriends.org/junior-ranger-program-creek-critters/

Title: 10 03 13 SCHS vs Sanford, Full program, Creek wins! - YouTube
URL: http://www.youtube.com/watch%3Fv%3Dv0GEi-7LgYg

Title: Programcreek.com: Backreferences in Java Regular Expressions ...
URL: http://www.java.net/story/programcreekcom-backreferences-java-regular-expressions

Title: California, Marin County, McIsaac Ranch, STRAW program creek ...
URL: http://www.davidsanger.com/stockimages/2-216-47.STRAW

3. Put it into a Loop to get more results

You can put this in a loop and change start# in each loop to get more results.

for (int i = 0; i < 20; i = i + 4) {
	String address = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&start="+i+"&q=";
 
	String query = "Programcreek";
	String charset = "UTF-8";
 
	URL url = new URL(address + URLEncoder.encode(query, charset));
	Reader reader = new InputStreamReader(url.openStream(), charset);
	GoogleResults results = new Gson().fromJson(reader, GoogleResults.class);
 
	// Show title and URL of each results
	for (int m = 0; m <= 3; m++) {
		System.out.println("Title: " + results.getResponseData().getResults().get(m).getTitle());
		System.out.println("URL: " + results.getResponseData().getResults().get(m).getUrl() + "\n");
	}
}

You should be aware that Google does not allow too many requests in a certain amount of time.

Category >> Google API

If you want someone to read your code, please put the code inside <pre><code> and </code></pre> tags. For example:

<pre><code> 
String foo = "bar";
</code></pre>

Julien Khaleghy

There is also a java library that does all of that: https://github.com/serpapi/google-search-results-java
Ata Bahar

the below api is depracated… use this and look for its protocol to collect data:https://www.googleapis.com/customsearch/v1?key=AIzaSyC18pt10gbTHeaROmXuisxKGcxJsxRugjw&cref&q=
Alishba

why do I get null response data
Zain Ul Haq

I am having a similar problem any progress?
Rana Prathap B

Hey… a basic question… does google says in any of its documentation to use the below address?
if not how did we know to get the addess like this?
http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=
KESHEEKA SHEEKSHA HEERAMUN

how to retrieve the summary of each search results as well like google does on displaying search results with a brief description/summary
Mr.Choi

Hi, thanks for the tutorial. By the way, may i know is that a way to search news with specified date?
CS Lee

I am looking for a way to parse into google search, a list of values. In other words, to make a query, then save the results into a file; then query the next item in the list, until the list is exhausted. anyone tried that? is there an api for that?
leesf

Hi,if i want to get more results(500),then, i should query the api 125 times, but there is a limit.can i change the 4 results to a bigger number ?
shams

Hi, thanks for the nice tutorial. I was trying to add site: option with the search string. When I add some specific websites as the site option it doesn’t show any result, though manual google search shows some entries. What could be the possible reason?
Harminder Singh Deol

How would i go about turning this into a voice search, such that i speak and i get an answer back in voice. Is there one google api that does all that or will it be combination of google speech to text & text to speech API.
jihen

Hi, how can I get the results in Frensh and not on English (url and title) ???
Chengfeng Lv

Hi, if I just want to get the results from Web(no images or video and so on), what should I do to the address arguments??Is there any way to solve it. Thanks.
farhan007

how could i pass a String to JTextField() to search and from the console output to the JPanel ?? Please help if any idea here….
Joe Tran

I see. But i’m willing to pay to get more results. I have the key but it still limit at 100 results. Any idea? Thanks.
ryanlr

It is controlled on the Google server side. So I don’t think there is a solution.
Joe Tran

Hi, thanks for your code, it really hepls me a lot. However, I got a question, I want to have about 200 result at once so I change the value at “i<20" to "i<200". But it only shows maximum 100 results. Is there any way to solve it. Thanks.
liujiefeng

The method fromJson(Reader, Class) from the type Gson refers to the missing type GoogleResults ：when I run this code ,the result finds the error .why? Thank you for your adivce
ryanlr

How to get more results is added in the code.
Deaw Nibbāna

Could you pleas help me? I don’t understand also about the my result how to more then 4
please share exsample ?, Thank you for your advance
User

Yes, i’m trying but when i put something like : “System.out.println(results.getResponseData().getResults().get(5).getTitle());” this give me : “java.lang.NullPointerException.”

The address looks like that : “String address = “http://ajax.googleapis.com/ajax/services/search/web?v=1.0&start=”+ i + “&q=”; ” where i is 4,8 .. an so on.
ryanlr

add a parameter to the url “start=#”. If this number is 4, this we have 5-8 results, etc.
User

I don’t understand how can I modify the code so this can return me more then 4 results ?
long

Nice work, BTW, thanks a lot.
long

I ran into the same problem, but when you use rsz=8 instead of start= 20, things sort out. The maximum value is 8 otherwise the default 4 pages would be used instead.
Monica

I tried to add “start=20” to the url but it still shows only 4 results.I would truly appreciate it very much if you can tell me exactly how to get more than 4 search results. So what I wrote is
URL url = new URL(address + “start=20” + URLEncoder.encode(query, charset));

but it does not work.

Thank you in advance!

Call Google search API in Java

Related posts: