Server returned HTTP response code: 429 for URL JAVA Reddit JSON

Question

I am trying to get the JSON file found here: https://www.reddit.com/r/arabfunny/top.json?limit=100

I have the following code:

static void getPost() throws Exception {
    String webPage = "https://www.reddit.com/r/arabfunny/top.json?limit=100";

    URL url = new URL(webPage);
    URLConnection request = url.openConnection();
    request.connect();

    JsonParser jp = new JsonParser();
    JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent()));
    JsonObject rootobj = root.getAsJsonObject();
}

This code throws the following error when run:

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 429 for URL: https://www.reddit.com/r/arabfunny/top.json?limit=100

HTTP Status Code 429: The user has sent too many requests in a given amount of time ("rate limiting"). — Gilbert Le Blanc, Aug 09 '20 at 01:20
Why is my code sending more than one request? I only want it to download the file once. What should I look at/change? — Luke Prior, Aug 09 '20 at 01:24
How many times did you test your code in a short period of time? Each test counts as 100 requests. — Gilbert Le Blanc, Aug 09 '20 at 01:26
This was the first time, how can I make it do just one request? — Luke Prior, Aug 09 '20 at 01:36
Using Spring Boot you can make use of [RestTemplate](https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/client/RestTemplate.html), you can see a quick guide [here](https://www.baeldung.com/rest-template). This will help you to consume Api rest by http method very easily. — Mauro Valc, Aug 09 '20 at 01:26
The URL works fine in my browser. You might have to set the User-Agent to pretend your code is coming from a standard browser. https://stackoverflow.com/questions/2529682/setting-user-agent-of-a-java-urlconnection — Gilbert Le Blanc, Aug 09 '20 at 01:43

score 1 · Answer 1 · answered Aug 09 '20 at 01:58

Fix problem is set Content type to UrlConnection

request.setRequestProperty("Content-Type", "application/json; utf-8");

Full code:

package com.example;

import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;

/**
 * Hello world!
 */
public class App {
    public static void main(String[] args) throws IOException {
        String webPage = "https://www.reddit.com/r/arabfunny/top.json?limit=100";

        URL url = new URL(webPage);
        URLConnection request = url.openConnection();
        request.setRequestProperty("Content-Type", "application/json; utf-8");

        JsonParser jp = new JsonParser();
        JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent()));
        JsonObject rootobj = root.getAsJsonObject();
        System.out.println(rootobj);
    }
}

This does not work and still returns Server returned HTTP response code: 429 for URL: https://www.reddit.com/r/arabfunny/top.json?limit=100 — Luke Prior, Aug 09 '20 at 03:35
Retry review example and effectively not working first execution but is problem of rest **arabfunny**, second execution was success. — sercheo_87, Aug 10 '20 at 17:25

score 1 · Accepted Answer · answered Aug 09 '20 at 03:39

1

The problem was fixed by adding:

request.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2");

answered Aug 09 '20 at 03:39

Luke Prior

885
2
11
34

Server returned HTTP response code: 429 for URL JAVA Reddit JSON

2 Answers2