org.jsoup.helper.HttpConnection Java Examples

The following examples show how to use org.jsoup.helper.HttpConnection. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: MovieRatingChanger.java    From J-Kinopoisk2IMDB with Apache License 2.0 6 votes vote down vote up
/**
 * Sends POST request and changes Movie rating,
 * and {@link Movie#imdbId}
 *
 * @param movie Movie which rating to change
 * @throws IOException If an I/O error occurs
 */
@Override
public ExchangeObject<Integer> prepare(@NonNull Movie movie) throws IOException {
    val movieRatingChangeLink = new URL("https://www.imdb.com/ratings/_ajax/title");

    val postData = new ImmutableMap.Builder<String, String>()
            .put("tconst", movie.getImdbId())
            .put("rating", String.valueOf(movie.getRating()))
            .put("pageId", movie.getImdbId())
            .put("tracking_tag", "title-maindetails")
            .put("pageType", "title")
            .put("subpageType", "main")
            .build();

    val request = HttpConnection.connect(movieRatingChangeLink)
            .method(Connection.Method.POST)
            .userAgent(config.getString("user_agent"))
            .timeout(config.getInt("timeout"))
            .cookie("id", config.getString("auth"))
            .header("Content-Type", "application/x-www-form-urlencoded")
            .ignoreContentType(true)
            .data(postData)
            .request();

    return new ExchangeObject<>(request, new JSONPOSTResponseProcessor());
}
 
Example #2
Source File: FormElement.java    From jsoup-learning with MIT License 6 votes vote down vote up
/**
 * Get the data that this form submits. The returned list is a copy of the data, and changes to the contents of the
 * list will not be reflected in the DOM.
 * @return a list of key vals
 */
public List<Connection.KeyVal> formData() {
    ArrayList<Connection.KeyVal> data = new ArrayList<Connection.KeyVal>();

    // iterate the form control elements and accumulate their values
    for (Element el: elements) {
        if (!el.tag().isFormSubmittable()) continue; // contents are form listable, superset of submitable
        String name = el.attr("name");
        if (name.length() == 0) continue;

        if ("select".equals(el.tagName())) {
            Elements options = el.select("option[selected]");
            for (Element option: options) {
                data.add(HttpConnection.KeyVal.create(name, option.val()));
            }
        } else {
            data.add(HttpConnection.KeyVal.create(name, el.val()));
        }
    }
    return data;
}
 
Example #3
Source File: IMDBHTMLExchangeStrategy.java    From J-Kinopoisk2IMDB with Apache License 2.0 5 votes vote down vote up
/**
 * {@inheritDoc}
 */
@Override
public Connection.Request buildRequest(@NonNull final Movie movie) {
    val searchLink = "http://imdb.com/find";
    val queryParams = new ImmutableMap.Builder<String, String>()
            .put("q", movie.getTitle())
            .put("s", "tt")
            //.put("exact", "true")
            .put("ref", "fn_tt_ex")
            .build();

    return HttpConnection.connect(HttpUtils.buildURL(searchLink, queryParams)).request();
}
 
Example #4
Source File: OMDBExchangeStrategy.java    From J-Kinopoisk2IMDB with Apache License 2.0 5 votes vote down vote up
@Override
public Connection.Request buildRequest(@NonNull Movie movie) {
    val searchLink = "http://www.omdbapi.com";
    val queryParams = new ImmutableMap.Builder<String, String>()
            .put("t", movie.getTitle())
            .put("plot", "short")
            .put("r", "json")
            .put("apikey", config.getString("omdbApiKey"))
            .build();

    return HttpConnection.connect(HttpUtils.buildURL(searchLink, queryParams))
            .ignoreContentType(true)
            .request();
}
 
Example #5
Source File: MovieWatchlistAssigner.java    From J-Kinopoisk2IMDB with Apache License 2.0 5 votes vote down vote up
/**
 * Sends POST request and adds Movie to IMDB list, using {@link Movie#imdbId}
 *
 * @param movie Movie which should be added to IMDB list
 * @throws IOException If an I/O error occurs
 */
@Override
public ExchangeObject<Integer> prepare(@NonNull Movie movie) throws IOException {
    val list = config.getString("list");
    String url;
    Connection.Method method = Connection.Method.POST;
    if (list.equals("watchlist")) {
        url = "https://www.imdb.com/watchlist/" + movie.getImdbId();
        method = Connection.Method.PUT;
    }
    else {
        url = "https://www.imdb.com/list/" + list + '/' + movie.getImdbId() + "/add";
    }
    val movieAddToWatchlistLink = new URL(url);

    val postData = new ImmutableMap.Builder<String, String>()
            .put(config.getString("authControlKey"), config.getString("authControlValue"))
            .build();

    val request = HttpConnection.connect(movieAddToWatchlistLink)
            .method(method)
            .userAgent(config.getString("user_agent"))
            .timeout(config.getInt("timeout"))
            .cookie("session-id", config.getString("authSessionId"))
            .cookie("at-main", config.getString("authAtMain"))
            .cookie("ubid-main", config.getString("authUbidMain"))
            .header("Content-Type", "application/x-www-form-urlencoded")
            .ignoreContentType(true)
            .data(postData)
            .request();

    return new ExchangeObject<>(request, new JSONPOSTResponseProcessor());
}
 
Example #6
Source File: Jsoup.java    From astor with GNU General Public License v2.0 2 votes vote down vote up
/**
 * Creates a new {@link Connection} to a URL. Use to fetch and parse a HTML page.
 * <p>
 * Use examples:
 * <ul>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").userAgent("Mozilla").data("name", "jsoup").get();</code></li>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").cookie("auth", "token").post();</code></li>
 * </ul>
 * @param url URL to connect to. The protocol must be {@code http} or {@code https}.
 * @return the connection. You can add data, cookies, and headers; set the user-agent, referrer, method; and then execute.
 */
public static Connection connect(String url) {
    return HttpConnection.connect(url);
}
 
Example #7
Source File: Jsoup.java    From astor with GNU General Public License v2.0 2 votes vote down vote up
/**
 * Creates a new {@link Connection} to a URL. Use to fetch and parse a HTML page.
 * <p>
 * Use examples:
 * <ul>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").userAgent("Mozilla").data("name", "jsoup").get();</code></li>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").cookie("auth", "token").post();</code></li>
 * </ul>
 * @param url URL to connect to. The protocol must be {@code http} or {@code https}.
 * @return the connection. You can add data, cookies, and headers; set the user-agent, referrer, method; and then execute.
 */
public static Connection connect(String url) {
    return HttpConnection.connect(url);
}
 
Example #8
Source File: Jsoup.java    From astor with GNU General Public License v2.0 2 votes vote down vote up
/**
 * Creates a new {@link Connection} to a URL. Use to fetch and parse a HTML page.
 * <p>
 * Use examples:
 * <ul>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").userAgent("Mozilla").data("name", "jsoup").get();</code></li>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").cookie("auth", "token").post();</code></li>
 * </ul>
 * @param url URL to connect to. The protocol must be {@code http} or {@code https}.
 * @return the connection. You can add data, cookies, and headers; set the user-agent, referrer, method; and then execute.
 */
public static Connection connect(String url) {
    return HttpConnection.connect(url);
}
 
Example #9
Source File: Jsoup.java    From jsoup-learning with MIT License 2 votes vote down vote up
/**
 * Creates a new {@link Connection} to a URL. Use to fetch and parse a HTML page.
 * <p>
 * Use examples:
 * <ul>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").userAgent("Mozilla").data("name", "jsoup").get();</code></li>
 *  <li><code>Document doc = Jsoup.connect("http://example.com").cookie("auth", "token").post();</code></li>
 * </ul>
 * @param url URL to connect to. The protocol must be {@code http} or {@code https}.
 * @return the connection. You can add data, cookies, and headers; set the user-agent, referrer, method; and then execute.
 */
public static Connection connect(String url) {
    return HttpConnection.connect(url);
}
 
Example #10
Source File: Jsoup.java    From astor with GNU General Public License v2.0 votes vote down vote up
/**
 Fetch a URL, and parse it as HTML. Provided for compatibility; in most cases use {@link #connect(String)} instead.
 <p>
 The encoding character set is determined by the content-type header or http-equiv meta tag, or falls back to {@code UTF-8}.

 @param url           URL to fetch (with a GET). The protocol must be {@code http} or {@code https}.
 @param timeoutMillis Connection and read timeout, in milliseconds. If exceeded, IOException is thrown.
 @return The parsed HTML.

 @throws java.net.MalformedURLException if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
 @throws HttpStatusException if the response is not OK and HTTP response errors are not ignored
 @throws UnsupportedMimeTypeException if the response mime type is not supported and those errors are not ignored
 @throws java.net.SocketTimeoutException if the connection times out
 @throws IOException if a connection or read error occurs

 @see #connect(String)
 */
public static Document parse(URL url, int timeoutMillis) throws IOException {
    Connection con = HttpConnection.connect(url);
    con.timeout(timeoutMillis);
    return con.get();
}
 
Example #11
Source File: Jsoup.java    From astor with GNU General Public License v2.0 votes vote down vote up
/**
 Fetch a URL, and parse it as HTML. Provided for compatibility; in most cases use {@link #connect(String)} instead.
 <p>
 The encoding character set is determined by the content-type header or http-equiv meta tag, or falls back to {@code UTF-8}.

 @param url           URL to fetch (with a GET). The protocol must be {@code http} or {@code https}.
 @param timeoutMillis Connection and read timeout, in milliseconds. If exceeded, IOException is thrown.
 @return The parsed HTML.

 @throws java.net.MalformedURLException if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
 @throws HttpStatusException if the response is not OK and HTTP response errors are not ignored
 @throws UnsupportedMimeTypeException if the response mime type is not supported and those errors are not ignored
 @throws java.net.SocketTimeoutException if the connection times out
 @throws IOException if a connection or read error occurs

 @see #connect(String)
 */
public static Document parse(URL url, int timeoutMillis) throws IOException {
    Connection con = HttpConnection.connect(url);
    con.timeout(timeoutMillis);
    return con.get();
}
 
Example #12
Source File: Jsoup.java    From astor with GNU General Public License v2.0 votes vote down vote up
/**
 Fetch a URL, and parse it as HTML. Provided for compatibility; in most cases use {@link #connect(String)} instead.
 <p>
 The encoding character set is determined by the content-type header or http-equiv meta tag, or falls back to {@code UTF-8}.

 @param url           URL to fetch (with a GET). The protocol must be {@code http} or {@code https}.
 @param timeoutMillis Connection and read timeout, in milliseconds. If exceeded, IOException is thrown.
 @return The parsed HTML.

 @throws java.net.MalformedURLException if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
 @throws HttpStatusException if the response is not OK and HTTP response errors are not ignored
 @throws UnsupportedMimeTypeException if the response mime type is not supported and those errors are not ignored
 @throws java.net.SocketTimeoutException if the connection times out
 @throws IOException if a connection or read error occurs

 @see #connect(String)
 */
public static Document parse(URL url, int timeoutMillis) throws IOException {
    Connection con = HttpConnection.connect(url);
    con.timeout(timeoutMillis);
    return con.get();
}
 
Example #13
Source File: Jsoup.java    From jsoup-learning with MIT License votes vote down vote up
/**
 Fetch a URL, and parse it as HTML. Provided for compatibility; in most cases use {@link #connect(String)} instead.
 <p>
 The encoding character set is determined by the content-type header or http-equiv meta tag, or falls back to {@code UTF-8}.

 @param url           URL to fetch (with a GET). The protocol must be {@code http} or {@code https}.
 @param timeoutMillis Connection and read timeout, in milliseconds. If exceeded, IOException is thrown.
 @return The parsed HTML.

 @throws java.net.MalformedURLException if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
 @throws HttpStatusException if the response is not OK and HTTP response errors are not ignored
 @throws UnsupportedMimeTypeException if the response mime type is not supported and those errors are not ignored
 @throws java.net.SocketTimeoutException if the connection times out
 @throws IOException if a connection or read error occurs

 @see #connect(String)
 */
public static Document parse(URL url, int timeoutMillis) throws IOException {
    Connection con = HttpConnection.connect(url);
    con.timeout(timeoutMillis);
    return con.get();
}