On Adding HTTP/2 Support to jsoup

I recently released jsoup 1.19.1, which includes support for making HTTP/2 requests through jsoup’s Connection client. HTTP/2 support had been on my mind for a while, and although implementing it proved relatively straightforward, it involved some interesting design and integration work. The resulting pattern will be reusable for adding other future features, so I thought I’d share some notes on the process.

As background, jsoup handles HTTP requests using Java’s built-in HttpUrlConnection, which has existed since time immemorial. When Java introduced HTTP/2 in Java 11, rather than extending the existing HttpUrlConnection API, it created a completely new HttpClient interface. This meant existing code couldn’t benefit from HTTP/2 simply by upgrading the JDK. But, the new API did clean up some old warts, and bring a path for future version upgrades.

A key goal in adding HTTP/2 support was to make it a seamless, drop-in upgrade for jsoup users. At the same time, I wanted to maintain backward compatibility, keep support for Java 8 (the minimum required by jsoup), and continue full Android compatibility.

So, how to square this circle?

To leverage this, I refactored jsoup’s HttpConnection implementation by abstracting out the HttpUrlConnection logic into a delegate RequestExecutor abstract class. Then, I provided two implementations: one for HttpUrlConnection, and another for HttpClient placed exclusively in the Java 11 version directory. This structure ensures that earlier Java versions continue to compile and run jsoup without issues.

To maintain Android compatibility, I created a RequestDispatch provider that attempts to instantiate the HttpClient implementation and falls back to HttpUrlConnection if necessary.

You can see it come together in the Pull Request: Support HTTP/2 requests via HttpClient.

One significant advantage of HTTP/2 for clients is faster repeated requests to the same host. In Java’s HttpClient, this is implemented by maintaining a connection pool in a HttpClient instance. I integrated this capability into jsoup’s existing Connection.newSession() method, as seen in this commit.

Testing the Implementation

jsoup already has comprehensive unit and integration tests for its connection code, and I wanted to ensure that the new HttpClient implementation had at least the same coverage. This proved very simple to implement; just adding test classes (in the 11 version) that extend the existing tests classes (which then execute all the existing tests in a new context), and use a setup method to toggle-on the HttpClient. And other than a few implementation nuances, all the existing paths pass correctly.

You can see the testing approach here:

• Integration tests
• HttpClient-specific tests

I found using jenv invaluable to flip between different Java environments and verify everything worked OK.

---

Given the significance of this change, the HTTP/2 feature in jsoup 1.19.1 is currently gated behind an opt-in flag (jsoup.useHttpClient). Users can enable it to test and uncover potential integration issues. I’ve been running it in production (e.g., on Try jsoup) and have found it stable and performant. Once it’s had more time to bake, I’ll enable it by default in future releases.

Try it out!

To enable HTTP/2 requests, make sure you upgrade to jsoup 1.19.1, and add this line to your request code:

static {
    System.setProperty("jsoup.useHttpClient", "true");
}

This will enable HTTP/2 support for all requests made through jsoup.

If you find any issues, please report them on the jsoup issue tracker.