How to refresh token for Google API when using service account

Question

I am lost in maze of Google API Auth types, tokens, procedures, tutorials with links to other 5 tutorials, samples, consoles...

I am using Service Account for authorisation. But it stops to work after 1 hour and I get:

com.google.api.client.auth.oauth2.TokenResponseException: 401 Unauthorized
    at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
    at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
    at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
    at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:384)
    at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
    at com.azimo.tool.androidPublisherService.AndroidPublisherAuth.init(AndroidPublisherAuth.java:36)
    at com.azimo.tool.androidPublisherService.AndroidPublisherService.<init>(AndroidPublisherService.java:24)

And this is how I set up my connection/authorisation with Google Api. Like I said it works for 1 hour:

public class AndroidPublisherAuth {

    private static final String CREDENTIAL_FILE_DIR = "credentials.p12";

    private static JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
    private static HttpTransport HTTP_TRANSPORT;

    public static AndroidPublisher init(String applicationName,
                                    @Nullable String serviceAccountEmail) throws Exception {
        Preconditions.checkArgument(!Strings.isNullOrEmpty(applicationName),
            "applicationName cannot be null or empty!");

        newTrustedTransport();
        Credential credential = authorizeWithServiceAccount(serviceAccountEmail);

        return new AndroidPublisher.Builder(
            HTTP_TRANSPORT, JSON_FACTORY, credential).setApplicationName(applicationName)
            .build();
    }

    private static void newTrustedTransport() throws Exception {
        if (null == HTTP_TRANSPORT) {
            HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
        }
    }

    private static Credential authorizeWithServiceAccount(String serviceAccountEmail) throws Exception {
        return new GoogleCredential.Builder()
                .setTransport(HTTP_TRANSPORT)
                .setJsonFactory(JSON_FACTORY)
                .setServiceAccountId(serviceAccountEmail)
                .setServiceAccountScopes(Collections.singleton(AndroidPublisherScopes.ANDROIDPUBLISHER))
                .setServiceAccountPrivateKeyFromP12File(new File(CREDENTIAL_FILE_DIR))
                .build();
    }
}

How to refresh this connection? (what kind of object should I use for it, what lib should I add to my project, any hint is fine)

I have simple Java project not an Android Project.