Auth Fail using private key to connect to client service

Question

I am trying to connect to our client side where files are held so I can grab them from the server. The problem is I am getting AuthFail error using Jsch 0.1.55. I have looked at many issues on SO regarding this, but can't seem to get it to work. I have tried a .ppk key along with this pem key. I can connect via WinSCP and Putty so I know my key is correct and the public key is held on the client server. It seems if I debug in my intelliJ it always fails with Auth Fail. However, running without debugging it I don't get the auth fail exception and it seems to work.

  private String PRIVATE_KEY = "C:\\developer\\workspace\\project\\src\\main\\resources\\pk_rsa_S3.pem";

@Override
  public InputStream retrieveFileViaSFTP(String fileName) {

    JSch jsch = new JSch();
    Session session = null;
    Channel channel = null;
    ChannelSftp channelSftp = null;

    InputStream is = null;
    try {
      jsch.addIdentity(PRIVATE_KEY);
      log.info("Private Key added.");
      session = jsch.getSession(SFTP_USER, SFTP_HOST, SFTP_PORT);

      Properties config = new Properties();
      config.put("StrictHostKeyChecking", "no");
      session.setConfig("PreferredAuthentications", "publickey");
      session.setConfig(config);
      session.connect();  // FAILS WHEN CONNECTING TO SESSION
      channel = session.openChannel("sftp");
      channel.connect();
      log.info("SFTP channel connected");

      channelSftp = (ChannelSftp) channel;
      channelSftp.cd(SFTP_WORKING_DIR);

      // Grab all files at once or one at a time?

      is = channelSftp.get(fileName);

    } catch (JSchException | SftpException e) {
      log.error(e.getMessage());
    } finally {
      if (channelSftp != null) {
        channelSftp.disconnect();
        channelSftp.exit();
      }
      if (channel != null) channel.disconnect();

      if (session != null) session.disconnect();
    }
    return is;