Getting Authentication failed for user on running owasp ZAP scanAsuserAPI

Question

I have used ZAP Desktop using form based authentication, zap runs perfectly fine on Desktop app. However as the web application i am using also has _csrf_token is passed along with username and Password I chose to automate it with manual authentication using selenium.

Below is the error that i am getting -

1112496 [ZAP-ProxyThread-473] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1112601 [ZAP-ProxyThread-481] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1112602 [ZAP-ProxyThread-481] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1112624 [ZAP-ProxyThread-470] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1112624 [ZAP-ProxyThread-470] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1112648 [ZAP-ProxyThread-482] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1112648 [ZAP-ProxyThread-482] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1117079 [ZAP-ProxyThread-488] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1117080 [ZAP-ProxyThread-488] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1117082 [ZAP-ProxyThread-485] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1117088 [ZAP-ProxyThread-485] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1119534 [ZAP-ProxyThread-489] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1119535 [ZAP-ProxyThread-489] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1120768 [ZAP-ProxyThread-490] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1120768 [ZAP-ProxyThread-490] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1120770 [ZAP-ProxyThread-491] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1120770 [ZAP-ProxyThread-491] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1124677 [ZAP-ProxyThread-500] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1124682 [ZAP-SpiderInitThread-4] INFO  org.zaproxy.zap.extension.spider.SpiderThread - Starting spidering scan on https://****/web at 2021-10-21T19:12:37.019+0530
1124682 [ZAP-SpiderInitThread-4] INFO  org.zaproxy.zap.spider.Spider - Spider initializing...
1124707 [ZAP-SpiderInitThread-4] INFO  org.zaproxy.zap.spider.Spider - Starting spider...
1124709 [ZAP-SpiderInitThread-4] INFO  org.zaproxy.zap.spider.Spider - Scan will be performed from the point of view of User: abc
1124714 [ZAP-SpiderThreadPool-4-thread-1] INFO  org.zaproxy.zap.users.User - Authenticating user: abc
1124715 [ZAP-SpiderThreadPool-4-thread-1] INFO  org.zaproxy.zap.users.User - Authentication failed for user: abc
1125460 [ZAP-SpiderThreadPool-4-thread-1] INFO  org.zaproxy.zap.spider.Spider - Spidering process is complete. Shutting down...
1125462 [ZAP-SpiderShutdownThread-4] INFO  org.zaproxy.zap.extension.spider.SpiderThread - Spider scanning complete: true on https://****/web at 2021-10-21T19:12:37.799+0530

My code looks like below -

@Test
    public void zapScanOnTest(String jsonContextsFileName, String testCaseFile, String scanType, String scanPolicyName) throws InterruptedException {
        SoftAssert softassert = new SoftAssert();
            boolean login = loginPage.login(UsrName, Pwd);
            softassert.assertTrue(login, "zapScanTest : Logged in");
            Thread.sleep(10000);
            
            runScanAsUserOnURLs(jsonContextsFileName,"abc_"+HostIP, "after_login_url", UsrName,scanType,scanPolicyName);
    }


public void runScanAsUserOnURLs(String jsonContextsFileName, String zapContextName, String nodeName,
        String UserName, String scanType, String scanPolicyName){
        List<ApiResponse> listOfContext;
        try {
            listOfContext = ((ApiResponseList) clientApi.context.contextList()).getItems();
            String contextID = setAndGetContextID(listOfContext, zapContextName);
            
            log.info("Checking if the User already exists in Context if not Add the user to the context");
            String userID = setAndGetUserID(UserName, contextID);

            log.info("Fetching Json file path and reading all the URL's mentioned in JSON File");
            List<String> urlLists = readJsonFileConvertUrlsToList(jsonContextsFileName, nodeName);
            
            includeAllURLSToContext(urlLists, contextID, zapContextName);
            spiderCrawlScanAsUser(contextID,userID,urlLists);
        }
            catch (InterruptedException e) {    
                e.printStackTrace();
            }
            catch (ClientApiException e) {
                e.printStackTrace();
        }
    }


public String setAndGetContextID(List<ApiResponse> listOfContext, String contextName) throws ClientApiException {
        String contextID = null;
        if (listOfContext.isEmpty() || isContextPresent(listOfContext, contextName) == false) {
            ApiResponse newContext = clientApi.context.newContext(contextName);
            contextID = newContext.toString();
            log.info("Context is Created and the ID is : " + contextID);
        } else {
            Context context = new Context((ApiResponseSet) clientApi.context.context(contextName));
            contextID = context.getId();
            log.info("ID of existing Context is : " + contextID);
        }
        listOfContext = ((ApiResponseList) clientApi.context.contextList()).getItems();
        return contextID;
    }

public boolean isContextPresent(List<ApiResponse> listOfContext, String contextName) {
        boolean isPresent = false;
        String str = "Context is not available in list : " + listOfContext + " let's create a new context";
        log.info("Checking if provided context name " + contextName + " is already present in list of context");
        for (int i = 0; i < listOfContext.size(); i++) {
            String zapContext = listOfContext.get(i).toString();
            if (zapContext.equals(contextName)) {
                isPresent = true;
                str = "Context Name Already Exists : No need to create a Context again : " + listOfContext;
            }
        }
        log.info(str);
        return isPresent;
    }


public String setAndGetUserID(String mcUser, String contextID) throws ClientApiException {
        String userID = null;
        List<ApiResponse> usersListInContext = ((ApiResponseList) clientApi.users.usersList(contextID)).getItems();
        if (usersListInContext.isEmpty() || isUserPresentInContext(usersListInContext, mcUser, contextID) == false) {
            userID = clientApi.users.newUser(contextID, mcUser).toString();
            log.info("User is added to the Context and the user ID is : " + userID);
            log.info("Enabling the User");
            ApiResponse setUserEnabled = clientApi.users.setUserEnabled(contextID, userID, "true");
            log.info("User is Enabled and the status is : " + setUserEnabled);
            log.info("Setting Forced User");
            ApiResponse setForcedUser = clientApi.forcedUser.setForcedUser(contextID, userID);
            log.info("User is set as Forced User and the status is : " + setForcedUser);
            log.info("Enabling Forced User Mode");
            ApiResponse setForcedUserModeEnabled = clientApi.forcedUser.setForcedUserModeEnabled(true);
            log.info("Enabled Forced User Mode and the status is : " + setForcedUserModeEnabled);
        } else {
            for (ApiResponse userListResponse : usersListInContext) {
                String userList = userListResponse.toString(0);
                boolean userPresentInContextList = userList.contains("name = " + mcUser);
                boolean contextIDPresent = userList.contains("contextId = " + contextID);
                if (userPresentInContextList == true && contextIDPresent == true) {
                    userID = userList.substring(userList.indexOf("id = ") + 5, userList.indexOf("enabled"));
                    log.info("User ID is : " + userID);
                    log.info("Enabling the User");
                    ApiResponse setUserEnabled = clientApi.users.setUserEnabled(contextID, userID, "true");
                    log.info("User is Enabled and the status is : " + setUserEnabled);
                    log.info("Setting Forced User");
                    ApiResponse setForcedUser = clientApi.forcedUser.setForcedUser(contextID, userID);
                    log.info("User is set as Forced User and the status is : " + setForcedUser);
                    log.info("Enabling Forced User Mode");
                    ApiResponse setForcedUserModeEnabled = clientApi.forcedUser.setForcedUserModeEnabled(true);
                    log.info("Enabled Forced User Mode and the status is : " + setForcedUserModeEnabled);
                    break;
                }
            }
        }
        return userID;
    }


public boolean isUserPresentInContext(List<ApiResponse> usersListInContext, String mcUser, String contextID) {
        boolean isPresent = false;
        String str = "User is not available in Context List let's add the user";
        log.info("Checking if provided User name " + mcUser + " is already present in list of context");

        for (ApiResponse userListResponse : usersListInContext) {
            String userList = userListResponse.toString(0);
            boolean userPresentInContextList = userList.contains("name = " + mcUser);
            boolean contextIDPresent = userList.contains("contextId = " + contextID);
            if (userPresentInContextList == true && contextIDPresent == true) {
                isPresent = true;
                str = "User is already added to the context, no need to add the user again";
            }
        }
        log.info(str);
        return isPresent;
    }



public List<String> readJsonFileConvertUrlsToList(String jsonContextsFileName, String nodeName) {
        String filePath = readJsonFile.getJsonFilePath(jsonContextsFileName);
        log.info("File Path is : " + filePath);
        FileInputStream fis;
        List<String> urlList = new ArrayList<>();
        try {
            fis = new FileInputStream(filePath);
            JSONTokener tokener = new JSONTokener(fis);
            JSONObject jsonObject = new JSONObject(tokener);
            JSONArray contextJsonArray = jsonObject.getJSONArray("contexts");
            for (int i = 0; i < contextJsonArray.length(); i++) {
                JSONObject testJsonObject = contextJsonArray.getJSONObject(i);
                JSONArray urlJsonArray = testJsonObject.getJSONArray(nodeName);
                log.info("Running ZAP Scan on " + urlJsonArray.length() + " URL's");
                for (int j = 0; j < urlJsonArray.length(); j++) {
                    String urlEndPoints = urlJsonArray.get(j).toString();
                    urlList.add(mcHostUrl + urlEndPoints);
                }
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }
        return urlList;
    }


public void includeAllURLSToContext(List<String> listOfURL, String contextID, String contextName) {
        try {
            log.info("Going to include URLs to context : " + contextName);
            List<ApiResponse> includeContextRegex = ((ApiResponseList) clientApi.context.includeRegexs(contextName)).getItems();
            for (int i = 0 ; i< listOfURL.size();i++) {
                String zapTargetURL = listOfURL.get(i);
            if (includeContextRegex.isEmpty() || isContextRegexPresent(includeContextRegex, zapTargetURL) == false) 
                clientApi.context.includeInContext(contextName, zapTargetURL);
                log.info("Included Context Regex to Context : " + contextName);
            }
        } catch (ClientApiException e) {
            e.printStackTrace();
        }
    }


public void spiderCrawlScanAsUser(String contextID, String userID,List<String> urlList)
            throws InterruptedException, ClientApiException {
        
        for (int i = 0 ; i< urlList.size();i++) {
            String zapTargetURL = urlList.get(i);
            log.info("PREPARING FOR SPIDER CRAWL ON TARGET HOST :" + zapTargetURL);
            log.info("Starting Spider Scan");
            ApiResponse apiResponse = clientApi.spider.scanAsUser(contextID, userID, zapTargetURL, "500",
                "true", "true");
            int progress;
            String scanId = ((ApiResponseElement) apiResponse).getValue();
            do {
                Thread.sleep(5000);
                progress = Integer.parseInt(((ApiResponseElement) clientApi.spider.status(scanId)).getValue());
                log.info("Scan progress: {}{}", progress, "%");
            } while (progress < 100);
            log.info("Spider scan completed");
            List<ApiResponse> spiderResults = ((ApiResponseList) clientApi.spider.results(scanId)).getItems();
            log.info("spider results {}", spiderResults);
        }
    }

Am I missing anything in the above code? I am not able to authenticate the user at all.

score 0 · Answer 1 · answered Oct 21 '21 at 14:49

0

To be honest I wouldnt recommend configuring ZAP that way. I recommend testing everything in the desktop, making sure its all working and then exporting the context - you can then import that via the API. I also notices that you are using Forced User Mode - thats really there for manual testing so I wouldnt use that for automation either. Instead specify the user when you run the spiders and active scanner.

We will bee adding more authentication docs soon and enhancing the Automation Framework to support authentication - those 2 things should make things easier.

answered Oct 21 '21 at 14:49

Simon Bennetts

5,479
1
14
26

Directly Added User and removed the forced user. still the same error - Running the ZAP On desktop daily on different servers would be really time consuming task and hence automating the whole flow this way – Ashu123 Oct 21 '21 at 15:04
No, I'm not saying you should run this manually all of the time, just while you are debuging problems like this :) Its much easier to debug these problems in the ZAP Desktop. – Simon Bennetts Oct 21 '21 at 16:10
If you do that then you will be able to see the requests and responses ZAP makes and hopefully see why the user authentication is failing. – Simon Bennetts Oct 21 '21 at 16:11
In zap Desktop everything runs perfectly fine . Even Form based authentication worked extremely well. Attached screenshot. When i saw all of it working very well on Desktop app and started converting them for automated scripts. – Ashu123 Oct 22 '21 at 03:51
OK, in that case check the zap.log file to see if there are any errors earlier on - eg related to loading any of the scripts you are using. – Simon Bennetts Oct 25 '21 at 09:01

Getting Authentication failed for user on running owasp ZAP scanAsuserAPI

1 Answers1