Read Nifi Counter value programmatically

Question

I am developing a custom processor in which I want to read value of Nifi counters. Is there a way to read Counters' value other than using Nifi Rest Api "http://nifi-host:port/nifi-api/counters"?

Answer 1

No. Apache NiFi doesn't have any straightforward APIs available to read the counter value programmatically. An easy approach would be to use GetHTTP processor and use the NiFi REST API URL that you had mentioned: http(s)://nifi-host:port/nifi-api/counters.

Then use EvaluateJsonPath to just parse and read the counter value from the response JSON received from the GetHTTP processor.

Answer 2

Based on Andy's suggestion, I have used refelection to read Counters as follows:

private void  printCounters(ProcessSession session) throws NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, NoSuchMethodException, InvocationTargetException {
        Class standardProcessSession=session.getClass();
        Field fieldContext = standardProcessSession.getDeclaredField("context");
        fieldContext.setAccessible(true);
        Object processContext =  fieldContext.get(session);
        Class processContextClass = processContext.getClass();
        Field fieldCounterRepo = processContextClass.getDeclaredField("counterRepo");
        fieldCounterRepo.setAccessible(true);
        Object counterRepo = fieldCounterRepo.get(processContext);
        Method declaredMethod = counterRepo.getClass().getDeclaredMethod("getCounters");

        ArrayList<Object> counters = (ArrayList<Object>)declaredMethod.invoke(counterRepo);
        for(Object obj:counters) {
            Method methodName = obj.getClass().getDeclaredMethod("getName");
            methodName.setAccessible(true);
            Method methodVal = obj.getClass().getDeclaredMethod("getValue");
            methodVal.setAccessible(true);
            System.out.println("Counter name: "+methodName.invoke(obj));
            System.out.println("Counter value: "+methodVal.invoke(obj));
        }
    }

NOTE: NIFI Version is 1.5.0

Answer 3

While it is not as easy to read/write counter values as it is to modify flowfile attributes, Apache NiFi does have APIs for modifying counters. However, the intent of counters is to provide information to human users, not for processors to make decisions based on their values. Depending on what you are trying to accomplish, you might be more successful using local maps or DistributedMapCacheServer and DistributedMapCacheClientService. If the values are only relevant to this processor, you can just use an in-memory map to store and retrieve the values. If you need to communicate with other processors, use the cache (example here).

Pierre Villard has written a good tutorial about using counters, and you can use ProcessSession#adjustCounter(String counter, int delta, boolean immediate) to modify counter values. Because counters were not designed to allow programmatic there is no way to retrieve the CounterRepository instance from the RepositoryContext object. You may also want to read about Reporting Tasks, as depending on your goal, this may be a better way to achieve it.