Java collection with expiring entries

Question

I need to track certain events over a specified time frame and act if the number of events reaches a certain number. In more detail, I connect to an external service and submit requests which are acknowledged with a status that equals CONF or FAIL. I need to be able to monitor the responses to detect if I get an unusual number of fails in a given time frame, e.g. >3 fails during the last 5 seconds, so that I can check for errors and act accordingly. I could alternatively check for 3 fails in a row but I prefer a time based approach.

I have been testing Guava's CacheLoader after reading this post but while entries (I only store FAIL-events) in the Cache appears to expire as expected, a call to size() (to determine number of fails) includes also the expired entries. This appears to be how it is supposed to work according to the documentation, if I have not misunderstood things?? Is there any way to get the number of 'active' events from a Cache?

I guess an alternative solution is to use a CEP-framework like Esper but it seems like overkill and cumbersome for my simple needs. Does anyone have a completely different approach to suggest that would facilitate my requirement? Thanks

Answer 1

Getting the exact number of active elements from the Cache would require locking the entire cache, which is extremely expensive. You might be able to use the cleanUp() method to make sure that size is not accidentally counting entries that have been quietly evicted, though.

I would not depend on this giving you exact results, but it should improve the accuracy of the results significantly.

Answer 2

You could decorate a collection implementation to do that. Something like this:

public class ExpirableArrayList<E> extends ArrayList<E> {

    private final Date creation = new Date();

    private final long timeToLiveInMs;

    public ExpirableArrayList(long timeToLiveInMs, int initialCapacity) {
        super(initialCapacity);
        this.timeToLiveInMs = timeToLiveInMs;
    }

    public ExpirableArrayList(long timeToLiveInMs) {
        this.timeToLiveInMs = timeToLiveInMs;
    }

    public ExpirableArrayList(long timeToLiveInMs, Collection<? extends E> c) {
        super(c);
        this.timeToLiveInMs = timeToLiveInMs;
    }

    private void expire() {
        if (System.currentTimeMillis() - creation.getTime() > timeToLiveInMs) {
            clear();
        }
    }

    @Override
    public int size() {
        expire();
        return super.size();
    }

    @Override
    public boolean isEmpty() {
        expire();
        return super.isEmpty();
    }

    @Override
    public boolean contains(Object o) {
        expire();
        return super.contains(o);
    }

    @Override
    public Iterator<E> iterator() {
        expire();
        return super.iterator();
    }

    @Override
    public Object[] toArray() {
        expire();
        return super.toArray();
    }

    @Override
    public <T> T[] toArray(T[] a) {
        expire();
        return super.toArray(a);
    }

    @Override
    public boolean add(E e) {
        expire();
        return super.add(e);
    }

    @Override
    public boolean remove(Object o) {
        expire();
        return super.remove(o);
    }

    @Override
    public boolean containsAll(Collection<?> c) {
        expire();
        return super.contains(c);
    }

    @Override
    public boolean addAll(Collection<? extends E> c) {
        expire();
        return super.addAll(c);
    }

    @Override
    public boolean addAll(int index, Collection<? extends E> c) {
        expire();
        return super.addAll(index, c);
    }

    @Override
    public boolean removeAll(Collection<?> c) {
        expire();
        return super.removeAll(c);
    }

    @Override
    public boolean retainAll(Collection<?> c) {
        expire();
        return super.retainAll(c);
    }

    @Override
    public E get(int index) {
        expire();
        return super.get(index);
    }

    @Override
    public E set(int index, E element) {
        expire();
        return super.set(index, element);
    }

    @Override
    public E remove(int index) {
        expire();
        return super.remove(index);
    }

    @Override
    public int indexOf(Object o) {
        expire();
        return indexOf(o);
    }

    @Override
    public int lastIndexOf(Object o) {
        expire();
        return lastIndexOf(o);
    }

    @Override
    public ListIterator<E> listIterator() {
        expire();
        return listIterator();
    }

    @Override
    public ListIterator<E> listIterator(int index) {
        expire();
        return listIterator();
    }

    @Override
    public List<E> subList(int fromIndex, int toIndex) {
        expire();
        return subList(fromIndex, toIndex);
    }
}

Answer 3

I think Guava collection with the nearest functionality to what you want is MinMaxPriorityQueue with a limited maximum size. You'd have to put failure events in chronological order and check periodically for the difference between first and last element and whether it is full.

But what you essentially want is a meter. You can try this Meter from Coda Hale's Metrics library.

Answer 4

I haven't used it but it looks like this might satisfy your needs