0
FileStatus[] status = hdfs.listStatus(new Path(args[5] + "/" + typeOfFlow + "/" + args[4] + "/" + decileValue + "/"));

// you need to pass in your hdfs path
ArrayList<FileStatus> directories1 = new ArrayList<FileStatus>();
for (FileStatus f : status) {
    if (f.isDir()) {
        directories1.add(f);
    }
}
FileStatus[] directories = directories1.toArray(new FileStatus[directories1.size()]);
Arrays.sort(directories, new Comparator<FileStatus>() {
    public int compare(FileStatus o1, FileStatus o2) {
        return (int) (o1.getModificationTime() - o2.getModificationTime());
    }
});

I am getting below exception :

Caused by: java.lang.IllegalArgumentException: Comparison method violates its general contract!
                at java.util.TimSort.mergeHi(TimSort.java:899)
                at java.util.TimSort.mergeAt(TimSort.java:516)
                at java.util.TimSort.mergeForceCollapse(TimSort.java:457)
                at java.util.TimSort.sort(TimSort.java:254)
                at java.util.Arrays.sort(Arrays.java:1438)

I made a test class and checked for any of two object null or object.getModeificationTime null but the stack trace is not same :

Exception in thread "main" java.lang.NullPointerException
    at MyObject.getModificationTime(Main.java:45)
    at Main$1.compare(Main.java:35)
    at Main$1.compare(Main.java:33)
    at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
    at java.util.TimSort.sort(TimSort.java:220)
    at java.util.Arrays.sort(Arrays.java:1438)

I want to know what is the case I am getting exception at the first.

colt_XX
  • 1
  • 1
  • One thing about sorting specifically files by timestamp is that if the timestamp changes while you're sorting, the sorting algorithm may get confused. Are the file timestamps changing? – Andy Turner Oct 23 '17 at 06:26
  • 1
    @KenY-N It is wrong. Comparator should never expect to return only 1 0 -1. The contract stated clearly it is just +ve, 0, -ve – Adrian Shum Oct 23 '17 at 06:34
  • 2
    Since modificationTime is a long value, `(int) (o1.getModificationTime() - o2.getModificationTime())` can wrap around (i.e. return a negative value even though `o1.getModicationTime()` is larger than `o2.getModificationTime()`. You should compare them using `Long.compare(o1.getModificationTime(), o2.getModificationTime())` – Thomas Kläger Oct 23 '17 at 06:35

0 Answers0