Java : Merging multiple log files by date

Question

I have 2 log files. I need to merge those 2 files based on the timestamp. I am struggling to find the correct algorithm to do the same.

I have the file 'File1' with content:

2016-07-18 09:58:19,243 : INFO: My Test File1 - 1
2016-07-18 09:58:19,244 : INFO: My Test File1 - 2
2016-07-18 09:58:19,255 : INFO: My Test File1 - 3
2016-07-18 09:58:19,255 : INFO: My Test File1 - 4
2016-07-18 09:58:19,258 : INFO: My Test File1 - 5

The 'File2' with content:

2016-07-18 09:57:09,674 : INFO: My Test File2 - 1
2016-07-18 09:57:09,674 : INFO: My Test File2 - 2
2016-07-18 09:57:09,679 : INFO: My Test File2 - 3
2016-07-18 09:57:09,679 : INFO: My Test File2 - 4
2016-07-18 09:57:09,680 : INFO: My Test File2 - 5
2016-07-18 09:58:49,685 : INFO: My Test File2 - 6
2016-07-18 09:58:49,686 : INFO: My Test File2 - 7
2016-07-18 09:58:49,686 : INFO: My Test File2 - 8

Expected Output File after merge:

2016-07-18 09:57:09,674 : INFO: My Test File2 - 1
2016-07-18 09:57:09,674 : INFO: My Test File2 - 2
2016-07-18 09:57:09,679 : INFO: My Test File2 - 3
2016-07-18 09:57:09,679 : INFO: My Test File2 - 4
2016-07-18 09:57:09,680 : INFO: My Test File2 - 5
2016-07-18 09:58:19,243 : INFO: My Test File1 - 1
2016-07-18 09:58:19,244 : INFO: My Test File1 - 2
2016-07-18 09:58:19,255 : INFO: My Test File1 - 3
2016-07-18 09:58:19,255 : INFO: My Test File1 - 4
2016-07-18 09:58:19,258 : INFO: My Test File1 - 5
2016-07-18 09:58:49,685 : INFO: My Test File2 - 6
2016-07-18 09:58:49,686 : INFO: My Test File2 - 7
2016-07-18 09:58:49,686 : INFO: My Test File2 - 8

The below is the code which I have wrote to merge the 2 file.

private static final String DATE_FORMAT_REGEX = "(\\d{4})-(\\d{2})-(\\d{2}) (\\d{2}):(\\d{2}):(\\d{2}).*";
private static final Pattern DATE_FORMAT_PATTERN = Pattern.compile(DATE_FORMAT_REGEX);
private static final String COLON = " : ";
// 2016-07-18 09:57:09,674
private static final String DATE_PATTERN = "yyyy-MM-dd HH:mm:ss,SSS";
File file1 = new File("C:\\Users\\Admin\\Downloads\\log.log");
File file2 = new File("C:\\Users\\Admin\\Downloads\\log2.log");

LineIterator it = FileUtils.lineIterator(file1,"UTF-8");
LineIterator it1 = FileUtils.lineIterator(file2,"UTF-8");
PrintWriter writer = new PrintWriter("C:\\logMerge.txt", "UTF-8");
SimpleDateFormat sdf = new SimpleDateFormat(DATE_PATTERN);
      try {
            while (it.hasNext()) {
            String line = it.nextLine();
                    String date = line.split(COLON)[0].trim();
                    Matcher matcher = DATE_FORMAT_PATTERN.matcher(date);
                    if (matcher.matches()) {
                        System.out.println(line);
                        while (it1.hasNext()) {
                            String line1 = it1.nextLine();
                            String date1 = line1.split(COLON)[0].trim();
                            Matcher matcher1 = DATE_FORMAT_PATTERN.matcher(date1);
                            if (matcher1.matches()) {
                                Date convertedDate = sdf.parse(date);
                                Date convertedDate1 = sdf.parse(date1);
                                if (convertedDate.before(convertedDate1)) {

                                    writer.println(line);
                                    break;
                                }
                                else if (convertedDate1.before(convertedDate)) {
                                    writer.println(line1);
                                }
                                else {
                                    writer.println(line1);
                                }
                            }
                            else {
                                writer.println(line1);
                            }
                        }   
                    }
                    else {
                        writer.println(line);
                    }
                }
            }
            catch (ParseException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            finally {
                it.close();
            }

But with the above code I am not able to merge both the record. Any help is appreciated.

Edit:

Actual result

2016-07-18 09:57:09,674 : INFO: My Test File2 - 1
2016-07-18 09:57:09,674 : INFO: My Test File2 - 2
2016-07-18 09:57:09,679 : INFO: My Test File2 - 3
2016-07-18 09:57:09,679 : INFO: My Test File2 - 4
2016-07-18 09:57:09,680 : INFO: My Test File2 - 5
2016-07-18 09:58:49,685 : INFO: My Test File2 - 6
2016-07-18 09:58:49,686 : INFO: My Test File2 - 7
2016-07-18 09:58:49,686 : INFO: My Test File2 - 8

`String date = line.split(COLON)[0].trim();` Why bother splitting? Date/Time has a fixed length from start of line ... — Fildor, Aug 22 '16 at 07:51
@Fildor: The line is '2016-07-18 09:58:19,243 : INFO: My Test File1 - 1'. In order to get the date-time, need to split the date by colon (:). — Unknown, Aug 22 '16 at 08:01

Jean-François Fabre · Answer 1 · 2016-08-22T08:56:29.983

3

I wouldn't even bother parsing the dates. A simple alphanumerical sort using sort would provide the correct result since the log writer has the nice idea of writing year-month-day format which matches alphanum format (otherwise a date parsing would have been necessary)

MSYS/linux (using GNU sort)

sort File1 File2 > Sorted.txt

Windows CMD-line (using Windows sort):

(type File1 & type File2) | sort > Sorted.txt

unless you absolutely want to bother Java for that, but since you are writing a main program, I suspect that it is equivalent to the cmdline solution.

edited Aug 22 '16 at 08:56

answered Aug 22 '16 at 07:42

Jean-François Fabre

137,073
23
153
219

I like this idea, but it would not necessarily keep order of 2 lines with same timestamp, will it? – Fildor Aug 22 '16 at 07:47
order? you mean in the same log? Depends on if the sort algorithm is stable. If it is stable, then yes order is preserved. – Jean-François Fabre Aug 22 '16 at 07:51
That's my point: sorting alphanumerically on the whole line won't be stable in regard to only the timestamp. – Fildor Aug 22 '16 at 07:52
you are right. If that matters the "zig-zag" method from Sanjeev works fine since both files are already sorted internally the required way. On the other hand my method needs 0 coding line. All the question is does it matter if an event occurring at the same millisecond is printed before or after? – Jean-François Fabre Aug 22 '16 at 07:56
Exactly. Maybe you should add this to your answer to improve it. As I said, I after all like the code-less approach of it. – Fildor Aug 22 '16 at 07:58
When I run the command type log.log.0 & type log2.log | sort > merge.txt, It only gives the result of log2.log in merge.txt – MMA Aug 22 '16 at 08:40

Unknown · Accepted Answer · 2016-08-24T15:36:37.173

IMHO the basic algorithm is the following:

Get the FileList.
Create BufferedReader list with respect to the file.
Read the 1st line of each line and store it in another list.
Store the date from the 1st line of each file.
While BufferedReader's size is not zero then, (repeat the process from 6 - 12 if 5 is true)
Take the index of minimum date from dateList
Get the Line with the index you got in the previous step (lineToWrite) and then write the lineToWrite.
Get the buffered reader with the index.
If the BR is not null then read the line.
If line is not equal to null then remove the lineList from the index and add the line to index. If line is null then remove the min indexed line from lineList,dateList,BufferedReader list. Do BufferedReader--.
If the line length is greater than 23 (yyyy-MM-dd HH:mm:ss,SSS) then take the first 23 String from the line. Check if the date matches to the pattern, if matches then add the date to dateList
Repeat the process from Step5

Here is the code which does combine the log files.

    public static void main(String[] args) throws Exception {
    final String DATE_FORMAT_REGEX = "(\\d{4})-(\\d{2})-(\\d{2}) (\\d{2}):(\\d{2}):(\\d{2}).*";
    final Pattern DATE_FORMAT_PATTERN = Pattern.compile(DATE_FORMAT_REGEX);
    final String DATE_PATTERN = "yyyy-MM-dd HH:mm:ss,SSS";
    SimpleDateFormat sdf = new SimpleDateFormat(DATE_PATTERN);
    String file1 = "C:\\templog\\myLog1.log";
    String file2 = "C:\\templog\\myLog2.log";
    PrintWriter writer = new PrintWriter("C:\\templog\\expectedLog.txt", "UTF-8");
    // 1) Get the file list
    List<String> fileFile = Arrays.asList(file1, file2);
    ArrayList<BufferedReader> readerIndex = new ArrayList<BufferedReader>();
    ArrayList<String> lineList = new ArrayList<String>();
    ArrayList<Date> dateList = new ArrayList<Date>();
    for (String file : fileFile) {
        // 2) Create BufferedReader list with respect to the file.
        BufferedReader br1 = new BufferedReader(new FileReader(file));
        readerIndex.add(br1);
        // 3) Read the 1st line of each line and store it in another list.
        String line = br1.readLine();
        lineList.add(line);
        // 4) Store the date from the 1st line of each file.
        String date = line.substring(0, 23);
        Date convertedDate = sdf.parse(date);
        dateList.add(convertedDate);
    }
    int index = readerIndex.size();
    // 5) While BufferedReader's size is not zero then,
    while (index > 0) {
        // 6) Take the index of minimum date from dateList
        int indexMin = minDateIndex(dateList);
        // 7) Get the Line with the index you got in the previous step (lineToWrite).
        String lineToWrite = lineList.get(indexMin);
        writer.println(lineToWrite);
        // 8) Get the buffered reader with the index.
        BufferedReader br1 = readerIndex.get(indexMin);
        if (br1 != null) {
            //  9) If the BR is not null then read the line.
            String line = br1.readLine();
            if (line != null) {
                // 10)If line is not equal to null then remove the lineList from the index and add the line to index.
                lineList.remove(indexMin);
                lineList.add(indexMin, line);
                if (line.length() > 23) {
                    // 11)If the line length is greater than 23 (yyyy-MM-dd HH:mm:ss,SSS) then take the first 23 String from the line.
                    String date = line.substring(0, 23);
                    Matcher matcher = DATE_FORMAT_PATTERN.matcher(date);
                    if (matcher.matches()) {
                        // 12)Check if the date matches to the pattern, if matches then add the date to dateList
                        Date convertedDate = sdf.parse(date);
                        dateList.remove(indexMin);
                        dateList.add(indexMin, convertedDate);
                    }
                }
            } else {
                //If line is null then remove the min indexed line from lineList,dateList,BufferedReader list. Do BufferedReader--.
                lineList.remove(indexMin);
                dateList.remove(indexMin);
                readerIndex.remove(indexMin);
                br1.close();
                index--;
            }
        }
    }
    writer.close();
}

private static int minDateIndex(ArrayList<Date> dateList) {
    // return index of min date
    Date minDate = dateList.get(0);
    int minIndex = 0;
    for (int i = 1; i < dateList.size(); i++) {
        Date currentDate = dateList.get(i);
        if (minDate != null) {
            if (currentDate != null) {
                if (minDate.after(currentDate)) {
                    // We have a new min
                    minDate = dateList.get(i);
                    minIndex = i;
                } else {
                    // we keep current min
                }
            } else {
                // we keep current min
            }
        } else {
            minDate = currentDate;
            minIndex = i;
        }
    }
    return minIndex;
}

Test:

myLog1.log

2016-08-24 16:32:13,888[main] INFO  org.hibernate.cfg.Environment - HHH000206: hibernate.properties not found
2016-08-24 16:40:13,888 [main] INFO  o.h.tool.hbm2ddl.SchemaUpdate - HHH000396: Updating schema

myLog2.log

2016-08-24 16:33:13,888 : SEVERE : http-bio-8983-exec-3 : StandardWrapperValve.invoke :    Servlet.service() for servlet [Faces Servlet] in context with path [/TestApp] threw exception [Cannot create a session after the response has been committed] with root cause
java.lang.IllegalStateException: Cannot create a session after the response has been committed
    at org.apache.catalina.connector.Request.doGetSession(Request.java:2882)
    at org.apache.catalina.connector.Request.getSession(Request.java:2316)
    at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:898)
    at com.sun.faces.context.ExternalContextImpl.getSession(ExternalContextImpl.java:155)
    at com.sun.faces.mgbean.BeanManager$ScopeManager$SessionScopeHandler.handle(BeanManager.java:575)
    at com.sun.faces.mgbean.BeanManager$ScopeManager.pushToScope(BeanManager.java:458)
    at com.sun.faces.mgbean.BeanManager.createAndPush(BeanManager.java:410)
    at com.sun.faces.mgbean.BeanManager.create(BeanManager.java:269)
    at com.sun.faces.el.ManagedBeanELResolver.resolveBean(ManagedBeanELResolver.java:244)
    at com.sun.faces.el.ManagedBeanELResolver.getValue(ManagedBeanELResolver.java:116)
    at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
    at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
    at org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:72)
    at org.apache.el.parser.AstValue.getTarget(AstValue.java:94)
    at org.apache.el.parser.AstValue.setValue(AstValue.java:210)
    at org.apache.el.ValueExpressionImpl.setValue(ValueExpressionImpl.java:253)
    at org.apache.jasper.el.JspValueExpression.setValue(JspValueExpression.java:89)
    at javax.faces.component.UIComponent.processEvent(UIComponent.java:2185)
    at com.sun.faces.lifecycle.RestoreViewPhase$1.visit(RestoreViewPhase.java:276)
    at com.sun.faces.component.visit.FullVisitContext.invokeVisitCallback(FullVisitContext.java:151)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1446)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIForm.visitTree(UIForm.java:333)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at com.sun.faces.component.visit.VisitUtils.doFullNonIteratingVisit(VisitUtils.java:75)
    at com.sun.faces.lifecycle.RestoreViewPhase.deliverPostRestoreStateEvent(RestoreViewPhase.java:271)
    at com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:251)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:111)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:312)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:349)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.ow2.jonas.web.tomcat7.CheckOpenResourcesValve.invoke(CheckOpenResourcesValve.java:69)
    at org.ow2.jonas.web.tomcat7.tx.TransactionValve.invoke(TransactionValve.java:89)
    at org.ow2.jonas.web.tomcat7.ResetAuthenticationValve.invoke(ResetAuthenticationValve.java:95)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.ow2.jonas.web.tomcat7.versioning.VersioningValve.invoke(VersioningValve.java:105)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)

After running the program in expectedLog.txt

2016-08-24 16:32:13,888[main] INFO  org.hibernate.cfg.Environment - HHH000206: hibernate.properties not found
2016-08-24 16:33:13,888 : SEVERE : http-bio-8983-exec-3 : StandardWrapperValve.invoke :    Servlet.service() for servlet [Faces Servlet] in context with path [/TestApp] threw exception [Cannot create a session after the response has been committed] with root cause
java.lang.IllegalStateException: Cannot create a session after the response has been committed
    at org.apache.catalina.connector.Request.doGetSession(Request.java:2882)
    at org.apache.catalina.connector.Request.getSession(Request.java:2316)
    at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:898)
    at com.sun.faces.context.ExternalContextImpl.getSession(ExternalContextImpl.java:155)
    at com.sun.faces.mgbean.BeanManager$ScopeManager$SessionScopeHandler.handle(BeanManager.java:575)
    at com.sun.faces.mgbean.BeanManager$ScopeManager.pushToScope(BeanManager.java:458)
    at com.sun.faces.mgbean.BeanManager.createAndPush(BeanManager.java:410)
    at com.sun.faces.mgbean.BeanManager.create(BeanManager.java:269)
    at com.sun.faces.el.ManagedBeanELResolver.resolveBean(ManagedBeanELResolver.java:244)
    at com.sun.faces.el.ManagedBeanELResolver.getValue(ManagedBeanELResolver.java:116)
    at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
    at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
    at org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:72)
    at org.apache.el.parser.AstValue.getTarget(AstValue.java:94)
    at org.apache.el.parser.AstValue.setValue(AstValue.java:210)
    at org.apache.el.ValueExpressionImpl.setValue(ValueExpressionImpl.java:253)
    at org.apache.jasper.el.JspValueExpression.setValue(JspValueExpression.java:89)
    at javax.faces.component.UIComponent.processEvent(UIComponent.java:2185)
    at com.sun.faces.lifecycle.RestoreViewPhase$1.visit(RestoreViewPhase.java:276)
    at com.sun.faces.component.visit.FullVisitContext.invokeVisitCallback(FullVisitContext.java:151)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1446)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIForm.visitTree(UIForm.java:333)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at com.sun.faces.component.visit.VisitUtils.doFullNonIteratingVisit(VisitUtils.java:75)
    at com.sun.faces.lifecycle.RestoreViewPhase.deliverPostRestoreStateEvent(RestoreViewPhase.java:271)
    at com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:251)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:111)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:312)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:349)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.ow2.jonas.web.tomcat7.CheckOpenResourcesValve.invoke(CheckOpenResourcesValve.java:69)
    at org.ow2.jonas.web.tomcat7.tx.TransactionValve.invoke(TransactionValve.java:89)
    at org.ow2.jonas.web.tomcat7.ResetAuthenticationValve.invoke(ResetAuthenticationValve.java:95)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.ow2.jonas.web.tomcat7.versioning.VersioningValve.invoke(VersioningValve.java:105)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
2016-08-24 16:40:13,888 [main] INFO  o.h.tool.hbm2ddl.SchemaUpdate - HHH000396: Updating schema

Hope it helps !

score 1 · Answer 3 · answered Aug 22 '16 at 07:38

I didn't take a deep look at the code but if it's possible for you to sort the files first and then read them in zig-zag manner, you can achieve what you are looking for.

By zig-zag I meant -

Read first lines from both files. Split them and find the timestamp, say T1 and T2.
If T1 is < T2 then write T1 line from 1st file (in an output file)
Now read the next line from the 1st file. Suppose this time it was T3.
Check if T3 < T2 then write T3 line in output file and read the next line from 1st file only. If T2 was < T3 then write T2 line in output file and read the next line from 2nd file.

Have fun !!!

score 1 · Answer 4 · edited May 23 '17 at 12:33

1

I think there is a best way to do that problem:

Parse each line as object 'MyLog' (implements Comparator) with properties : date, info...

Implement the method 'compare' between dates.

    @Override
    public int compare(Date o1, Date o2) {
        return o1.compareTo(o2);
    }

Add this object to a ArrayList
Use method sort.

Maybe this link is your answer: https://stackoverflow.com/a/5927408/2269677

edited May 23 '17 at 12:33

Community

1
1

answered Aug 22 '16 at 07:46

MrElephant

302
4
26

1

From experience, log files tend to be huge. I doubt keeping two of them in memory is a good idea. – Fildor Aug 22 '16 at 07:48
okay, but how to sort data without having all the data in memory ? – Jean-François Fabre Aug 22 '16 at 07:49
@Jean-FrançoisFabre https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#lines-java.nio.file.Path-java.nio.charset.Charset- "Unlike readAllLines, this method does not read all lines into a List, but instead **populates lazily as the stream is consumed**." – Fildor Aug 22 '16 at 07:56
okay but to be able to sort you need all the data in memory. That's one of the hard issues of sorting. Sanjeev solution is the best IMHO since it relies on the list being sorted independently and his zigzag method works (files can be read line by line no surprises) – Jean-François Fabre Aug 22 '16 at 07:59
@Jean-FrançoisFabre If we can assume that the two original logfile already are sorted, it is enough to look at and compare 2 entries at a time. So it is in fact exactly why you do not need to keep everything in memory. – Fildor Aug 22 '16 at 08:12
This does not work, since the log contain lines without a timestamp (e.g. java stack traces). – MMA Aug 22 '16 at 08:38

Java : Merging multiple log files by date

4 Answers4