1

I have a log file needed to extract and I'm using a regex way in python to extract the data but i can't figure out the correct condition to extract the bunch of text.

Right now for the current condition was selecting the undesired text but still able to get the portion of the text i needed.

Hope someone can help me.

Here is the regex condition that im using:

(Item|Chamber|DC|Etch|Depo)\s+[^:].+

here is the url where i test it:

http://myregexp.com/

sample log file:

Nexus Cluster Tool Controller     Process Data Log

Version CTCflex V2.102.00

Data Log Name: 2017-03-01_001_A2B

Tool ID: B

User ID: Maintenance-C                   

Run Number: 01

Number of Steps:  0

Schedule: DLCX_ARO2_6535_15A_PORD2=PRD

Run Date: 2017-03-01 (Y M D)

Run Start Time: 00:20:56

Wafer from LLA Slot 02 (Wafer ID:2017-03-01_001_A2B)


Module


Module              : PM3 (DLCXIC)

PM Software Version : PMflex2008 (DLCX) Version 02.104.00.00

Process Name        : PM3_ARO2_Ni35A_fillin_2AD2.prc





Step Number     : 1

Step Name       : CouponcheckD2

Step Time       : 00:00:15.000

Run Time        : 01Mar17 00:22:39

Chamber Pressure    : 1.6e-006



Depo Source-DC Source Parameters

    Use Source                : False

    Continuous Beam           : False

    Depo ControlMode          : Continuous

    Depo RegulationMode       : Continuous

    Depo RampTime       (Sec) :     0.000

    Depo Power            (W) :     0.0

    Depo Voltage          (V) :     0.0

    Depo JoulesQuantity       :     0.0

    Depo PulseMode            : False

    Depo PulseFrequency  (kHz):     0.0

    Depo PulseReverseTime (uS):     0.0

     Depo ArcDetectEnabled     : False

     Depo ArcCountAlarm        : 10000

     Depo ArcResetDelay        :     1

    Depo Gas 1 Flow    (SCCM) :     0.0

    Depo Gas 2 Flow    (SCCM) :     0.0



Etch Source Parameters

    Use Source                : False

    Beam At Start             : Shutoff

    Beam At End               : Shutoff

    Neutralizer On            : False

    Continuous Beam           : False

    Beam Voltage          (V) :     0.0

    Beam Current         (mA) :     0.0

    Suppressor Voltage    (V) :     0.0

    Incident RF Power     (W) :     0.0

    K Factor                  :     0.0

    Etch Gas 1 Flow    (SCCM) :     0.0

    Etch Gas 2 Flow    (SCCM) :     0.0

    Etch Gas 3 Flow    (SCCM) :     0.0

    PBN Gas Flow       (SCCM) :     0.0

    Source Magnet             : 

        Use Source Magnet     : False

        Current           (A) :     0.000



Control Parameters

    Step Ends By              : Time

    Time Total Seconds        :    15.000

    End Pressure Setpoint     : 2.0e-007

    Power Down Sources        : False



Fixture Parameters

    Tilt Angle                : -55.0

    Rotate Fixture            : True

    Rotation Speed      (RPM) : 30.0



Shutter Parameters

    Depo Shutter At Start     : Closed

    Depo Shutter At Process   : Closed

    Depo Shutter At End       : Closed

    Etch Shutter At Start     : Open

    Etch Shutter At Process   : Open

    Etch Shutter At End       : Open



Data Tracing Parameters

    Tracing Enabled           : True

    Interval            (Sec) :    1



Ellipsometer Parameters

    Monitor                   : True

    Step Type                 : View

    Points or Angstroms       :     4.0

    Model Name                : Model1





Process Results

    Step Elapsed Time         : 00:00:15.000

    Step Remaining Time       : 00:00:00.000





Process Statistics



Item Name                                       Minimum     Maximum        Mean      StdDev      Median

Chamber Pressure                                1.6e-006    1.7e-006    1.6e-006    2.5e-008    1.6e-006

DC Current Current  (mA)                           0.010       0.010       0.010       0.000       0.010

DC Current Impedance                               0.000       0.000       0.000       0.000       0.000

DC Current Power (W)                               0.000       0.000       0.000       0.000       0.000

DC Current Voltage (V)                             0.000       0.000       0.000       0.000       0.000

Etch Beam Voltage (V)                              4.746       4.746       4.746       0.000       4.746

Etch Beam Current (mA)                             3.900       3.900       3.900       0.000       3.900

Etch Suppressor Voltage (V)                        1.409       1.593       1.438       0.069       1.409

Etch Suppressor Current (mA)                       0.000       0.000       0.000       0.000       0.000

Etch Forward RF Power (W)                          0.000       0.000       0.000       0.000       0.000

Etch Reflected RF Power (W)                        0.000       0.000       0.000       0.000       0.000

Etch PBN Filament (A)                              0.104       0.110       0.105       0.002       0.104

Etch PBN Body (A)                                  0.001       0.002       0.002       0.001       0.002

Etch Source Magnet (A)                             0.003       0.004       0.004       0.000       0.004

Depo Gas 1 Actual Flow (SCCM)                      0.679       0.679       0.679       0.000       0.679

Depo Gas 2 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

Etch Gas 1 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

Etch Gas 2 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

Etch Gas 3 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

Etch PBN Gas Actual Flow (SCCM)                    0.000       0.000       0.000       0.000       0.000







Events Which Occurred During The Process:

03/01/2017 00:22:10.279   U:Event:   Current Process Recipe Name = PM3_ARO2_Ni35A_fillin_2AD2.prc [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.279   U:Event:   Current Step Number = 1 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.279   U:Event:   Current Step = CouponcheckD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.281   U:Event:   Current Step Number = 2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.281   U:Event:   Current Step = ArWarmupD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.283   U:Event:   Current Step Number = 3 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.283   U:Event:   Current Step = ArStableD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.284   U:Event:   Current Step Number = 4 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.284   U:Event:   Current Step = ArO2WarmupD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.286   U:Event:   Current Step Number = 5 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.286   U:Event:   Current Step = ArO2_EtchD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.288   U:Event:   Current Step Number = 6 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.288   U:Event:   Current Step = ArO2PurgeD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.290   U:Event:   Current Step Number = 7 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.290   U:Event:   Current Step = EtchwarmD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.292   U:Event:   Current Step Number = 8 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.292   U:Event:   Current Step = EtchstableD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.294   U:Event:   Current Step Number = 9 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.294   U:Event:   Current Step = PTREtch_6535D2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.296   U:Event:   Current Step Number = 10 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.296   U:Event:   Current Step = SiNstartD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.298   U:Event:   Current Step Number = 11 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.298   U:Event:   Current Step = SiNfillinD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.300   U:Event:   Current Step Number = 12 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.300   U:Event:   Current Step = SiNviewD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.301   U:Event:   Current Step Number = 13 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.302   U:Event:   Current Step = EtchbackNewD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.303   U:Event:   Current Step Number = 14 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.303   U:Event:   Current Step = SiNviewD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.305   U:Event:   Current Step Number = 15 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.305   U:Event:   Current Step = SiNstartD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.306   U:Event:   Current Step Number = 16 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.306   U:Event:   Current Step = SiNdepo_2AD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.308   U:Event:   Current Step Number = 17 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.308   U:Event:   Current Step = SiNviewD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.472   U:Event:   Current Step Number = 1 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.472   U:Event:   Current Step = CouponcheckD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.475   U:Event:   Waiting for Process Pressure [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.660   U:Event:   Waiting for Process Start Pressure [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.821   U:Event:   Process Starting [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:10.821   U:Event:   Waiting for Source PreStart [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:13.743   U:Event:   Clamping Wafer for Processing [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:22.500   U:Event:   Wafer is clamped for Processing [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:22.955   U:Event:   Configuring Rotation Axis to Step Configuration [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:23.001   U:Event:   Sequence MotionToInitialPosition - Moving Fixture to process position [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:42.232   U:Event:   Waiting for Step to Complete [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:57.041   U:Event:   Step Completed. [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:58.002   U:Event:   Current Step Number = 2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:58.003   U:Event:   Current Step = ArWarmupD2 [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:59.312   U:Event:   Configuring Rotation Axis to Step Configuration [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:22:59.398   U:Event:   Sequence MotionToInitialPosition - Moving Fixture to -135 degrees [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:09.269   U:Event:   Sequence MotionToInitialPosition - Moving Etch Shutter to Initial Position [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:15.622   U:Event:   SourceManager RF Generator Status Update: External Interlock - OK [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:15.758   U:Event:   Etch : Ramping Started [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:17.767   U:Event:   Etch: Gas is stable... [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:25.779   U:Event:   Etch: Turning On RF [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:39.709   U:Event:   Etch: Igniting Beam... [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:40.797   U:Event:   Etch: Waiting for the beam to stabilize/regulate... [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:44.370   U:Event:   Etch: Beam stabilized/regulated. [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:46.120   U:Event:   Source 'SourceManager_Etch' has indicated an error. [LLA_02] [2017-03-01_001_A2B]











Alarms Which Occurred During The Process:

03/01/2017 00:23:46.013   U:Alarm:   Etch: Maximum Suppressor Current Limit Detected (SI=0.298)! [LLA_02] [2017-03-01_001_A2B]

03/01/2017 00:23:46.120   U:Alarm:   Sequence IgniteBeam - Failed to Ignite all Sources [LLA_02] [2017-03-01_001_A2B]





Data Log Event: Process Aborted

Process has been abnormally terminated. - 03/01/2017 00:23:46.251

    Processing should be continued from:

        Process Name:   PM3_ARO2_Ni35A_fillin_2AD2.prc

        Step Name:      ArWarmupD2

        Step Number:    2

        Elapsed Time:   0.000

        Remaining Time: 60.000

        Total Steps:    17

Here is the sample portion i want to extract per step:

 Item Name                                       Minimum     Maximum        Mean      StdDev      Median

    Chamber Pressure                                1.6e-006    1.7e-006    1.6e-006    2.5e-008    1.6e-006

    DC Current Current  (mA)                           0.010       0.010       0.010       0.000       0.010

    DC Current Impedance                               0.000       0.000       0.000       0.000       0.000

    DC Current Power (W)                               0.000       0.000       0.000       0.000       0.000

    DC Current Voltage (V)                             0.000       0.000       0.000       0.000       0.000

    Etch Beam Voltage (V)                              4.746       4.746       4.746       0.000       4.746

    Etch Beam Current (mA)                             3.900       3.900       3.900       0.000       3.900

    Etch Suppressor Voltage (V)                        1.409       1.593       1.438       0.069       1.409

    Etch Suppressor Current (mA)                       0.000       0.000       0.000       0.000       0.000

    Etch Forward RF Power (W)                          0.000       0.000       0.000       0.000       0.000

    Etch Reflected RF Power (W)                        0.000       0.000       0.000       0.000       0.000

    Etch PBN Filament (A)                              0.104       0.110       0.105       0.002       0.104

    Etch PBN Body (A)                                  0.001       0.002       0.002       0.001       0.002

    Etch Source Magnet (A)                             0.003       0.004       0.004       0.000       0.004

    Depo Gas 1 Actual Flow (SCCM)                      0.679       0.679       0.679       0.000       0.679

    Depo Gas 2 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

    Etch Gas 1 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

    Etch Gas 2 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

    Etch Gas 3 Actual Flow (SCCM)                      0.000       0.000       0.000       0.000       0.000

    Etch PBN Gas Actual Flow (SCCM)                    0.000       0.000       0.000       0.000       0.000
iamcoder
  • 529
  • 2
  • 4
  • 23
  • Could you show the actual (full) relevant Python code? Are you using `re.search` or `re.match`, or something else? –  Mar 31 '17 at 06:45
  • hi im using re.match for the current task...i'm i doing it correctly? – iamcoder Mar 31 '17 at 07:05
  • `re.match` only matches at the beginning of a line. See the [relevant documentation](https://docs.python.org/3/library/re.html#search-vs-match). You'll want `re.search` instead. –  Mar 31 '17 at 08:36
  • Possible duplicate of [What is the difference between Python's re.search and re.match?](http://stackoverflow.com/questions/180986/what-is-the-difference-between-pythons-re-search-and-re-match) –  Mar 31 '17 at 08:39

1 Answers1

0

Regex by your request is:

(?P<Item>(Item|Chamber|DC|Etch|Depo)(\s{1,3}\w*){1,5}(\((mA|A|W|V|SCCM)\))?)\s{4,}(?P<minimum>[0-9]\.[0-9e-]+)\s+(?P<maximum>[0-9]\.[0-9e-]+)\s+(?P<mean>[0-9]\.[0-9e-]+)\s+(?P<stddev>[0-9]\.[0-9e-]+)\s+(?P<median>[0-9]\.[0-9e-]+)

This way you will have 6 regex named groups: item, minimum, maximum mean, stddev and median which you can extract in python code by groupdict() method.

Also don't forget to enable MULTILINE flag if you're using web sites for check your regex (in python it's re.MULTILINE).

This is how groupdict() method works:

import re
h_regex = re.match(r'(?P<hello>H)', 'Hello', re.MULTILINE)
print(h_regex.groupdict())

Good luck and thanks for such complex and interesting question! :)

You can check it with details in http://pythex.org/, it's more convenient than http://myregexp.com/ in my opinion.

UPD: python 3 script (just get correct input filename): 

import re

input_filename = 'log.txt'
pattern = re.compile(r'(?P<item>(Item|Chamber|DC|Etch|Depo)(\s{1,3}\w*){1,5}(\((mA|A|W|V|SCCM)\))?)\s{4,}(?P<minimum>[0-9]\.[0-9e-]+)\s+(?P<maximum>[0-9]\.[0-9e-]+)\s+(?P<mean>[0-9]\.[0-9e-]+)\s+(?P<stddev>[0-9]\.[0-9e-]+)\s+(?P<median>[0-9]\.[0-9e-]+)')
matched = []

with open(input_filename, 'r') as f:
    log_content = f.readlines()

for line in log_content:
    result = re.match(pattern, line)
    if result:
        matched.append(result)

for line in matched:
    print(line.groupdict())
valex
  • 5,163
  • 2
  • 33
  • 40
  • hi thank you for your detailed response but i'm not able to compile it correctly? h_regex = re.match('(?P(Item|Chamber|DC|Etch|Depo)(\s{1,3}\w*){1,5}(\((mA|A|W|V|SCCM)\))?)\s{4,}(?P[0-9]\.[0-9e-]+)\s+(?P[0-9]\.[0-9e-]+)\s+(?P[0-9]\.[0-9e-]+)\s+(?P[0-9]\.[0-9e-]+)\s+(?P[0-9]\.[0-9e-]+)', myString, re.MULTILINE) print(h_regex.groupdict()) – iamcoder Mar 31 '17 at 10:50