0

i have to make a database access fast which contains large data of unique index mstr_nbr how can i make it fast. in which get first mstr taking 0ms and get next mstr takes most of the time 0ms but sometimes takes 1ms means in 180000 for each mstr runs 12000 runs takes 1 ms which is increasing the time and if no of serial loop increases every time 12000ms second also increase this is a webspeed generated webpage how can i make it fast...anyone help

            DEFINE QUERY Mstr FOR mstr scrolling.
            OPEN QUERY Mstr FOR EACH mstr no-lock 
                        where (Mstr_status = "close" or Mstr_status = "closed").
            FOR EACH serial
                WHERE (serial_pallet = f_pallet AND serial_f_chr11 <> "BOX") 
                       or (serial_key begins f_pallet)
                    NO-LOCK    BREAK BY serial_pallet by serial_parent by serial__chr11 QUERY-TUNING(LOOKAHEAD CACHE-SIZE 32768 DEBUG EXTENDED):
                GET FIRST Mstr.
                define variable roID as rowid no-undo.
                roID = rowid(mstr).
                DO WHILE NOT QUERY-OFF-END('Mstr'):
                    for each det fields(detnbr detmodel detlot detqty) no-lock
                    where (detnbr = mstr_nbr) and (detmodel = serial_parent and detlot = serial__chr11):
                        tinspected = tinspected + detqty.
                    end. /* for each */
                    GET NEXT Mstr.
                END.
                reposition mstr to rowid roID.
            end.

index for mstr table

index-name     field-name

badgenew       1 badgenew      Active    
datenew        1 datenew       Active    
nbridx         1 nbr           Unique Primary Active                   
pallet         1 pallet        Active    
proddesc       1 proddesc      Active    
prodline       1 prodline      Active    
status         1 status        Active    
type           1 type          Active

indexes for table serial:

 actual_prod_day    1 dte04         2 serial_chr01 Active
 actual_prod_line   1 serial_pallet 2 serial_dte04 3 serial_chr01 4 serial_line Active
 pallet_prod        1 serial_pallet 2 serial_dte04 Active
 pallet_prod_line   1 serial_pallet 2 serial_dte04 3 serial_line Active
 
 serial_chr01       1 serial_chr01 Active
 serial_chr05       1 serial_chr05 Active
 serial_chr06       1 serial_chr06 Active
 serial_chr11       1 serial_chr11 Active
 serial_chr14       1 serial_chr14 Active
 serial_dte04       1 serial_dte04 Active
 serial_int01       1 serial_int01 Active
 
 serial_line        1 serial_line Active
 
 serial_pallet      1 serial_pallet Active
 
 serial_parent      1 serial_parent Active
 
 serial_serial__key 1 serial_serial__key 2 serial_parent Unique Primary Active

serial_pallet serial_key and serial_c11 all are character data type

indexes for table det:

detidx      1   detnbr 2    detpallet 3 detprodline 4   detbox 5    detlot 6 detshift Unique Primary Active  
detlot      1   detlot Active    
detmodel    1   detmodel Active  
detnbr      1   detnbr Active    
detpallet   1   detpallet Active     
detprodline 1   detprodline Active
Tom Bascom
  • 13,405
  • 2
  • 27
  • 33
  • "how can i make it fast" - by ditching Progress? It's an old-world 4GL system that's since been discontinued. – Dai Jun 29 '20 at 10:14
  • More seriously though - check your indexes in the Data Dictionary tool. What version of Progress are you using? – Dai Jun 29 '20 at 10:14
  • Do you have to use Progress' 4GL for this? Have you thought about using Progress' SQL engine instead? – Dai Jun 29 '20 at 10:15
  • progress ver 11 yes i have to use because i work for a project which needs progress – kushal bhatia Jun 29 '20 at 10:21
  • 2
    Progress is not “discontinued”. It is very much alive and well. – Tom Bascom Jun 29 '20 at 11:48
  • Is the target db a progress db? You’re using query tuning hints, which often means that your target db is something else. – Tom Bascom Jun 29 '20 at 11:51
  • it is progress db ,i was just checking them forget to remove them sorry – kushal bhatia Jun 29 '20 at 11:52
  • What exact version of Progress. You said "11". That spans 11.0 to 11.7 over about 10 years. There are a lot of improvements along the way. Also, as Dai says, it is very important to know what indexes are available for these tables. You should add that information to your question. The data dictionary has an "indexes" report. It's ugly but it is informative. 3rd - what are your server startup and client connection parameters? Your code may be fine but if the db server or the connection isn't properly configured might be slowing things down substantially. – Tom Bascom Jun 29 '20 at 12:24
  • index for mstr table badgenew 1 badgenew Active datenew 1 datenew Active nbridx 1 nbr Unique Primary Active pallet 1 pallet Active proddesc 1 proddesc Active prodline 1 prodline Active status 1 status Active type 1 type Active except Mstr every table is working fine Mstr is taking 12sec for 1complete for each iteration and for version i can't do anything because i'm just wroking on whatever environment my company providing it's not in my hand – kushal bhatia Jun 29 '20 at 12:46
  • The specific Progress version matters. Sure it might be nice to upgrade and I will probably suggest that (unless you tell me that you are on 11.7.6). But it is also helpful because certain features, that might help you, are introduced with certain versions. It isn't very helpful to suggest that you use a feature that isn't available. The same goes for startup parameters. There might, for instance, be something easy to do if you are running 11.6 rather than 11.3. But if you don't need any help feel free to ignore requests for more information. – Tom Bascom Jun 29 '20 at 13:47
  • okay but can u suggest anything to improve my query anything you have experience i have seen many helpful posts from you means how can i choose index to improve my query or what changes i can do just some helpful advice and I'm using webspeed webtools its just showing database progress version 11 in data browser section – kushal bhatia Jun 29 '20 at 14:11
  • It is entirely possible that there is nothing wrong with your query but that some parameter setting may benefit you. That's why I keep asking about the version. But it's up to you. You either want help. Or you don't. – Tom Bascom Jun 29 '20 at 16:04
  • hello tom, i have found the version it is version 11.3 i know it's old thanks for your time and effort – kushal bhatia Jun 29 '20 at 18:08
  • Are you changing your answer about the type of db? On ProgressTalk you are claiming SQL Server 2014. Here you are saying 6.03. What is it really? – Tom Bascom Jun 30 '20 at 10:30

1 Answers1

0

Given what we know this is how I would code it:

define temp-table tt_mstr
  field mstr_nbr as integer                /* or whatever the proper data type is */
  index mstr_nbr-idx is primary unique     /* I am assuming mstr_nbr is unique    */
.

for each mstr no-lock
   where mstr.mstr_status = "close"
      or mstr.mstr_status = "closed":

  create tt_mstr.
  tt_mstr.mstr_nbr = mstr.mstr_nbr.  
      
end.

for each serial no-lock
   where ( serial_pallet = f_pallet and serial_f_chr11 <> "box" )       /* <> "box" is going to perform poorly, there may be better ways to do this     */
      or ( serial_key begins f_pallet ):

         /* break by serial_pallet by serial_parent by serial__chr11: ** this sort of pointless, you're just adding up "tinspected", the order and the break groups have no impact */  
  
  for each det fields( detnbr detmodel detlot detqty ) no-lock
     where detmodel = serial_parent and detlot = serial__chr11:

    find tt_mstr where tt_mstr.mstr_nbr = detnbr no-error.       
      if available tt_mstr then
        tinspected = tinspected + detqty.               
    
  end.
  
end.

Using a temp-table avoids refetching all of the "close" and "closed" records with every iteration of the "serial" table.

Maybe there is some context missing but the scrolling query and repositioning of the row seem pointless.

The selection of "serial" records does not look very efficient but I would need to know what indexes are available and what the data in serial_pallet serial_key and serial_c11 looks like. If there are just a few discrete values there may be better ways to write that.

You have single component indexes on each of the fields serial_pallet, serial_f_chr11, and serial_key. So there is not much help there.

This is speculation but if the number of discrete values in serial_f_chr11 is small you would probably be better off with a series of equality matches and OR. Suppose the valid values are BOX, JAR, BAG, and LOOSE. In that case, instead of:

where ( serial_pallet = f_pallet and serial_f_chr11 <> "box" )
   or ( serial_key begins f_pallet )

you could write:

where ( serial_pallet = f_pallet and serial_f_chr11 = "jar" )
   or ( serial_pallet = f_pallet and serial_f_chr11 = "bag" )
   or ( serial_pallet = f_pallet and serial_f_chr11 = "loose" )
   or ( serial_key begins f_pallet )

That would be even better if you have a composite index on serial_pallet + serial_f_chr11.

If the number of discrete valid serial_f_chr11 values is larger or if new values might get added then it would be better to add them to a temp-table and join on that instead.

Another option is that rather than loop over all of the master records to find matching details, select details that match the serials first. Then find the appropriate master record. That eliminates a whole layer of looping. But it depends on proper indexes in the "det" table. You must have an index that has serial_parent and detlot as leading components.

If mstr_nbr is not the same field as "nbr" (as shown in your index listing) then you need to build the TT and add an index on mstr_nbr. If, however, nbr is actually the same as mstr_nbr then you could skip the TT and directly query the database table efficiently.

Tom Bascom
  • 13,405
  • 2
  • 27
  • 33
  • Index : actual_prod_day 1 dte04 2 serial_chr01 Active actual_prod_line 1 serial_pallet 2 serial_dte04 3 serial_chr01 4 serial_line Active pallet_prod 1 serial_pallet 2 serial_dte04 Active pallet_prod_line 1 serial_pallet 2 serial_dte04 3 serial_line Active serial_chr01 1 serial_chr01 Active serial_chr05 1 serial_chr05 Active serial_chr06 1 serial_chr06 Active serial_chr11 1 serial_chr11 Active serial_chr14 1 serial_chr14 Active serial_dte04 1 serial_dte04 Active – kushal bhatia Jun 29 '20 at 14:48
  • serial_int01 1 serial_int01 Active serial_line 1 serial_line Active serial_pallet 1 serial_pallet Active serial_parent 1 serial_parent Active serial_serial__key 1 serial_serial__key 2 serial_parent Unique Primary Active serial_pallet serial_key and serial_c11 all are character data type – kushal bhatia Jun 29 '20 at 14:48
  • hello tom, i have found the version it is version 11.3 and sql server 6.03 i know it's old thanks for your time and effort – kushal bhatia Jun 30 '20 at 04:48
  • Thanks. 11.3 is very old and rules out some easy recommendations. Why is SQL server relevant? Earlier you said that it is a Progress database, not a data server. Are you changing that answer? – Tom Bascom Jun 30 '20 at 10:18
  • i'm using web speed webtools the data is stored in progress db . sql server is installed in my remote computer so i mentioned here – kushal bhatia Jun 30 '20 at 11:14
  • yes i tried but for each mstr no-lock where mstr.mstr_status = "close" or mstr.mstr_status = "closed": create tt_mstr. tt_mstr.mstr_nbr = mstr.mstr_nbr. end. this part still taking time – kushal bhatia Jun 30 '20 at 11:50
  • Yes, of course it does. But instead of doing a table scan with every iteration of your FOR EACH SERIAL loop it only does it once. That should be a significant overall improvement unless the FOR EACH SERIAL is only finding a single record. – Tom Bascom Jun 30 '20 at 12:19
  • I have modified the nested loops in my answer to eliminate looping over tt_mstr. That only makes sense if you have proper indexes on your "det" table. You haven't shared those so I don't know. – Tom Bascom Jun 30 '20 at 12:44
  • added index for table det sorry I'm pretty much new in progress stuff really sorry for the inconvenience – kushal bhatia Jun 30 '20 at 14:05
  • It looks like you do have useful indexes on detail. So the revised answer code _should_ work better than the original. There is still uncertainty about if the mstr_nbr field is named mstr_nbr or nbr. If the "nbr" field in the mstr indexes is the same as mstr_nbr then the temp-table is not needed. But if there really are two distinct fields you still need the temp-table. – Tom Bascom Jun 30 '20 at 15:38