3

I’ve been battling with a bug with cflocation ever since I upgraded from CF9 to CF10. I’ve worked on it a number of times but always give up but today it’s finally annoyed me enough that it’s time to solve the problem (I hope).

Firstly, apologies as I have tried to write a simple test case to reproduce this problem but so far I’ve been unable to reproduce it outside of original code.

I have a multi-step import process that should run overnight. The process is made up of about 10 files that chain together. When calling the process via a scheduled task a URL parameter (scheduledtask) is also passed. If the parameter does not exist, at the end of each step it stops and waits for the user to click a link to the next step. If the parameter does exist it uses cflocation to move to the next step.

Of all the steps, the 2nd one is the most complex and time consuming (data from multiple web service requests that then need joining, cleaning and inserting into a database). It’s this second step that has a problem after moving to CF10. The page seems to work fine as it’s getting to the very bottom of the page where the cflocation tag is but it never fires cflocation. I’ve added in a cfmail tag to email me as the cflocation is being called, which is always sent, but again cflocation doesn’t fire.

It can’t get much more basic than:

<cfif scheduledtask EQ "true">
   <cfmail from="xxxxx” to="xxxx” subject="About to call duplicate" type="text/html"> 
      <p>calling duplicate check - scheduledtask</p>
   </cfmail>

   <cflocation url="importDupCheck.cfm?scheduledtask=true" addtoken="false">
</cfif>

I know that there is a problem with cfflush and cflocation and I’ve already checked for that.

As a test I tried moving the above logic up the page to just after all the data is retrieved by the webservice but before it’s processed and this time the cflocation worked. I thought maybe it’s a timeout problem where the scheduled task engine has given up waiting even though the page is being allowed to fully load. I tried testing this by having a basic file that had: 

<cfset sleep(240000)>
 * 4 minutes

before it called cflocation but it worked fine.

The other steps in the process are working as expected with cflocation, just not this one. All this code runs fine under CF9, just failing with CF10.

Does anyone has some pointers on where I should be looking?

Regards Mark

* Update * So I’ve now worked out how to easily reproduce this in a few lines of code. Turns out it has nothing to do with scheduled tasks.

index.cfm

<h1>Testing cflocation</h1>

<cfloop index="i" from="1" to="7000" ><!--- For me it stops working once the loop goes beyond 6808 rows --->
    row <cfoutput>#i#</cfoutput>: abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789<br/>
</cfloop>

<p>Now loading resultpage via cflocation</p>
<cflocation url="resultpage.cfm" addtoken="false">

resultpage.cfm

<h1>Made it!</h1>
<p>Reload the <a href="index.cfm">first page</a></p>

In my testing I found that if I looped 6808 times it works but 6809 times fails. With 7000 records we are only talking about a 36 KB file….why is this failing only in CF10?

One last thing I should note in case it has an impact- everything I do is done via https.

James A Mohler
  • 11,060
  • 15
  • 46
  • 72
tHeSmUrF
  • 169
  • 1
  • 3
  • 10
  • I am surprised that `cflocation` works at all while running as a scheduled task. Basically all it does is send redirect headers back to the browser to request the subsequent page. The "browser" that is used for scheduled tasks is fairly limited. I don't have any references but I thought using `cflocation` within scheduled tasks was a no-no. I typically use a main `.cfm` page for my scheduled tasks and then have it call other pages (URLs) using the `cfhttp` tag. This also allows for setting different timeouts/error handling on different URLs that may be processing. I guess if it worked before?? – Miguel-F Oct 17 '13 at 15:52
  • HI Miguel-F. My assumption was that a scheduled task behaved similar to cfhttp in that it's running like it's a browser so it understands changes to http headers and the like. Nothing to back that up other than it seemed to work that way. Anyway, I've updated my original question as it turns out it has nothing to do with scheduled tasks. – tHeSmUrF Oct 18 '13 at 00:10

4 Answers4

3

Ok I worked it out

In CF Administrator

Maximum Output Buffer size 1024

After which a flush will occur which will stop cflocation working.

Increase it and it works.

Dale Fraser
  • 4,623
  • 7
  • 39
  • 76
  • Thank you Dale. In my test case above it's only generating a total of around 770KB so I'm a little confused why setting the buffer to higher than 1024KB works, but it does with the test case. Thank you (will also thank you in the cfaussie google group). – tHeSmUrF Nov 21 '13 at 03:50
2

I have no idea why your code as it stands doesn't work, I'm afraid. At a guess, it's down to ColdFusion 10's replacement of the scheduled task engine, and it not liking some vagary of client-side redirects.

However maybe you can bypass the situation anyhow.

ColdFusion 10's scheduler has a concept of chaining tasks, so that when a task finishes, another one is kicked off. You could conceivably revise your scheduler to leverage that instead?

Adam Cameron
  • 29,677
  • 4
  • 37
  • 78
  • 1
    I've been playing all afternoon and I no longer think it's related to the scheduler as I'm able to reproduce it in my browser. I'm really stuck as to why this is happening but I've found that part way down the page where I'm looping over an array that once it hits 80 loops the cflocation at the bottom of the code won't work. If I make the loop exit at 79 records, cflocation works. At first I thought it might have been something in the data causing a problem...but using a different set of data it stops at 77 records. I'm going to play some more and then update my question. This is weird. – tHeSmUrF Oct 17 '13 at 06:29
  • 1
    Got to be kidding me. You know what it is? It's the size of the page that is stopping it. When I'm looping the array it's outputting a single line for each record to the browser. If I remove the output, it runs fine. This is weird because even when it doesn't follow cflocation it's still displaying everything fine....just won't follow cflocation (which is 100s of lines of code below where the array loop finishes) and everything in those 100's of lines also display fine. This has given me an idea on how I might be able to make something to demo the problem. yah? – tHeSmUrF Oct 17 '13 at 06:51
  • 1
    Sounds like CF is spooling output down to the browser. As soon as the HTTP headers have been sent down, you cannot do CFLOCATION any more, as it is done via HTTP headers. – Adam Cameron Oct 17 '13 at 09:52
  • Hi Adam, I've updated my original question to include a way to reproduce this. Why would this change between CF9 and CF10? – tHeSmUrF Oct 18 '13 at 00:06
  • Oh! THAT is beautiful! – Phillip Senn Jan 22 '14 at 18:07
1

I had a similar issue, but I was using the location() function in a cfscript. But I suspect the same principle applies.

Prior to the location() function, we were calling a function that was not wrapped in <cfsilent> tags. As a result, every line of the function, including loops, was being output as an empty line of HTML. When I viewed the source code, there were over 20,000 lines of empty HTML code. That is probably enough to reach the 1024 kb buffer limit and trigger the flush that aborts the redirect.

By adding <cfsilent> tags to the custom function, I reduced the number of empty lines of HTML to 39, and now the location() function works as expected.

To determine if this is the issue you are having, add abort; (or <cfabort>) just ahead of the location(), then view the source code of the resulting - presumably empty - page.

M.Shute
  • 347
  • 1
  • 2
  • 11
0

Have you tried adding CFABORT after CFLOCATION? That's what I always do - as the code after CFLOCATION should not be executed.

If you check the response from the server (using Charles proxy, Fiddler or something similar) you may find something in the HTML/JS that's preventing the redirection.

Lucas
  • 1,402
  • 1
  • 12
  • 18
  • Hi Lucas, my understanding is that cfabort isn't needed as cflocation pretty much stops processing at the point it's called (with the exception if you have an OnRequestEnd()). Anyway to just to rule it out, I did try and it made no difference. I've managed to narrow in on getting it to work and stop (see comment to Adam above) but it's just made the problem even more weird. I'm going to investigate a little more so I can document it and then update my original question. – tHeSmUrF Oct 17 '13 at 06:33