I have a website that is supposed to post a report from time to time to a Yuku forum.
The code has been working for really a long time (5 years or more). Just recently, it stopped working.
Stops working as in.... no errors are reported, but the report isn't posted.
It looks like the forum's form page must have changed in some way that my submission is no longer valid.
The form that it is trying to fill in is this one:
http://webandofbrothers.yuku.com/forum/newtopic/id/2
... the form response from this URL (for a logged in user) is pasted here
The code that's doing the post is appended. It reports that it succeeds, but it doesn't - nothing happens (no post is made).
The request that it posts is this:
POST http://webandofbrothers.yuku.com/forum/post
Content-Length: 160
Content-Type: application/x-www-form-urlencoded
identity=2356541&posticon=0&title=IfYouSeeThisItsGood&sig_id=sig_id&topic=2&id=2&convert_urls=1&parse_smilies=1&cancel_url=%2Fforums%2F2&post=Post&body=blahblah
as reported by
print $cgi->pre($request->as_string);
This seems to compare well with what Chrome "inspect element" says is submitted when I submit the form manually:
identity=2356541&posticon=0&title=More+testing%2C+nothing+to+see+here&body=More+testing&sig_id=sig_id&convert_urls=1&parse_smilies=1&topic=2&id=2&cancel_url=%2Fforums%2F2&post=Post
I can't see either what could be making the form unhappy about the submission, nor why it reports as "no error".
Note that the user agent definitely succeeds logging in (as far as all tests I've done show), and the logged in user can definitely post (permissions-wise) - I've tested that manually.
What I have found, by printing the response even if it's not supposedly an error, is that the response contains this:
HTTP/1.1 200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: close
Date: Tue, 14 Apr 2015 13:06:12 GMT
Pragma: no-cache
Server: Apache
Vary: Accept-Encoding
Content-Length: 32
Content-Type: text/html; charset=UTF-8
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Client-Date: Tue, 14 Apr 2015 13:06:11 GMT
Client-Peer: 209.132.196.103:80
Client-Response-Num: 1
Set-Cookie: crwg_rsid=1045; path=/;
forum.php: failed to insert lead
So for some reason my POST sent by perl is not the same as the manual form POSTing, but I can't figure out why...
my $ua = LWP::UserAgent->new;
$ua->cookie_jar({ file => "./.cookies.txt" });
my $loggedIn = LogInToEZBoard($ua);
if (!$loggedIn)
{
print $cgi->p("URK ... looks like I couldn't log into Yuku to post your AAR");
return;
}
# post the message
# This is the AAR forum POST URL:
my $formAddTopicURL = 'http://webandofbrothers.yuku.com/forum/post';
# First we have to ask for a new topic page, so we can grok our yuku "identity"
# ... so let's find that...
my $response;
$response = $ua->get("http://webandofbrothers.yuku.com/forum/newtopic/id/2");
# Now we will parse this response to find the idenity...
# first, here is the parser...
my $p = HTML::Parser->new(api_version => 3,
marked_sections => 1,
);
# and here is the variable where we will place the final answer we're looking for...
my $yukuIdentity = 0;
# Now, set up a start tag handler that will extract the final value that we're looking for
# from an option tag ... this handler will be installed when we're in the correct select tag...
# When this anonymous routine is called, during $p->parse (below)
# closure ensures that $yukuIdentityRef will be pointing at the right place to let us update
# $yukuIdentity and bail out from parsing
my $yukuIdentityRef = \$yukuIdentity;
my $extractIdentity = sub
{
my ($tag, $attrs, $self) = @_;
if (($tag eq "option") &&
($attrs->{'value'}))
{
print $cgi->pre("Got identity $attrs->{'value'}");
$$yukuIdentityRef = $attrs->{'value'};
$self->eof();
}
else
{
print $cgi-pre("not what we're looking for $tag " . keys(%$attrs));
}
};
# Now here is a start tag handler to find the correct select tag...
# ... if we hit the identity select tag then we go looking for the identity.
my $findIdentitySelect = sub
{
my ($tag, $attrs, $self) = @_;
if (($tag eq "select") &&
($attrs->{'name'}) &&
($attrs->{'name'} eq 'identity'))
{
print $cgi->pre("Now in identity select...");
$self->handler(start => $extractIdentity, "tagname,attr,self");
}
else
{
print $cgi->pre("Not what we're looking for: $tag");
}
};
# Now we have handlers, load them up and away we go...
$p->handler(start => $findIdentitySelect, 'tagname,attr,self');
$p->parse($response->content);
# tada. By the time we get here, $yukuIdentity ought to be set!
if (!$yukuIdentity)
{
die "Hmmm - bit of a problem, wasn't able to understand Yuku's post form. Better let GaJ know...\n";
}
print $cgi->pre("Ok, my ID is $yukuIdentity, sending request...");
# So at last we can post our AAR...
my $request = POST $formAddTopicURL,
[identity => $yukuIdentity,
posticon => '0',#,$postIcon, # 7 = smile
title => "IfYouSeeThisItsGood", #"$gameType $update: $axisPlayer vs $alliedPlayer in $scenarioName",
sig_id => 'sig_id',
topic => '2',
id => '2',
convert_urls => '1',
parse_smilies => '1',
cancel_url => '/forums/2',
post => "Post",
body => $aarBody];
print $cgi->pre($request->as_string);
eval
{
$response = $ua->request($request);
};
if ($@)
{
print $cgi->p("Oh-oh: something went wrong sending the request to post the AAR to yuku....");
print $cgi->pre($@);
}
else
{
if (!$response->is_error)
{
print $cgi->p("...looks like it posted OK... the response was:");
}
else
{
print $cgi->p("Oh no - yuku didn't like the post requset I sent. It said:");
}
print $cgi->pre($response->status_line);
print $cgi->pre($response->as_string);
}
