Why Duplicate posts are being added while fetching data from RETS server?

Question

I am trying to add posts in wordpress from rets server using PHRETS. Unfortunately duplicate posts are being added. I have used the WP Query to check the existing post using meta key and value. This code is running well when I am trying to add 10 or 50 posts but when I set the limit to 4000 it start adding duplicate posts. I have run this code so many time and flushing the database so many time. Here is a code sample:

$query = "(922=MIAMI), (246=A)";
$search = $rets->SearchQuery("Property", $class, $query, array("SystemName" => 1, 'Limit' =>4550));

if ($rets->NumRows($search) > 0) {
    $fields_order = $rets->SearchGetFields($search);

    while ($record = $rets->FetchRow($search)) {
        foreach ($fields_order as $fo) { 
            if ($fo == 'sysid') { $systemid = $record[$fo] ; }
            if ($fo == '881') { $saddress = isset($record[$fo]) ? $record[$fo] : ""; }
            if ($fo == '214') { $sremarks = isset($record[$fo]) ? $record[$fo] : ""; }
        }

        $porpertytitle = $saddress;

        $args = array(
            'numberposts' => -1,
            'post_type' => 'property',
            'post_status' => 'publish',
            'meta_key' =>'sysid',
            'meta_value' => $systemid
        );

        $the_query = new WP_Query($args);

        if($the_query->have_posts()) {
            while ($the_query->have_posts()) {
                $the_query->the_post();
                unset($systemid);
                unset($args);
            }
        } else {
            $my_listing = array(
                'post_title' => $porpertytitle,
                'post_type' => 'property',
                'post_content' => $sremarks,
                'post_status' => 'publish',
            );

            $listing_post_id = wp_insert_post($my_listing);

            if($listing_post_id > 0) {
                update_post_meta($listing_post_id, 'sysid', $systemid);
            }

            unset($systemid);
            unset($args);
            unset($listing_post_id);
        }
        wp_reset_postdata();
    }
}

Sorry for this long code. But I have also tried unset the variables after every loop but not working.

Does it duplicate all records or just a few? Can you please post a few non-duplicate records and records that have been duplicate. Please post full complete records and not partial. — dj_goku, Jan 08 '15 at 03:14
I could not say strongly that does it duplicate all records or just a few because it does not follow the pattern. Some time it duplicate all the posts and some time just the few post with the same query and same code. I think the problem is with if and while condition on top of the code. because I think the Wp_Query is fine. And the problem is something else! — Vijay Lal, Jan 14 '15 at 04:59
I highly doubt there isn't a pattern to the duplicates. I would suggest picking a few records that have been duplicated and step through the code to figure out what is causing the duplication. — dj_goku, Jan 16 '15 at 03:14

score 2 · Accepted Answer · answered Mar 03 '15 at 07:39

I had a similar seemingly random duplicate problem myself. I finally determined the problem is that RETS Server offsets are 1 based, not zero based. Also, if you don't pass an offset with your request the server will sort the results differently than if you do.

If you can inspect the request being sent to the server, see if it includes offset=1 on the first request. If it is zero or missing you'll get repeats of the first page's results scattered in subsequent pages, which can be a lot with a 4000 page size

Why Duplicate posts are being added while fetching data from RETS server?

1 Answers1

Linked