This is going to be an obvious post for the web developers out there, but I just spent entirely too much time figuring out the wall that I was running into here.

Little context: One of the sites that I run is Lukky Us Games, which focuses (right now at least) around drinking games for movies. I utilize a number of custom-built functions to pull data from various sources to add/update posts. An example of this is rather than manually adding movie poster image, title, actors, etc. to each post, I automatically pull this type of data from The OMDB (since IMDB doesn’t allow external API access).

Recently, it was requested that I add to the site which streaming services the particular movies were available on, so that users could utilize that as part of the decision tree for what they were going to watch. It took me a bit, but I was able to get that added in relatively short order utilizing Utelly’s free service through RapidAPI. The problem is that I was making the API calls to pull that data in as the page was being viewed, rather than storing it. Since you have to start paying for this service after 1000 calls, I ended up paying a fair bit just to run that part on the site.

So I set about to start storing this data, which wasn’t terribly hard utilizing the Repeatable Field Groups functionality within Toolset Types. The problem is that these characteristics about a movie are not static, services (somewhat frequently) add and remove particular movies from their offerings. Until now, I’d been slacking a bit and just pulling the data in when I published the post and the leaving it static after that, but it’s been at least 8 months now and I’m sure many of them were out of date, so it was time to refresh them. I was able to get a second function set up like below.

/* Manual update of streaming services for every movie */
 function streaming_services_manual() {
     $movieargs = array(
         'post_type' => 'movie',
         'numberposts' => -1,
         );
     $movies_updated_number = 0;
     $movie_query = new WP_Query($movieargs);
     while ($movie_query->have_posts()) {
         $movie_query->the_post();
         $moviename = get_the_title();
         $movieid = get_the_id();   
    $moviestr = rawurlencode($moviename);
    $serviceoutput = "";
    $curl = curl_init();

    curl_setopt_array($curl, array(
        CURLOPT_URL => "https://utelly-tv-shows-and-movies-availability-v1.p.rapidapi.com/lookup?term=".$moviestr."&country=us",
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_ENCODING => "",
        CURLOPT_MAXREDIRS => 10,
        CURLOPT_TIMEOUT => 30,
        CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
        CURLOPT_CUSTOMREQUEST => "GET",
        CURLOPT_POSTFIELDS => "",
        CURLOPT_HTTPHEADER => array(
            "Content-Type: application/json",
            "Postman-Token: ***",
            "X-RapidAPI-Key: ***",
            "cache-control: no-cache"
        ),
    ));
    $json = curl_exec($curl);
    $err = curl_error($curl);
    curl_close($curl);

    // Decode JSON data into PHP associative array format
    $arr = json_decode($json, true);
    if(!empty($arr["results"])){
        $searchargs = array(
            'post_type' => 'streaming-available',
            'numberposts' => -1,
            'toolset_relationships' => array(
                'role' => 'child',
                'related_to' => $movieid,
                'relationship' => 'streaming-available'
             ) 
        );
        $query = new WP_Query($searchargs);
        if ($query->have_posts()){
            while ($query->have_posts()) {
                $query->the_post();
                wp_delete_post(get_the_ID());
            }
        }
        foreach($arr["results"][0]["locations"] as $service){
            $ssoutput = $service["display_name"];
            wp_set_object_terms( $movieid, $ssoutput, 'streaming-service', true );
            $streamingicon = $service["icon"];
            $streamingname = $service["display_name"];
            $streamingurl = $service["url"];
            //Create a new RFG Item
            $new_rfg = array(
                'post_title'    => $streamingname,
                'post_status'   => 'publish',
                'post_type' => 'streaming-available'
            );
                $new_rfg_id = wp_insert_post( $new_rfg );
            //Add Fields to the RFG
            update_post_meta($new_rfg_id, 'wpcf-streaming-icon', $streamingicon);
            update_post_meta($new_rfg_id, "wpcf-streaming-name", $streamingname);
            update_post_meta($new_rfg_id, "wpcf-streaming-url", $streamingurl);
            //Now connect the new RFG to a Post
            toolset_connect_posts('streaming-available', $movieid, $new_rfg_id);
        }
    }
    
    $movies_updated .= $moviename . " Updated";
    $movies_updated_number++;
}
$movies_updated_return = $movies_updated . $movies_updated_number . " Movies Updated";
wp_reset_postdata();
return $movies_updated_return;
}

This worked really great, except it would only return/update 100 posts, and for the life of me I couldn’t figure out why. I started looking into limits of WP_Query, or the have_posts() function, or even the increment function that I used literally just to provide some feedback of how many posts were updated. Everything that I found indicated that setting “numberposts => -1” was the way to return all posts, essentially.

Well I was wrong. Even with numberposts set to -1, WP_Query still honors the WordPress setting under Settings->Reading->Blog pages to show at most. Since I had that set to 100 (I can’t remember if that’s the default or if I set it that way at one point), it would only return 100 posts. Setting that higher now allowed me to update all of them. Like I said, every web developer out there knew what the answer was 30 seconds in.