mlsamuelson.com

Getting Started with Drupal's Batch API

The following is a posting of my code from this month's Boise Drupal User Group presentation I did on Drupal's Batch API as it exists in Drupal 6. It also showcases a simple technique one could use for firing off batch operations via Drupal's admin interface. It's light on prose, and heavy on code.

screenshot of batch api in action: progress bar

Drupal's Batch API is a great friend to have on hand when you have a lot of data to process and are in danger of exceeding PHP script execution time limits, or when you have a complex set of operations to run in sequence, or both.

Implementing Drupal's Batch API is straightforward, and I've ventured to keep it simple using a ridiculous example that could be extended into something useful.

Our example here is a simple script that uses Drupal's menu system to map an interface into Drupal's admin section whereby we can trigger batch operations via form buttons.

First, create a folder for your module in your sites/all/modules/ directory (or sites/all/modules/custom, if you prefer).

I created sites/all/modules/batch_demo/ and the first file I put in there is batch_demo.info.

; $Id$

name = "batch demo"
description = "Batch API Demo"
core = 6.x
package = "Batches"
dependencies[] = profile

This file is used in Drupal as metadata for our module. More info on .info files.

Next up we create a file named batch_demo.module.

<?php
// $Id$

/**
* @file
* Batch API Demo for presentation at Boise Drupal User Group.
*/

require_once(drupal_get_path('module', 'batch_demo') .'/includes/batch_demo.users.inc');

/**
* Implementation of hook_menu().
*
* We're simply mapping our admin page into Drupal's menu system.
*/
function batch_demo_menu() {

 
$items = array();
 
 
$items['admin/build/batch'] = array(
   
'title' => 'Demo Batches',
   
'description' => 'Run batch operations.',
   
'page callback' => 'drupal_get_form',
   
'page arguments' => array('batch_demo_form'),
   
'access arguments' => array('administer site configuration'),
   
'file' => 'batch_demo.admin.inc',
   
'file path' => drupal_get_path('module', 'batch_demo') . '/includes',
   
'type' => MENU_NORMAL_ITEM,
   );
  
  return
$items;
}
?>

More on hook_menu().

As you can see from the require_once() and the hook_menu() implementation, we are going to need two new files in an includes folder. So create that folder, and pop the next two files in there.

batch_demo.admin.inc

<?php
// $Id$

/**
* @file
* Contains forms and form handler functions that make up the batch demo administration section.
*/


/**
* Use Drupal's Form API (FAPI) to wire up buttons to kick off batch operations.
*/
function batch_demo_form(&$form_state) {

 
$form['user_batches'] = array(
  
'#type' => 'fieldset',
  
'#title' => 'User Batches',
  
'#description' => 'Run batch operations for users.',
  
'#collapsible' => TRUE,
  
'#collapsed' => FALSE,
  );
 
 
$form['user_batches']['batch_add_first_last'] = array(
   
'#type' => 'submit',
   
'#value' => t('Add first and last names'),
  );
 
$form['user_batches']['batch_happy_message'] = array(
   
'#type' => 'submit',
   
'#value' => t('Make me happy'),
  );
 
  return
$form;
}

/**
* Submit handler for batch_demo_form();
*/
function batch_demo_form_submit($form, &$form_state) {

  require_once(
drupal_get_path('module', 'batch_demo') .'/includes/batch_demo.users.inc');

 
// Use a switch to catch which button was clicked and route us
  // to the appropriate function.
 
switch ($form['#post']['op']) {

    case
t("Add first and last names"):
     
batch_demo_first_last(); // Function to kick off a batch API implementation.
     
break;

    case
t("Make me happy"):
     
// And, okay, so this is where you'd call another function that would
      // trigger a batch API implementation. But I'm not bothering to write
      // another implementation... just displaying a message for the user.
     
drupal_set_message('[Snaps fingers.] You are now happy.');
      break;

   
// Add more cases to trigger batch operations for more buttons.

 
}

}
?>

screenshot of our admin interface

The curious can read up on Drupal's Forms API via the Quickstart Guide and the Forms API Reference.

So where's the Batch API at this point?, you're probably asking... Right around the corner, in:

batch_demo.users.inc

<?php

/**
* Set up batch for first and last name loading.
*
* This is where Drupal's Batch API comes into play.
* It's as simple as defining $batch, and then calling batch_set($batch)
*/
function batch_demo_first_last() {

 
// Update the user profiles to add values to newly added profile fields
 
$batch = array(
   
'title' => t('Updating User Accounts'), // Title to display while running.
   
'operations' => array(), // Operations to complete, in order. Defined below.
   
'finished' => '_batch_demo_first_last_batch_finished', // Last function to call.
   
'init_message' => t('Initializing...'),
   
'progress_message' => t('Fixed @current out of @total.'),
   
'error_message' => t('Updating of profiles encountered an error.'),
  );

 
// Add as many operations as you need. They'll run in the order specified.
  // Parameters can be defined in the (currently) empty arrays and will need
  // to also be added following the $context parameters for the operation
  // functions below.
 
$batch['operations'][] = array('_batch_demo_first_last_batch', array());
 
$batch['operations'][] = array('_batch_demo_another_first_last_batch', array());

 
// Tip the first domino.
 
batch_set($batch);
}

// Our first batch operation.  Note: $context contains Batch API data.
function _batch_demo_first_last_batch(&$context) {

 
$limit = 20;

 
// This is not very robust, but you get the point.
  // The profile values are saved in {user.data} if we have a profile_firstName there, exclude.
 
$count = db_fetch_object(db_query("SELECT COUNT(*) as count FROM {users} WHERE 1 AND uid != 0 AND (data NOT LIKE '%profile_firstName%' OR data IS NULL)"));
 
$results = db_query("SELECT * FROM {users} WHERE 1 AND uid != 0 AND (data NOT LIKE '%profile_firstName%' OR data IS NULL) LIMIT %d", $limit);

  while (
$row = db_fetch_object($results)) {

   
$account = user_load($row->uid);

   
// Add first and last names.
    // We hardcode these in, but you could load from csv, another db table, etc...
   
$edit = array(
     
'profile_firstName' => 'Jimmy',
     
'profile_lastName' => 'James',
      ); 
       
   
// Save user.
   
user_save($account, $edit);

   
drupal_set_message('Updated fields for ' . $account->name . ' [' . $account->uid . ']');

  }

 
// Until $context['finished'] returns true, this will continue to iterate,
  // updating 20 user profiles at a time.
 
if ($count->count > 0) {
   
$context['finished'] = 0;
  }
  else {
   
$context['finished'] = 1;
  }

}

/**
* Our useless second operation, here just to show how that's done.
*/
function _batch_demo_another_first_last_batch(&$context) {

   
drupal_set_message('Fired another batch operation.');

   
$context['finished'] = 1;
}

/**
* The function called when we finish. Displays a success or error message,
* but could do anything.
*/
function _batch_demo_first_last_batch_finished($success, $results, $operations) {

  if (
$success) {
   
$message = t('All users have had first and last names processed.');
  }
  else {
   
$message = t('Finished with error.');
  }
 
drupal_set_message($message);
}
?>

So in the site I'll be using this in, I've enabled Drupal core's profile module and added two profile fields - profile_firstName and profile_lastName.

screenshot of profile fields

Then I installed the Devel and Devel Generate modules from Drupal's contributed modules repository (they're bundled together) and generated 5,000 users for my site. I now want to load a first and last name into Drupal for each of these users. That's what the code above does. For simplicity's sake, I'm setting everyone's profile name to "Jimmie James," but something more helpful such as creating users from a CSV file could easily be done.

Triggering the batches I've wired up is now easy. I visit admin/build/batch and click the button I want.

On a recent project I used Batch API to do a node_load() on all nodes of a specific content type in a site (4,500 nodes in all) and resave them as two different nodes with two new content types (now 9,000 nodes). For each two "new" nodes, one retained the old node ID and the other was related to that node via a nodereference cck field. Fun stuff. It took hours to run that batch, and would have been a lot more work to write without the Batch API.

Resources for learning more about the Batch API:

Download the code:

AttachmentSize
batch_demo.tar_.gz1.83 KB

5 comments

MikeW wrote 7 years 20 weeks ago

Thanks, but one thing...

First, thank you so much for this, it was very easy to understand and to adapt to my project.

The only thing I would change is this:
switch ($form['#post']['op']) {
Could be this:
switch ($form_state['values']['op']) {

joe wrote 6 years 28 weeks ago

Excellent post!

This batch post serves as a great intro into making Drupal modules. My utility is working but one detail remains: The "init message" stays present and the progress indicator doesn't move to represent portion completed. Does this code produce a working progress bar for you?

mlsamuelson wrote 6 years 28 weeks ago

Progress bar....

Nope, this code doesn't really delve into using the progress bar with any granularity. Check out the links for learning more about the Batch API to learn more about that.

Pierre Paul Lefebvre wrote 6 years 24 weeks ago

Excellent!

The best article about Drupal Batch API. I really hope you would rank higher in Google. Thanks!

Omprakash wrote 4 years 9 weeks ago

Yes it's great and helpful

Yes it's great and helpful link for batch process API. But here need some fixes:
The condition:

if ($count->count > 0) {
$context['finished'] = 0;
}

Will make iteration in infinite loop.

Instead of that please do somewhat like :

function _batch_demo_first_last_batch(&$context) {

$limit = 5;

// This is not very robust, but you get the point.
// The profile values are saved in {user.data} if we have a profile_firstName there, exclude.
$count = db_fetch_object(db_query("SELECT COUNT(*) as count FROM {users} WHERE 1 AND uid != 0 LIMIT 2"));
$results = db_query("SELECT * FROM {users} WHERE 1 AND uid != 0 AND (data NOT LIKE '%profile_firstName%' OR data IS NULL) LIMIT %d", $limit);
$i = 1;
while ($row = db_fetch_object($results)) {
drupal_set_message('Updated fields for ' . $row->name . ' [' . $row->uid . ']');
$i++;
}
// Until $context['finished'] returns true, this will continue to iterate,
// updating 20 user profiles at a time.
if ($count->count == $i) {
$context['finished'] = 0;
}
else {
$context['finished'] = 1;
}

}

Really enjoyed this article.

Add your comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • You may post code using <code>...</code> (generic) or <?php ... ?> (highlighted PHP) tags.

More information about formatting options

Find Me Around

User login