Still to come before the next full version: Image support and (hopefully) a better way to authenticate into Google.

The code isn’t hard to grasp, but it will take some modification to get it to work.

The code is all on Github. Essentially it reads posts from WordPress via XMLRPC and creates InDesign Tagged Text files on a local drive.

I’ve written XMLRPC extenders for WordPress to go with this code. It is also on Github.

The XMLRPC extender simply allows us to query based on a lot more options and return a very full result. The real work is in the import script, and there you’ll have to do some modifications to make it work with your InDesign styles.

For us, this script is running on a server on our local network that everyone here has access to. We let it run every minute by setting up this simple cron job:


*/1 * * * * curl localhost/import.php


You’ll need a folder to store all the files in. By default it’s a directory called files. You’ll need to make sure that has the proper permissions to be written to.

By default all the stories will be saved in that folder named id.txt. We use this to programmatically pull the stories into InDesign, but that plugin isn’t released yet. You might want to consider naming the files using the story title to make it more easily human readable.

The templating process isn’t quite as easy as WordPress’, but if you’re familiar at all with styles and PHP you should be able to figure things out. The code is pretty well documented, I think, but if you have questions please leave them as a comment on this post and I will do my best to respond to all of them.

• Export WordPress posts to InDesign Tagged Text
• BDN WordPress XMLRPC extenders

This isn’t a standard feature of the plugin, but the plugin does include a few filters to modify how posts are formatted. One of these filters is pre_docs_to_wp_insert, and it can be leveraged as such:

<?php
/*
Plugin Name: Extend Docs to WP like so
*/

add_filter( 'pre_docs_to_wp_insert', 'bdn_split_post' );
function bdn_split_post( $post_array = array() ) {

	$exploded_fields = explode( '|', $post_array[ 'post_content' ] );
	
	//Sometimes people forget a pipe, and we don't want to put the entire post in the headline
	if( is_array( $exploded_fields ) && count( $exploded_fields ) >= 2 ) {

		//Save the old title in case you want to do something with it
		$old_title = $post_array[ 'post_title' ];

		//Set the title to the first occurance.
		$post_array[ 'post_title' ] = strip_tags( $exploded_fields[ 0 ] );
		
		//Unset the title
		unset( $exploded_fields[ 0 ] );
		
		//Now restore the post content and save it
		$post_array[ 'post_content' ] = implode( '|', $exploded_fields );
		
	}

	return $post_array;

}

I haven’t tested it but the above code should do the trick.

Coming soon: Details on moving stories from WordPress to InDesign!

This evening I released an update to that plugin that should resolve any problems people had. In addition, I made a small change to make bold and italics come through more consistently.

As always, you can download the latest version in the WordPress Plugin Repository.

At the BDN, we don’t actually insert images into posts, because we bring content directly from WordPress into InDesign and putting images in posts would almost certainly cause problems. We’ve developed a set of functions to instead query for the images attached to posts and display those. They’re, of course, posted to Github. You can use the code as a simple plugin.

We’ve also customized the image edit screen a bit. As I think I’ve mentioned before, we don’t use the alt tag field, for example, nor do we use the alignment or image size fields, since we don’t insert images. We also wanted the caption field, which our functions use to display the caption info, to be a textarea instead of a textbox. All this was incredibly easy to do:

<?php
add_filter('attachment_fields_to_edit', 'bdn_do_image_things', 20, 2);
function bdn_do_image_things( $fields, $post ) {

	//Unset the fields we don't need
	unset( $fields[ 'image_alt' ] );
	unset( $fields[ 'image-size' ] );
	unset( $fields[ 'align' ] );

	//Turn the caption field into a textarea.
	$fields['post_excerpt']['input'] = 'textarea';

	return $fields;
}

It should be noted that changing the caption field from a text box to a textarea caused a problem with double-encoding of apostrophes and straight quotes. I’ve submitted a ticket and a patch to core, and hopefully this will be fixed in 3.3.

But this all didn’t solve the overall problem of having to reupload a photo every time we wanted to use it on a new post, even if it was in the system. We use a lot of file photos, so there was a lot of searching for the photo in WordPress, downloading it and promptly reuploading it. That was a lot of time wasted and a lot of aggravation. So today I whipped up a quick plugin to solve this.

Duplicate Images adds a link when you’re browsing the image library from a post that will copy the attachment data and attach it to the current post you’re working on. It doesn’t copy the actual image — that isn’t necessary. Instead it copies all the metadata over and links back to the original image so that things don’t get messy. If you decide to crop the image, for example, the meta data for the new post will change but not for the original post. It’s a pretty lightweight way to allow for attaching images to multiple posts within WordPress.

There are a few other things we do to make handling media as simple as possible in WordPress. We use Scott Bressler’s excellent Media Credit plugin to handle credits for images. We’ve extended it a bit to automatically bring in the photographer credit when uploading images (by default, it just uses whoever uploaded the photo). The code to do that, as well as to populate the caption from the Instructions field (a field we picked at random for the publishable caption) is also available on Github.

I’m very interested to hear thoughts on how everyone else manages images.

The setup

We recently changed how we handle our sports games and scores on our website. When we started out, we used the Pods CMS Framework to create some new tables to store schools, teams and games. What we found is that this added a lot of overhead to every page load, it required a lot of maintenance and the value wasn’t there, so we regrouped.

Now, we have a custom post type Games and custom taxonomies Schools and Sports. Each game comes in as a post with the relevant schools and sport attached. Then it becomes pretty easy to build a page of scores for the day or display a game without any extra queries or database tables. Yay custom post types.

We built everything as a plugin and tested it as a proof of concept. It worked pretty well so we went ahead and imported all 2,400 high school sports games scheduled this season.

The trouble starts

Last night, just as I was dozing off, our site crashed. After a few frantic minutes of everyone trying to figure out what was going on, our sysadmin diagnosed the problem: We were out of disc space on our master database server. He also found a 10 GB test file, which we promptly deleted, and everything came back. This morning, though, the site went down again just as I was coming into work. Again, we were out of disc space on the master db server. And so we started to try and diagnose how we used 10 GB of file space while we were all asleep and almost nothing was being posted to the website.

The reveal

The problem ended up being our binlogs, which are filling up at an astronomical rate. And the reason was because almost all of the 2,400 games we imported into WordPress are marked as future posts and [for each of them, the Disqus plugin] spawned a one-off WP-Cron job, which is saved in the *_options table. Each time WP-Cron runs, which is sometimes multiple times a minute for us, the array of cron jobs is actually saved back to the *_options table, and each time our master db is updated the command is written to a binlog so it will propagate to our slave databases. Each time WP-Cron runs, we were adding almost 33 KB to the bin file.

The fix (see below for the real fix)

Well, we don’t have a permanent one yet. First, we disabled WP-Cron in favor of true cron jobs to make sure we’re not running things more than once a minute. One option is to remove all the future posts out of cron, or to change them to a different status such as To Be Played that doesn’t trigger a cron job. I think a better solution, though, is to patch WordPress so each new post doesn’t trigger a new cron job. My proposed solution is to search for stories with a status of future and a publish date less than or equal to now. There can also be a flag set every time a post is set to have a future date and then turned off when that post is actually published so that the query doesn’t turn up nothing. This would help avoid clogging up the cron job list with one-offs and would also help avoid cases where cron fails to run correctly and future posts don’t get published.

The true fix

At first we thought that WordPress spawned a new cron job for every future post. The culprit actually turned out to be Disqus, which was creating a new cron job to sync the comments on thousands of posts. The simple solution: In the Disqus settings page, disable comment sync.

The newest version of the cleaner extension passes correctly parses bold and italic text and headings, such as <h4>. It also strips out span tags to avoid a few instances where they would cause an extra line break.

In an earlier version of Docs, bold and italic text were surrounded by <em> and <strong> tags, as would be expected. It made it very easy to handle stylized text.

In the newest version of Docs, however, text formatting is done completely via CSS. So the header of each of the HTML versions of the Doc contains a stylesheet, where bold and italic text is given an arbitrary class name.

To carry that through to WordPress, I use preg_match to find the correct class name from the stylesheet and then preg_replace to convert span tags to <em> and <strong>. Thanks much to Andrew Nacin, Rob Flaherty Bill, who replied to my cry for help.

As always, the Docs to WordPress plugin can be found in the WordPress Plugin Repository.

So, what I’m looking for is a regex expression to turn <span class=”c0 c3″>My text</span> into <span class=”c0 c3″><strong>My text</strong></span> where class c3 is the bold class, for example.