Update to Google Docs to WordPress plugin

A few weeks ago, unbeknownst to me, Google made a (wise) change to its APIs to require SSL for all API calls. This caused parts (but not all, apparently) of the Docs to WordPress plugin to stop working.

This evening I released an update to that plugin that should resolve any problems people had. In addition, I made a small change to make bold and italics come through more consistently.

As always, you can download the latest version in the WordPress Plugin Repository.

Updated Docs to WordPress plugin: Now with better formatting

Version 0.3-beta of our Docs to WordPress plugin has been released. It’s a fairly minor release, and only affects people using the cleaner extension, but we just rolled it out internally this morning at the BDN and it’s exciting for us, so I wanted to pass it on to the public.

The newest version of the cleaner extension passes correctly parses bold and italic text and headings, such as <h4>. It also strips out span tags to avoid a few instances where they would cause an extra line break.

In an earlier version of Docs, bold and italic text were surrounded by <em> and <strong> tags, as would be expected. It made it very easy to handle stylized text.

In the newest version of Docs, however, text formatting is done completely via CSS. So the header of each of the HTML versions of the Doc contains a stylesheet, where bold and italic text is given an arbitrary class name.

To carry that through to WordPress, I use preg_match to find the correct class name from the stylesheet and then preg_replace to convert span tags to <em> and <strong>. Thanks much to Andrew Nacin, Rob Flaherty Bill, who replied to my cry for help.

As always, the Docs to WordPress plugin can be found in the WordPress Plugin Repository.

Appeal for help, because I’m terrible at regex

One of the most annoying things about using Google Docs is that none of the styles are inline. It used to be that bold text was wrapped in a <strong> tag and italic text was wrapped in am <em> tag. No longer. Now each style of text is wrapped in a span with a number of different classes applied to it. Those styles don’t carry through when we bring the text into WordPress and the names of the classes vary from article to article. This can be very annoying for columnists who bold names of subjects, for example.

So, what I’m looking for is a regex expression to turn <span class=”c0 c3″>My text</span> into <span class=”c0 c3″><strong>My text</strong></span> where class c3 is the bold class, for example.

OK, so you’ve built your site. Now what? Focus on performance. #wcbos

Here at the BDN, we’re all a bit relieved to finally have completed the relaunch of bangordailynews.com. And one of our tasks now is to clean up all the mistakes we’ve made and all the sloppy code we’ve written while rushing toward the finish line.

A great talk by Frederick Townes, CTO of Mashable.com and the man behind the must-use W3 Total Cache plugin, gave a great talk at WordCamp Boston this weekend about performance enhancements, and I thought it’d be good to repeat some of his points and add a few of my own.
Continue reading OK, so you’ve built your site. Now what? Focus on performance. #wcbos

New Plugin: Define SSL Pages

Just released: A very simple plugin that allows you to require SSL for certain pages on your site. For example, at the BDN we moved the login form from /wp-login.php to /login/, and wanted to require SSL for that page. So, using this plugin, we can force anyone who visits http://bangordailynews.com/login/ to https://bangordailynews.com/login/.

It’s in the WordPress plugin repository: http://wordpress.org/extend/plugins/require-ssl-for-pages/

Form ever follows function, eventually

Our online editor, Will Davis, has been explaining how we flowed text from Google Docs to WordPress to print and created a low-cost, portable front-end system for our newsroom at Bangor Daily News. I wanted to tell you a little about why we did it.

If you read what Jeff Jarvis, Chuck Peters and Clay Shirky have been writing about decaying newsrooms and the need for new models, it’s hard to believe they weren’t taking notes in some darkened corner of ours, perhaps in Sports behind the stacks of old game results and Mountain Dew cans.

Like many newsrooms, until very recently we were production heavy because we had to be. Moving stories to the web was a copy-and-paste affair, but that’s not where the trouble started. If you begin with a print-directed front-end system, as we did, how does that system accommodate a story being updated from the field? Or how would the full possibility of story assets land online, to be chosen among for print? Even simpler: When do reporters add links? The answers, as countless journalists know, are: It can’t; they won’t; they don’t. From there, it’s all production, not creation.

As we lost staff to cutbacks over the years, assembling our content into finished products was taking a larger and larger percentage of our time. Simply processing press releases seemed to suck up significant portions of editors’ days. No one wanted to be in this situation, but our infrastructure for moving content demanded it. We were trapped.

We needed reporters to get out of the tools they had been using for more than a decade to drive toward single shift-end deadlines. We needed to simplify the connections between what reporters wrote and what the public saw. We needed to link our bureaus so that they were much more a part of the daily news flow; mobility, so that any staff member with a cell phone could file from anywhere; web archiving that allowed us to expand on stories and retrieve content below the level of a story — in brief, we needed to match the way our audience now acquires information. Also, we didn’t have any money for this project.

Then along came JRC’s Ben Franklin Project, pointing the way. We had begun using Google Docs in our new media department in 2007, when the department was created and we suddenly had to keep and share records on web development, ad sales and commissions, good ideas and meeting minutes. Docs as front-end newsroom system became apparent as Google improved its product and we needed a place for reporters to store notes, interviews, story ideas and all the rest in a place they could organize. The WordPress CMS, as good as it is, didn’t seem like the place to do that.

As the newsroom has grown comfortable with Docs, it is becoming more efficient (links and headlines, for instance, travel from Docs to WordPress) and we are shifting staff members from production to content creation. We knew we had a winner in Docs when we had a major election story with two reporters in the field and an editor in the newsroom, all working simultaneously on the same breaking story, adding content, seeing in real time what each was adding, talking to each other through the chat function and responding with updated information. Fast, simple, low cost.

We’re a long way from done. We’re still working on ways to present data and extract pieces of story content to create a coherent, useful whole; and we are just beginning the process of providing our audience a range of tools to contribute their own content. But in the newsroom, the guiding ideas we have put into practice are to match the tool to the job we need done (rather than the reverse), reduce the number of steps required and anticipate how our audience will want the information next. And the cost should be next to nothing.

Quick update to the Docs to WordPress plugin

We’ve released a quick update to the Docs to WordPress plugin that allows the plugin to integrate with WP Cron rather than requiring you to create your own file to run cron against (though you can still do that, too). To take advantage of it, you’ll have to upgrade to the latest version and edit your wp-config file to define at least three variables:

DOCSTOWP_USER: Your Google Docs username

DOCSTOWP_PASS: Your Google Docs password

DOCSTOWP_ORIGIN: The ID of the origin folder you want to draw the docs from

DOCSTOWP_DESTINATION: This one’s optional, but if defined it will put the docs in this folder when everything is finished processing.

To define a variable:

define( 'DOCSTOWP_USER', 'example@gmail.com' );

To get the ID of the folder, look to the URL. It will look something like this:

https://docs.google.com/#folders/folder.0.!–ID STARTS HERE, AFTER THE 0 and the period –!

A quick note: I decided to hardcode the variables in wp-config instead of using a setting because I’d rather store a plaintext password on the server than in the database. Obviously, neither are ideal, so if anybody has a better solution I’m all ears.

Marrying Google Docs and WordPress (or really any CMS)

At the BDN, we love WordPress, but we didn’t feel it was the right place to have our reporters working. Even with distraction-free writing, there’s too much cruft that would confuse reporters or that we don’t want reporters touching. We wanted something lightweight, fast, simple to grasp and that would make editing easier.

Enter Google Docs. It’s easy to use, lightweight and a large number of people are already familiar with it. And, it’s got beautiful tools that enable real-time collaboration with reporters and editors.

That’s where we decided to start. Reporters write stories in Google Docs, and the docs are sorted in a series of collections, which are shared out to everybody. We have collections for each desk, such as metro, state, sports, etc.; collections for workflow, such as Needs Copy Editing, On Hold, etc.; and collections for actions, such as Send to Publish and Published.

The action collections are important — they’re how docs actually get from Google to WordPress. When we first started using Docs with our sports department in August or September of last year, Docs was much different, more neanderthal. But, that was nice in some ways. The docs came through with inline markup, and Docs supported XML-RPC, which made it easy to connect the two systems.

Then came an upgrade, which was on the whole a good one. It combined what they’d learned building Wave to bring real-time collaboration to a new level. But it also eliminated XML-RPC support, and docs aren’t marked up as nicely as they used to be.

So, we delved into the API, and I think what we have now is an even nicer system than XML-RPC could provide. You can find a version of what we built in the WordPress Plugin Repository.

On the whole, the process is fairly simple. When we’re all done copy editing an article, the doc goes into a collection called Send to Publish. We have a script running every two minutes that will grab docs from that collection, process them, and move them to WordPress. Then the script takes the doc out of Send to Publish and puts the doc in a collection called Published.

Here are a few features the plugin provides:

  • If the doc is a new post, it will go into WordPress as a draft. If the doc has already been put into WordPress, it will update the previous post.
  • Usernames in Docs must correspond with usernames in WordPress.
  • If a doc is in a collection and there is a corresponding category in WordPress, it will automatically put that the post in that collection. Else it will put it in the default category.

There are a few filters and actions in the plugin that allow you to extend it, and the plugin actually currently comes with one extender, which will strip out comments into a custom field and normalize the content. It’s a slightly stripped-down version of the extender we use.

We actually go one step further by fielding data using delimiters. We name the doc with a slug instead of a headline, and then the first line of the doc becomes a headline, followed by a pipe (|). After that comes the body copy, and at the very end we can add another pipe. Anything after that last pipe acts as a comment. We do this because we also use the API to put things, such as AP stories, into Google Docs, and we couldn’t figure out a way to add comments via the API.

To install the plugin, you’ll need to upload it to your /wp-content/plugins/ folder. Right now there isn’t an extender for wp-cron (there will be soon), so you’ll have to put the action on a page and point a cron job to it. I do this like so:

$docs_to_wp = new Docs_To_WP();
$gdClient = $docs_to_wp->docs_to_wp_init( 'me@example.com', 'mypassword' );
//We're just going to call one function:
$docs_to_wp->retrieve_docs_for_web( $gdClient, ID of origin folder, ID of destination folder );

As always, the code is on Github.

Chronicling the BDN on WordPress