Sean Boisen: Web Services

xpound.org and Web 2.0 Bible tagging

Wed, 02 Aug 2006 02:03:55 GMT

xpound.org is a new Web 2.0 site that provides passage search, blogging, and social connections, but with an interesting new twist of Bible tagging, along the lines of del.icio.us. (I'm not sure if it's pronounced with equal stress like "slashdot", or "ex-POUND", like the verb) The basic idea of tagging is that, rather than a top-down, authoritative organization and labeling of knowledge, people can simply attach whatever labels make sense to them, in a bottom-up, unstructured (and, some would say, chaotic) fashion. The natural advantage of this kind of folksonomic tagging is that, at internet scale, it can overcome a lot of the messiness, while highly structured knowledge management approaches don't always scale. As with other tagging sites, there's no guarantee that what somebody tags as, say, africa, will have meaning to anybody else. But it means something to the person who tagged it, and thus becomes a highly personalized way to organize information.

I think using this approach for Scripture makes some sense, and i've blogged about it previously. But i also have some questions. With del.icio.us, the item being tagged is clearly defined: it's a URL. But what's the natural unit for tagging Scripture? Verses are one answer, but they often don't have enough context. Books are generally too large, and chapter divisions don't necessarily line up with the content you'd want to tag. Of course, you can tag arbitrary passages: but here's where the comparison to del.icio.us breaks down. With del.icio.us, others who use the same tags as me can point me to sites i didn't know about. But where the passages aren't necessarily bounded the same, aggregation doesn't work quite the same way.

Here's an example: a few days ago Josh tagged Ephesians 5:3-4 with five tags: gratitude, greed, immorality, impurity, and saint.

3 But sexual immorality and all impurity or covetousness must not even be named among you, as is proper among saints. 4 Let there be no filthiness nor foolish talk nor crude joking, which are out of place, but instead let there be thanksgiving. (Eph.5.3-4)

If you search on these tags (like gratitude), you'll find this unit. If instead you search by verse for Eph5.3, you still see the tag for gratitude, presumably inherited from the larger unit that was tagged (though you could argue that gratitude really only applied to verse 4). And of course, the following verses also talk about immorality and impurity, though (since they weren't included in these tags) they're not retrieved.

One of the most interesting new capabilities that del.icio.us creates is knowledge discovery: if i find someone who has bookmarked several of the same sites as i have, i can go look at their other bookmarks, and often find new sites i was unaware of. This provides a kind of search by likeminded community intelligence, a really interesting counterpart to typical web search engines. I haven't found this capability in xpound, but it would be a great addition.

(Hat tip to the ESV Blog for pointing me to xpound.org)

Google Homepage Daily Bible Verse

Sat, 13 May 2006 02:50:49 GMT

Here's a fun little hack. For a while now, Google has offered people a way to customize their homepage (to do this, you need a Google account). This allows you to add things like RSS feeds by URL, or certain other bits (like weather, quote of the day, or Gmail) that Google has pre-defined.

They recently released an API that enables you to customize the content even further by creating (in Google parlance) a "gadget": for example, you can parse an RSS feed and format it yourself in HTML. This is especially useful for Daily Verse feeds: i don't generally want to see a reference to a verse, i want the verse content itself.

So i've put together a Google gadget that parses the ESV's Daily Verse RSS feed and displays the verse content, along with the reference (hyperlinked to the ESV site). To get it (assuming you have a Google homepage),

Click on the "Add Content" link on the upper left
On the next page, click on the "Add by URL" link (to the right of the "Search Homepage Content" button)
Paste http://www.semanticbible.com/blogos/gems/dailyverse.xml into the box and click Add

Hyper-Concordance and the ESV Web Service

Sat, 14 Jan 2006 16:57:28 GMT

I spent some time last weekend working on changing the hyper-concordance to a MySql backend. The current implementation simply generates a large number of static HTML files: easy to implement, but a pain to move that much data around. Since there's a file for each term, that's about 3000 files, and 30+ Mb of data. Worse, each verse is repeated for each of its indexed terms:

"For the kingdom of heaven is like a householder who went out early in the morning to hire laborers for his vineyard.(Matt.20.1, RSV)

winds up being stored nine times.

The obviously superior approach, unimplemented not because i'm stupid but because i'm lazy, is to put each verse in a database, create an index of terms to verses, and then serve pages that are generated dynamically and styled on the fly.

But my recent ruminations on Web 2.0 buzz got me thinking that it might be time to try building on the ESV Web Service API instead. Here's an outline of my thinking:

Use the same perl code i already have to map inflected terms back to their bases (more about this here: by the way, this is the only thing that seems even modestly new to me about the hyper-concordance)
a term request gets mapped into a series of verse requests using doPassageQuery (note i can't use doQuery: that would defeat the mapping back to base forms). Looks like you can retrieve multiple passages by specifying something like "matt.15.23, matt.15.24, matt.15.32, matt.15.39" as the reference (four verses from Matt 15 with different forms of "send").
the resulting XML gets tokenized to identify the base terms, and the processed to add in the hyperlinks and some CSS styling (for example, bolding the query term)
the results get sent to the browser

Some remaining practical questions:

the largest entries in my (yet unreleased) ESV index have hundreds of verses (i think "say" is the current winner). This can be reduced by adding more things to the stopword list, but only at the cost of losing them as hyper-terms. Will the API hold up when queries reference this many passages?
Is this acceptably fast?
is this well-behaved enough for the daily limit of 500 queries? I don't think i have that many users based on server logs, but i would be nice to be scalable

I'm looking forward to experimenting with this approach: stay tuned.

More Fun with Firefox and ESV Search

Fri, 12 Nov 2004 14:10:34 GMT

I keep finding new features i like in Firefox. The latest is keyword search in the "location bar" (where you normally type a URL). Type "weather" there and you get weather.com. Type "stock vz" and you get a quote for Verizon stock (please buy some and help get my options above water!).

The behavior is a little hard to predict, though. Entering "time" comes up empty-handed ("The operation timed out when attempting to contact time": pun unintended i'm sure), but "time now" brings up this page with (you guessed it) the current time. It seems to be hit or miss: "bible" brings up Bible Gateway, "new testament" brings up Goodacre's blog, "concordance" brings up this page with interfaces for searching Strong's. "semanticbible" (no spaces) brings up my site, with a space "semantic bible" it brings up Glenn's reference to my site, but "semantic" itself brings up nothing. I'd love to have a real description of what's happening behind the scenes.

The cool thing is you can define your own. Here's how i set up a passage search against the ESV:

Ctrl-B to view the bookmarks in the sidebar
right-click on Quick Searches to create a New Bookmark
Name = "ESV Passage Search"
Location = http://www.gnpcb.org/esv/search/?passage=%s
Keyword = "esv"
Building on their existing examples, i put 'Type "esv <ref>" to ook up passages in the English Standard Version Bible' for the description

Now you can go to the location bar, type "esv Col 3:16", and get the verse. The capability is fully general: whatever terms you supply get substituted for the %s in the location URL. Items in your search bar also have this capability (e.g. you can do "amazon <bookname>" and search that way).

Alas, entering "blogos" as a quick search doesn't bring up my blog as it should, but instead that upstart blog on language and technology that appropriated my name a full year after i'd been using it (me, bitter? never! ).

ESV Text for Blogging Via Webservice

Mon, 23 Aug 2004 14:32:45 GMT

I haven't posted an ESV webservice hack in a while, but since i'm on vacation, i had some time for recreational programming, and whipped up something i've wanted for a while: a simple program to take one or more verse references, retrieve the results from the ESV web service, and reformat them slightly to make it easier to compose blog posts. Here's the full story.

Is it crazy to program while on vacation? No! I find it energizing and enjoyable, and i don't usually have enough time for things like this.

Scripture Indexes in OPML

Sat, 20 Sep 2003 21:33:57 GMT

OPML (Outline Processor Markup Language) is an XML format designed by Dave Winer of Radio Userland. It's a lightweight XML structure meant for simple outlines that promotes sharing and hyperlinking. The Radio Userland weblogging tool also incorporates some nice OPML features, like an outline-based editor and automatic rendering and publishing mechanisms.

I wrote an XSL transformation to convert my Scripture indexes into an OPML format, sans the text of the verse itself (but hyperlinked to the ESV site). For the imperatives of Jesus from Luke's Gospel, here's the OPML version that corresponds to this XML version.

This is really only relevant if you already have some OPML tools at hand: otherwise it's just another format for the same information. I'm still learning about the benefits of OPML, but (given appropriate tools) it allows others to "subscribe" to these files. Since things always change, that might be a benefit. It doesn't directly expose all the attributes in the Radio Userland tool (like the scope, whether it's implicit, or who the imperative was directed to), though you can find them through a right-click and selecting Debug (though that seems like a curious interface). But like i said, i'm still learning...

More on Using Scripture Indexes ...

Wed, 17 Sep 2003 14:01:12 GMT

This is an update on the technique for creating posts from XML indexes of Scripture using the English Standard Version web services API. The earlier version is here (but i've replaced the code with newer version, so if you try to run the old one it may not quite work now).

I decided it made more sense to separate each stage of the process:

[read the rest...]

Sat, 09 Aug 2003 12:32:11 GMT

Ben Edgington has a nice explanation of using the English Standard Version web service via PHP.

Mon, 16 Jun 2003 11:54:06 GMT

Using Scripture Indexes to Create Posts

Tue, 03 Jun 2003 02:47:56 GMT

It didn't take me long when collecting verses for the Imperatives of Jesus project to get tired of all the cutting and pasting. Once i found the English Standard Version web services API, it was clear that provided a better way to compose the content. I had some programming time this weekend, so i worked out some code that i hope will get the project moving again. [read more ...]

Search the HoCo Library from an Amazon page

Sat, 19 Apr 2003 13:36:08 GMT

How many times have you been browsing Amazon, found an interesting book, and wondered whether you could just borrow it from the library?

Here's a "bookmarklet" for looking up books in the library catalog: Howard County, MD Library. Drag it to your link toolbar (not favorites! it doesn't work there).

How this works:

Search Amazon for a book of interest: note the big long hairy URL includes the ISBN number, which is a unique identifier
Click on the link on your tool bar, which runs a Javascript program
This parses the ISBN number out of Amazon's URL, constructs an appropriate URL for HoCo's IPac library catalog service, and submits it
If the book is there, you'll see the catalog display: otherwise you'll see something like "sorry, there are no matches"

This is the brainchild of Jon Udell's LibraryLookup project, a specific instance of turning URL's into APIs. Check these tips if it doesn't seem to be working.

Search the English Standard Version Bible

Sat, 19 Apr 2003 12:59:34 GMT