Sean Boisen: SemanticBible

Blogos RSS Feed has Moved: Please Update Your Reader URL

Thu, 31 Aug 2006 11:49:44 GMT

I've moved to a new blogging platform (goodbye Radio Userland, hello WordPress).

But if you read through an RSS aggregator (this is really important, so pay attention):

This is the last post to the current RSS feed (http://www.semanticbible.com/blogos/rss.xml)
You must change your feed URL to keep reading Blogos: the new feed is http://semanticbible.com/blogos/feed/.
If you've only been subscribed to a specific channel (e.g. http://www.semanticbible.com/blogos/categories/semanticbible/rss.xml), those have moved as well: the new one for SemanticBible-only posts is http://semanticbible.com/blogos/category/semanticbible/feed/ (note 'categories' -> 'category'), and others are constructed in similar fashion

If you read directly from the website, everything will work as before at my preferred URL, http://www.semanticbible.com/blogos/. The new site includes several syndication buttons that make it easy to add Blogos to your Bloglines, MyYahoo!, or other readers.

If you have any problems with this, please send me (sean) an email at semanticbible daht com. I don't want to lose any readers in the transition (there aren't that many to start with!).

Lexical vs. Conceptual Semantics for Humility

Tue, 08 Aug 2006 13:09:23 GMT

In a comment on my recent thoughts on semantic search, Matt asks a reasonable question: "Wouldn't Louw-Nida help?" Since i've recently gotten a copy of Logos 3 Scholar's Library: Silver (i'll have a lot more to say about that later, but here's the preview: it's a fantastic resource), i tried it out. For this particular question, the answer appears to be no.

Humility is under 88/G, Moral and Ethical Qualities and Related Behavior/Humility (note this is a conceptual label for the passage: the word humility doesn't actually occur). Related words here would include:

lord (as in "lord it over"): 37/D, Control, Rule/Rule, Govern.
exercise authority: same domain and subdomain
servant/serve: 35/B, Help, Care For/Serve
slave: either the same subdomain as "lord [it over]", the more figurative sense, or more literally as 87/E, Status/Slave, Free

This isn't too surprising: Louw-Nida is a lexical resource, but the fundamental issue here (and the point of my post) is that there are lots of significant semantic concepts above the level of words. That's exactly what makes notions like "topic" slippery in practice.

Search Interfaces for the Composite Gospel

Sat, 29 Jul 2006 18:47:48 GMT

I'm preparing a new version of the Composite Gospel Index pages, to standardize around the ESV text, and hopefully provide both more usability and more visual appeal. Designing an interface for this data poses some interesting challenges. There's a wealth of different attributes available, and while some (like traditional verse references) are familiar to most Bible students, i'm hoping to get outside the box a bit and do some novel things.

The whole point of the Composite Gospel is to provide a different way to look at the story of Jesus' life, in particular one that is more oriented around stories, many of which are common to multiple Gospels, and to show how they fit into the whole. So i'm hoping to reinforce this in the new interface. Right now there are two ways to access the Composite Gospel, the typical entry point being the Pericope Index, a traditional single static page listing the pericope ID, title, and references, with hyperlinks to the content pages. It's got a number of faults:

as soon as you click through to an individual pericope (here's Pericope 118, Jesus sends out the twelve disciples), you're back to looking through a keyhole, without the view of the whole sequence. It would be better to have a view of the whole index alongside the content for a selected pericope.
there's no help for finding pericopes with specific titles or Scriptural references (other than browser search)
while you can easily see how many sources are behind a given pericope (it's just a matter of how many columns are filled in its row in the table), the significance (as evidenced by size) is buried. Pericope 153: Jesus teaches about forgiving othersis only two verses: the next one, Pericope 154: Jesus tells the parable of the unforgiving debtor, has 13 verses. But there's no visual clues to this in the index.
let's face it, it's just ugly :-/

The individual pages themselves have different navigational elements: next/previous pericope, and also next/previous for a given Gospel author. These are okay as far as they go: my major complaint is they don't go far enough. I'm also hoping to add more supplemental information:

other pericopes with similar topics or content. For example, though i consider the cleansing of temple early in John (Pericope 031: Jesus clears the template) to be different from the one during the Passion Week (Pericope 249: Jesus clears the template again), clearly one ought to have a "see also" link to the other.
a list of names in the pericope in view, with navigation to other pericopes which mention the same name

It will be a while before i can do all this, though!

I've been searching for some time for the right visual metaphor (and corresponding interface code) to provide a much more visual index to replace the current text-heavy index. It would be great if you could scan a clear visualization of which authors covered a particular story, and how much content there is for it (number of tokens). Likewise, when you've selected an individual pericope, you should have a clear view of where it fits into the entire sequence.

In preparing for this, i got interested in the distribution of sources (an individual author's version) by their size. This graph shows that, binned in groups of 10: the black trend line smooths this a little further with a moving average (window of 3). There's quite a bit of variety (no surprise), ranging from a single source with just 9 tokens (Luke's description of the beginning of Jesus' Galilean preaching ministry, " And he was preaching in the synagogues of Judea.", Pericope 048: Jesus preaches throughout Galilee), to a single source with 566 tokens (Pericope 119: Jesus prepares the disciples for persecution, found in Matthew). But there's some approximation of a normal distribution (with an elongated tail on the high side), and clearly the bulk have from 30 to perhaps 270 tokens, with values near the median of around 30-40 instances (since i'm binning, this number itself isn't very meaningful). This suggests the cases i need to optimize for: i should be able to fit up to about 270 token displays on something close to a single page view (these days that really means 1024 x 768 pixels, though surprisingly i still get 15-20% of my visits from people with 800x600 displays).

Ultimately, i'd love to have a rich treemap interface to support exploring the data in a variety of different ways (this was the substance of my presentation at the Society for Biblical Literature last year). As publisher Tim O'Reilly notes in a recent post, treemaps are really made to be interfaces, not graphs: their power lies in your ability to interact with them to explore the data. Unfortunately, i don't know how to do this live on my website: i don't have permission to host the Treemap software i use myself from the University of Maryland, and i don't know of a good substitute (O'Reilly's post is about a Rails implentation, but that's outside my current scope).

Topic Labels and Semantic Bible Search

Tue, 18 Jul 2006 11:49:55 GMT

But Jesus called them to him and said, "You know that the rulers of the Gentiles lord it over them, and their great ones exercise authority over them. It shall not be so among you. But whoever would be great among you must be your servant, and whoever would be first among you must be your slave, even as the Son of Man came not to be served but to serve, and to give his life as a ransom for many." (Matt.20.25-28)

I've been thinking about topic labels for Scripture passages lately: a deceptively simple idea that's quite hard to nail down. The notion of topic includes many different things: a person might be a topic (Jesus talks about John the Baptist in Luke.7.24-30), but every mention of a person probably isn't a topic in quite the same sense (the same passage mentions the Pharisees, but the passage isn't really about them, it simply mentions them). Sometimes key words and phrases are topics ("luxury" is a word in the same passage, and a relatively distinct one at that: it only occurs 4 times in the New Testament). But if that's what you mean by a topic, then word searches will usually find what you want. The toughest cases (and therefore the most interesting ones) are when you don't have a distinctive lexical item for a topic decision.

The classic Librarian Problem is that whatever i call a topic may have different meaning to someone else, or fall outside the conceptual schema they're using for searching (Shirky has a nice overview of this). The kind of folksonomic tagging popularized by del.icio.us works well at a personal level (i know what my "facets" tag means to me, even though you may not), and it works well at the larger level because enough others might happen to use the same tags that aggregation adds value. I expect this kind of tagging for Scripture will start to show up in some interesting ways in the next year under the Web2.0 rubric.

Here's what got me thinking about this: i was reading Humility by Andrew Murraythis morning (highly recommended, by the way), and he discusses the passage above as an example of Jesus' teaching about humility. I'd agree (as would Naves, and most other topic-oriented indexes): but if you wanted to label such passages in some automated fashion, what evidence would you use? The words "humble" and "humility" are nowhere to be found, and neither are their direct antonyms like "proud". Jesus mentions the contrasting examples of Gentiles who "lord it over them" and others who "exercise authority over them": but these complex semantic constructs aren't easy to take apart (and the first one isn't very typical English: the Contemporary English Version's translation of "order their people around" is arguably more natural). Certainly being the servant of others implies the personal trait of humility, but the relationship is quite abstract.

Just another argument for why this kind of annotation of Scripture will probably be done the old-fashioned way (by hand) for the foreseeable future ...

Lexical Statistics in the New Testament

Sat, 15 Jul 2006 21:51:54 GMT

I've been putting some of the data behind the Hyper-concordance into MySQL, in preparation for computing some statistics on lexical co-occurrence. Along the way, i've been collecting some numbers that i thought others might find interesting. There are a number of other sources for NT statistics: for example, this page from Prof. Felix Just shows words per verse per chapter per book (in the Greek NT).

What's different about the numbers below is that they're based on Hyper-concordance's approach, which groups various inflected forms under their base form (what linguists call a lemma). For example, 'saying', 'says', and 'said' are all pooled under 'say' (as it turns out, the most common lemma in the New Testament, with 1946 occurrences). In the example from the Hyper-concordance home page (Mark.4.24), there are 10 content lemmas (9 of them unique) in this verse of 30 words: "say", "pay", "attention", "hear", "measure" (twice), "use", "still", "more", "add".

	Count	Unique
terms	73872	6333
base terms	73872	4526
name words	6638	593
non-name words	67234	3933
singletons	1444	1444
name words	281	281

"Count" is the actual instances, as opposed to the unique values (which we could call the content vocabulary of the New Testament). Some comments:

As a textual corpus, the New Testament is relatively small by modern lexico-statistical standards: only about 8000 verses, with a vocabulary of only a few thousand words. I take some consolation from the modest vocabulary size: i'm interested in creating lexical semantics for these terms, and while ~4500 terms is far from trivial, it's not so large as to be completely impossible to consider.
"name words" here means nothing more than a word written with a capital letter, about 1 in 10 words, which is actually rather large. I've only found three words that occur both capitalized and uncapitalized. The two obvious ones are God/god and Lord/lord: can you guess the other? (answer at the bottom)
the ratio of terms to base terms is really a measurement of the compression induced by the lemmatization approach of the Hyper-concordance. I'd expect this difference to be much larger for a larger corpus.
"singletons" here means words which occur exactly once (sometimes called hapax legomena). Clearly there can't be any variation in form here, so the instance and unique counts are the same. This is actually rather small, probably another consequence of the small corpus size: as a rule of thumb, for many large and general corpora, roughly half the words occur only once (though that's words, not lemmas), a consequence of Zipf's Law.
the 11 most common words:
- say (1946 instances)
- God (1343)
- come (1120)
- all (1006)
- Jesus (964)
- go (749)
- man (745)
- Lord (657)
- see (622)
- no (569)
- know (543)

Caveats:

this is all based on the ESV text, your mileage will certainly vary for other translations. You could argue (with some merit) that all such counts should be performed on the Greek text, rather than an English one. However, since the ESV takes an 'essentially literal' approach, i'd argue that the magnitude will generally be roughly correct, though of course the exact numbers will be slightly different.
Of course, these numbers for base forms depend on how you map forms back to their bases: i think my approach is credible, but certainly not perfect (i doubt 'perfect' here could even be well-defined).
the Hyper-concordance omits 44 function words that are very common and not very contentful (in information retrieval terms, stop words). I'd argue this is a good thing, but you might think otherwise.

(The second word that occurs in both capitalized and uncapitalized forms is much less obvious, though you'll figure it out if you think a lot about it ...)

Minor Update to Hyper-Concordance

Tue, 04 Jul 2006 23:17:41 GMT

I noticed quite a few errors in the server log for hyper-concordance hits where the folder name (an initial) was capitalized: this suggested that some browser/user agents were having case-sensitivity issues. So i've posted a minor revision to upper-case the folder names: if you have trouble, please let me know.

Disambiguating Names in the New Testament

Wed, 28 Jun 2006 11:50:22 GMT

I'm still working away (far too slowly for my impatient tastes) on the first complete first of New Testament Names, a semantic knowledgebase of named things in the New Testament and their relationships: you can get a sense of it from these representations of a browser prototype. But i'm also looking beyond to what will come next (one reason these projects take too long! I keep starting new ones ...).

After cataloging the names and their information, clearly the next step is to add Scriptural references. The first pass here can be done automatically (it's largely just string matching). But of course, there are a lot of different Johns, Marys and Simons in the New Testament, and it's a lot more useful to know which one is which: this is something people do so easily they hardly recognize it, but it can be surprisingly tough to do automatically.

As an example, there are 36 mentions of "Joseph" in the ESV NT text (in 35 verses: Acts.7.13 mentions him twice). Obviously in the birth narratives of Jesus, Joseph refers to Jesus' (earthly) father: by my count, that's 14 of the 36 references. Joseph of Arimathea is mentioned in the Passion narratives, since he provided a tomb for Jesus to be buried in: 7 of the 36 are this Joseph. As an aside, here's a case that requires a little more than string matching: Luke.23.50, "a man named Joseph, from the Jewish town of Arimathea". You'd need to be pretty smart about the use of context to figure out which Joseph this is.

For other cases like John.4.5, where the mention of Joseph refers to the Old Testament figure, only real human understanding of the text can determine the correct reference. This Joseph is more frequent outside the Gospels (though he's in Luke's genealogy), 10 of the NT Josephs in all. There are also two references in Acts to Joseph who the apostles nicknamed Barnabas, and then a few others: Jesus' brother (Matt.13.55, perhaps also Matt.27.56), and two Josephs in Luke's genealogy (Luke.3.24 and Luke.3.30).

My guess is maybe 80% of the name references in the New Testament can be easily disambiguated, either because they're not ambiguous in the first place, or because simple heuristics clarify them. But for the rest, somebody will actually have to look at them and make a decision (in a very few cases, some really hard decisions). Maybe Amazon's Mechanical Turk is an appropriate mechanism: the ESV blog reports on an interesting experiment here.

Portable Hyper-Concordance

Sun, 25 Jun 2006 16:03:24 GMT

A pastor wrote about using the Hyper-Concordance when he's traveling and unable to connect to the Internet. Since it's just a bunch of static HTML files (well, a BIG bunch: about 4500) with relative hyperlinks, it's no problem, as long as you've got enough space for your own local copy.

If you're interested too, then:

Download the data (15Mb .zip file: this may take awhile on slower connections)
Unzip it somewhere: you'll need about 80 Mb of disk space
Open the local version of the home page (/wherever_you_put_it/hyperconc-home.html) in a web browser, and you should be in business. You'll need Javascript enabled to get Size By Frequency/Size Equally behavior.

Now if i could just figure out how to work around IE's CSS problems so it looked better ...

New ESV Hyper-Concordance Released

Sun, 14 May 2006 17:00:44 GMT

It's hard to believe it's been three years since i first put the New Testament Hyper-Concordance up on SemanticBible. But today, i'm happy to announce that i've released a new version (1.2) of the Hyper-Concordance, one of the more popular features on SemanticBible. The start page that you should bookmark has moved here.

Some of the highlights:

The new release uses the English Standard Version text, thanks to the generosity of the folks at Crossway Bibles. In my opinion, this is the best contemporary English translation for study purposes, and i'm really happy to be able to offer it here.
I've added tabbed navigation to make it easier to find terms of interest, and cleaned up the displays.
There's a new feature that displays terms according to their frequency: you can turn it off if you like, but i find it helpful in navigating a long list of terms.

I'm still catching up with all the documentation on the website, so feel free to let me know if you find broken links or other problems. The cosmetics when viewed with Internet Explorer v. 6 are off in many cases, and downright bad on the index pages, because of IE's poor support for CSS. If you're a CSS maven and can tell me how to fix it, that would be great, though i'm not going to worry about it too much since you're better off with a more modern browser like Firefox anyway.

If you really prefer the previous RSV version (despite some known problems), you can find it here. I don't intend to maintain or update it, however.

More on RDF Bible Vocabulary

Fri, 28 Apr 2006 12:05:51 GMT

JP van de Giessen had a few comments on the previous post, some of which i want to address here. He asks about the relationship between Book and the class hierarchy under Text: doesn't Book belong there? The answer is no: this is what i meant in my brief statement that Book is an abstract class. That is, it doesn't talk about a particular rendering of a Bible book, but the general notion of a book. For example, Romans is a book authored by Paul, which addresses certain subjects, mentions certain people, etc. Those are true whether the particular text is in English, Italian, or Bengali, and regardless of which version it is. Other attributes, like what words are used (and how many of them there are), or the name of the woman Paul mentions in the beginning of chapter 16, are particular to specific texts and translations: in the KJV this is Phebe, in the ESV Phoebe, but it's Febe in the Reina-Valera Spanish translation. These are facts about Texts, not Books (as conceptualized here)

However, there probably should also be a subclass of Text that represents (the Textual representation of) an entire book.

Tim also left a comment about the difficulties of assigning authorship (i used the word problematic), and as i've thought more about this, i think it makes sense to either leave it out entirely, or only make statements that are relatively uncontroversial (though even that line isn't always clear). At minimum, it can be viewed as an add-on to the basic facts about the books themselves.

A Draft RDF Vocabulary for Bible Books

Sun, 23 Apr 2006 13:32:00 GMT

I've created a draft RDF vocabulary for books of the Bible, and i'm looking for feedback before officially publishing it. The vocabulary describes the books (with fairly traditional subclasses like NewTestamentBook > Epistle > GeneralEpistle), textual divisions, and authors. It includes some basic properties like names (in common, abbreviated, and full forms), authorship, and references for passages. RDF seemed to capture the majority of the most useful information, and it comes with fewer processing requirements, so i've used that rather than OWL (though that means losing a few niceties like inverse functions, and cardinality and disjointness constraints).

There's no information in this that would surprise anybody: the main benefit i see is providing URIs that uniquely define these books as resources for the emerging Semantic Web, something i haven't seen "out there" yet. This follows the recommendation of the Architecture of the World Wide Web that everything of importance should have a URI (Tim Berners-Lee even suggests that you provide one for yourself). So this provides a definitive URI for e.g. the book of Romans - http://semanticbible.org/ns/2006/05/biblebooks#Romans - which other resources can refer to (including creating their own versions and stating equivalence) and extend. These URIs are names, not (yet) retrievable resources: so if you visit any of these in a browser, you won't find anything (but that's accepted practice for URIs).

Note that "Book" here is an abstract class, not any particular version or translation of one (those are Texts in this scheme). I got here by starting to layer additional OWL on top of the Composite Gospel Index in RDF (which has been out for nearly a year and a half now), and realized i hadn't properly defined the vocabulary i was using. So here i'm attempting to provide a fairly generic basic vocabulary that other vocabularies can build upon.

Some issues:

i'm not completely happy with the URI naming conventions where authors and conventional book names collide (e.g. Mark, both an author and the conventional name of a Gospel). The current version uses the shorter name in constructing the identifier for the author (http://semanticbible.org/ns/2006/05/biblebooks#Mark), and a longer version for the book (http://semanticbible.org/ns/2006/05/biblebooks#GospelOfMark). That's alright, but it means some book URIs are formed differently than others (e.g. http://semanticbible.org/ns/2006/05/biblebooks#Romans, where no prefixing of "LetterTo" or other special treatment is required). An alternative is to use a separate namespace for authors, so you'd have http://semanticbible.org/ns/2006/05/authors#Micah for the author, and http://semanticbible.org/ns/2006/05/biblebooks#Micah for the book. But i'm a little uneasy with encoding classes in the namespace. Since there are distinct RDF properties for the names themselves, however, maybe this isn't such a big deal.
Typically you'd separate out instance data from the vocabulary itself, so the vocabulary can be used to define your own instances. In this case, though, the instances are really part of the reference information, and a closed set (at least in the conventional view!), so i've included them.
I've included a properties for authorship, though of course that's problematic, even if you only intend to represent traditional authorship claims.
The list of text types isn't very complete, though perhaps it's good enough for the New Testament, my main focus for the time being. Longer term, it ought to include other things like Prayer, perhaps Song?
I need to add more documentation before publishing it (though most of it ought to be clear).

Frankly, it seems to me this ought to be the work of some more respectable standards body like SBL or OSIS, rather than little old me. But since i'm already at a point where i need something like this, i'm forging ahead, and i'd appreciate any feedback that sem-web savvy Blogos readers might have.

The vocabulary is located here. Note this is a temporary location and should not be considered stable: it will definitely move once finalized.

SBL Paper: Weaving the New Testament into the Semantic Web

Fri, 24 Mar 2006 13:15:35 GMT

I was excited to learn that my paper has been accepted for the next Society for Biblical Literature annual meeting, in Washington DC this November. The title probably won't surprise regular Blogos readers: "Weaving the New Testament into the Semantic Web". Here's the abstract:

The World Wide Web, as a network of documents, has become a staple of everyday life, including Biblical scholarship. Recent activity by the World Wide Web Consortium (W3C) envisions moving beyond a web of HTML documents to a web of concepts and meaningful data called the Semantic Web. Foundational concepts include the use of Uniform Resource Identifiers (URIs) to describe resources, the Resource Description Framework (RDF) for describing properties of resources, and the Web Ontology Language (OWL), now a W3C recommended standard for representing ontological relationships in a web-friendly way. This paper will briefly overview what the Semantic Web is, and why it is relevant to Biblical scholars. It will also describe several Semantic Web projects underway at SemanticBible.org. For example, the NT Names project (http://www.semanticbible.org/ntn/ntn-overview.html) is producing a freely-sharable semantic catalog of people, locations, and other named objects in the New Testament, as well as various kinship, interpersonal, and locational relationships between them. Other projects address the creation of semantically-organized lexical resources, and semantic relationships within and between the Gospels. The paper will close with additional suggestions for how Semantic Web technologies can enable better integration, access, and search for Biblical data.

Perceptive readers will easily spot recent Blogos topics like lexical resources and the Composite Gospel Index. I'm hoping to make some real progress on the Imperatives and some other classification efforts as well.

It's been fortunate that the last two congresses (Philadelphia last year, DC this year) have been close enough that i could easily attend: as a self-funded amateur, i'd have a harder time justifying the expenses if i had to cough up a plane ticket too. But i found last year's meeting (my first) stimulating, and enjoyed giving my presentation on Visualizing the Gospels, so i'm definitely looking forward to another one. Hopefully this time i'll be better able to set my introversion aside and take more time to interact with others, especially those i know from cyberland.

Bratcher's New Testament Index

Wed, 22 Mar 2006 12:56:51 GMT

Yesterday i got a copy of the New Testament Indexby Robert G. Bratcher. This slim volume (only 37 pages) is nothing more than what the title suggests: "a guide in locating passages in the New Testament dealing with a wide range of subjects, persons and places." Since i've been thinking lately about indexes and indexing schemes, i wanted to compare it in detail to other approaches like Nave's Topical Bibleand similar works. Bratcher's index was published in 1963 as part of the Helps for Translators series by the United Bible Societies: so you'd expect it to be practical in orientation. Bratcher's introduction says it's intended for "readers who do not have a more complete Bible Concordance": most Bible software today has one of those, but concordances aren't the whole story in terms of making information accessible (more about that in a future post, i hope).

Here are the subjects from the "B" section. I've omitted names of people and places, which account for about half of the entries: there it's hard, though not impossible, to improve on a concordance. I've also left out the references to spare my fingers:

Baptism, baptize:
1. John the Baptist
2. Jesus
3. Christian
4. with the Spirit
Beast
Beatitudes
Beelzebul (references followed by "See Devil")
Belial (references followed by "See Devil")
Believe: see Faith
"Benedictus" [Luke.1.68-79 in case you're not familiar with that term]
Betray, betrayer
Birth
1. physical
2. spiritual
Bishop(s): see Church ministers
Blaspheme, blasphemy
Blessed
1. general
2. addressed to Christ or God
Blind, blindness
1. physical
2. spiritual
Blood
1. animals
2. men
3. Christ
Boast: see Pride
Body, bodies
1. human
2. of Christ
Book
1. general
2. of life
Born: see Birth
Bread
1. material
2. spiritual
3. of the Presence
4. Lord's Supper
Bride, bridegroom
Burial
1. of Jesus
2. other
3. figurative

Sometimes less is more: if you're looking only for general material, a selective index like Bratcher's has an advantage in that it is not exhaustive, so you don't have to wade through every last entry for common topics. And clearly this goes beyond a simple word concordance, which wouldn't contain "Benedictus". Some of the sub-heading lead in the general direction of taxonomic classification: Christian baptism versus that of John the Baptist. The contrast of physical and spiritual sub-headings occurs several times: directly under Birth and Blindness, and in related ways under Bread and Burial (where "figurative" might also be called "spiritual"). There's also some simple semantic grouping (for Believe, see Faith; for Bishop, see Church ministers).

As with all content indexes, it's interesting what's included and what's left out. Under "Book", we find the special NT idiom "Book of life", with seven references, all but one in Revelation. However, we don't find "bread of life", an idiom used several times in John's Gospel (though all in Pericope 125), under "Bread".

We know indexes are both imperfect and incomplete (Shirky has some typically thought-provoking comments on this). I would assume that the field of library science, where this is a well-established professional concern, has looked carefully at the critical quantitative issues of how well people are able to satisfy their information-seeking goals using existing indexes. However, i don't know of (but haven't looked for) any studies on this specifically related to concordances or Scriptural indexes. I'm still mulling over Stephen Smith's search statistics from the ESV site: but of course this only tells you about searching, not finding.

Draft OWL Ontology for Louw-Nida

Sun, 19 Mar 2006 22:39:09 GMT

I've created a draft OWL ontology for Louw-Nida's lexicon. It's not ready for publication yet, so rather than put it up on SemanticBible, it's here. It may move, i'm not sure i'll maintain or extend it, offer void where prohibited by law, etc.: but you're welcome to copy it or take a look. This is just a rendition of the domains and subdomains, over 700 in all. Alas, none of the lexical entries themselves as instances are included, which is really the useful meat: that material is copyright-protected. The intellectual property issues of converting somebody else's information to an ontology aren't clear to me, frankly: that's material for a lengthy later post. But i hope i'm not abusing their information here.

As published, their work consists of 93 top-level domains. However, in their introduction, they indicate that these have some further grouping: i've included that here, as Object Domains (1-12), Event Domains (13-57), and Abstract Domains (58-91). The picture to the left shows this, hiding the details for these three (large) groups. Those are below, in unreadably small versions: all these pictures link to their larger, full-size equivalents, but as you can tell from the ones below, the dimensions of the layout are such to not be very usable in the large version either. A more useful visualization might be a treemap where the size of each subdomain indicated the number of senses within it: i don't have that data right at hand, but it shouldn't be too hard to get.

The properties will show i'm thinking about relationships to WordNet, but i haven't gotten very far yet.

Below are images for Object Domains, Event Domains, and Abstract Domains:

Rethinking the New Testament Lexicon

Fri, 17 Mar 2006 23:44:54 GMT

The enduring interest of Greek lexicons like Strong's and Thayer's for ordinary students of the Bible is a testimony to our general desire to deepen our understanding of God's word. Maybe that helps explain why these tools are still so widely used, despite their many limitations. Suppose we were to put aside our existing conceptions and experience, and design a new Greek lexicon from scratch: what characteristics would it have?

For starters, it ought to be written for the web: the tools we use now obviously started life as print editions, and in many instances have only barely crossed over. The electronic versions of Thayers, for example, uses typography and indentation to indicate multiple senses: these should be distinguished structurally, both to enable clear separation of content and presentation, and also to allow applications to select which information they use. Entries should have unique URIs to support web-based retrieval. Hyperlinks should be authored in entries, not added after the fact by program (since that's an imperfect process).

One of the great advantages of Strong's lexicon for people who don't know Greek is the numeric key for each entry (assuming you have a resource keyed to Strong's system). This approach has some flaws: for one thing, Strong missed a few numbers in his scheme (i've always wondered if he just fell asleep at one point, and then, by the time he realized his error, couldn't bear to go back and redo the scheme). More problematic, the New American Standard concordance (NASEC), while keyed to Strongs, has a slightly different set of entries. For example, #3587 (xurao, 'to shave') is #3587a in NASEC. Other instances of this verb linked from the KJV turn out to be #3587b (xureo, also glossed as 'to shave') in NASEC. Of course, we've gotten much better manuscripts in the 350+ years between the KJV and the NASB, so it's not surprising. But this seemingly minor difference breaks the system: now the numbering system is stovepiped, because it doesn't work across resources any more (at least for these alternately-numbered cases, which are actually quite common). For example, the tool tips in eSword don't work for #3587 if you don't have the right dictionary selected. It would be better to consolidate these schemes and make them genuinely cross-functional.

You might wonder, why not just use the Greek words themselves? But the Web, while providing increasing support for non-ASCII character sets, is still strongly tilted that way. Zack Hubert's site takes this root: but it's just hard for me to stomach URLs like http://www.zhubert.com/word?word=%E1%BC%90%CE%BB%E1%BD%B5%CE%BB%CF%85%CE%B8%CE%B1%CF%82&;root=%E1%BC%94%CF%81%CF%87%CE%BF%CE%BC%CE%B1%CE%B9&number=666946.

Along with an identifier, there should be a full orthographic representation, including accents and breath marks, in Unicode: other encodings only bring trouble with them. For those who can't read the Greek character set, a pronunciation guide and an ASCII transliteration are helpful: the BGreek transliteration scheme may not be very aesthetically appealing, but it's consistent and probably the best candidate for a current standard, at least among Greek scholars. Ulrik Petersen's version of Strong's is keyed to both the traditional numbers and BGreek transliteration, e.g. you can find 'logos' at http://ulrikp.dk/strongsgreek/goto.php?strongs=LOGOS. Most decent transliterations could be generated automatically from the Greek (if it can't, that suggests some irregularies that might cause you to question whether it's really a good transliteration!).

Thayer's lexicon includes part of speech and gender (some tools make this more helpful by expanding out abbreviations). It also includes a few kinds of proper nouns (personal names, locations, ethnicities like Scythian [though interestingly not Greek used in the same way]), though not others (religious festivals like Passover). And there are some minor inconsistencies: "Stoic" is capitalized in Thayer's, but not indicated as a proper noun. NASEC indicates names by capitalization of the term in the entry: correct, but why be subtle when you can be explicit? Of course, some of these are judgment calls, since the originals weren't capitalized to indicate names. But all this information belongs in a good lexicon.

Several lexicons including derivational information describing the etymological roots of a given term. Like all lexical information in a language you don't know, this can be abused, because derivational history doesn't necessarily translate into current meaning. I can't remember how many times i've heard some pastor talk about the Greek word 'dunamis' ("power") being the root from which we get our word dynamite! It's true, but it doesn't tell you anything about what it meant to Greek speakers 2000 years ago. But all this is no reason not to include derivational history, when it's available (and reliable!).

Given the huge amount of study that's already gone into the Greek lexicon, it's also critical to cross-reference existing lexicons: Thayer's, but also Louw-Nida, the Theological Dictionary of the New Testament, etc. Of course, to the extent these are also referencable web-based resources, you can provide a hyperlink, which is vastly more useful than a pointer to paper, or a link in a proprietary system. It doesn't seem productive to try to overlay one master reference scheme on top of all these existing works, but it would be nice to at least provide the mapping. In other words, 'logos' ("word", "expression", etc.) maps to G3056 for Strongs/Thayers/NASEC, 4:69, 505 in TDNT, various entries in Louw-Nida (statement: 33.98; speech: 33.99; gospel: 33.260; treatise: 33.15; etc.), and so forth.

Last of course are the definitions themselves. This is by far the hardest part, since it requires some sense of who the audience is and what kind of definition they need. Strongs and Thayers have long served as lexicons for the common person: something like TDNT is too extensive and too expensive for this purpose, though critical for more detailed study. Louw-Nida is specifically aimed at the needs of translators (one of the things i find makes it particularly helpful). Theologians have different needs from students learning Greek.

Several lexicons include a list of the ways a particular word is used in a given translation: KJV for Strongs and Thayers, NASB for NASEC. This is something i'd argue really doesn't belong in a dictionary: that the task of a concordance, and in the information age we ought to be able to generate these pretty easily, as seems to be the case with the Greek lexicons at crosswalk.com. Some Bible study applications apparently filter these out in creating their resources: they don't seem to be in e-Sword, for example, and i don't miss them.

It's a long list: i compiled it in part to help myself realize just what a huge undertaking such a lexicon might be. But given the wealth of information on-line today, and how much knowledge we have about the Greek lexicon, it's definitely time we move beyond the simplistic approaches that are current today to something more powerful, more ubiquitously available, and more re-usable and re-purposable.

My own personal inclination at present is to get started on a semantically-organized lexicon along the lines of Wordnet: also a huge undertaking.

Ulrik Petersen's XML Strongs Lexicon

Tue, 14 Mar 2006 11:48:17 GMT

Yesterday i discovered (via Zack Hubert's site) that Ulrik Petersen (home page here) at morphgnt.org has created an XML version of Strong's Greek Lexicon. Furthermore, he's enhanced it by incorporating Unicode versions of the Greek terms: this is a crucial missing element from previous versions. He's even got a nice RESTian interface. Since i've been moving in this direction lately, it was a timely discovery. You can download it here.

Ulrik points out that, as a dictionary, Strong's is somewhat lacking. True enough: but the numbering system itself, apart from the definitions, provides the defacto standard for referencing the Greek lexicon of the New Testament, and numerous other reference works are keyed to Strongs numbers. That makes it an important starting point for any lexical endeavors. At this point, i'd like to start with the numbering scheme, terms, parts of speech, and transliterations and pronunciations, and create an RDF framework in which additional lexicon material can be added, and which can be referenced by other work.

Semantic Web Lexicon for the New Testament, part 2

Mon, 13 Mar 2006 11:50:13 GMT

Tim from SansBlogue left a comment about my post on a Semantic Web lexicon for the New Testament. Comments unfortunately don't work so well on my blog (in particular, the notification mechanism doesn't work, so i don't always know about them), so i wanted to repeat his comment and respond to it more directly, because it raises an important point.

Tim said (in part):

... I wonder if the biggest job, creating a concordance of senses is unnecessary and / or undesirable. Surely one of the key exegetical decisions in such cases is which sense is intended in THIS use. To predetermine this is surely to restrict exegesis unnecessarily, a straight concordance of the Greek and a cross link to the other sense(s) would suffice!

Tim's question is a relevant one, and if the goal were exegesis in the normal sense, i think our current approaches might be good enough for the most part. By "current approaches", i mean something like:

scratch your head over a passage you're reading
go back to the Greek (except for those of you erudites who were already reading Greek in step 1) and identify the lemma in question
go to some reference lexicon and survey the range of meanings for the lemaa, consider the context and other translations, scratch your head some more, and repeat until satisfied

Electronic references and hyperlinks make this process much more convenient: convenience and search alone are enough to keep me using Scripture software rather than the dead tree kind. But the process above is little different from consulting printed works: it's just easier to click a hyperlink than to pull a volume off the bookshelf and thumb through the pages.

My goals are somewhat different, though: i'd like to be able to index content semantically. The glosses that represent the senses for a given word in shorthand can never express the full range of possible meaning, of course, or the actual meaning in a particular context (which ultimately is the critical thing). But they still provide ways to organize content, or facilitate search, that go beyond words alone. As a simple example, i can create a search application that groups all the verses that talk about "light" in the sense of electromagnetic radiation, versus other senses. Perhaps more relevant, i can organize verse content semantically, and find e.g. verses discussing humility, without having to think of all the ways this concept is expressed (humble, lowly, etc.)

By the way, there's some discussion related to this at the Journal of the Hyperlinked Organization, where David Weinberger thinks 2006 will be the year of the Unique ID. We already have a pretty clear standard for uniquely identifying Greek words from the New Testament: the one created by Strong (though only web-enabled in limited ways). I'm not yet aware of one for senses (WordNet would be the closest for English, and Louw-Nida are most of the way there).

The Imperatives of Jesus, and Dake's 800 Commands of the New Testament

Wed, 08 Mar 2006 13:00:09 GMT

Stephen at the ESV Bible Blog remembered my interest in cataloging the imperatives of Jesus, and passed on a recent post by Adrian Warnock reflecting on the 800 commands of the New Testament. Adrian's post links to a PDF file at puritan-books.com, though i slightly prefer this one because it credits the author, one Finnis Jennings Dake, who composed Dake's Annotated Reference Bible. Reading a bit more about Dake's background and his reference Bible frankly put me off, but that's no argument against the validity of the Scriptures he assembles.

The commands are organized in a memorable way: for example,

Seven "Abstains" (mostly from the Jerusalem Council in Acts.15)
Seventy-four "Be's": be glad (Matt.5.12), be reconciled to your brother (Matt.5.24), be content with your wages (Luke.3.14), ...

as well as "Be not's", "Bewares", things to consider, one hundred "Let's", and so forth. Though it would be a mistake to consider these as a new Book of the Law for the Christian, i agree with Adrian's point that "practical preaching that tells people how to live" is worthwhile indeed. Dake's organization tends to obscure the original context of the individual items, always a danger in this list-making kind of approach: i tried to mitigate this a bit in my list from Luke by indicating, for example, when something seemed like a general command rather than specific to a particular situation.

I've recently restarted my effort to complete the imperatives across all four Gospels, including semantic categorization. I first reflected on this notion more than two years ago, and it's daunting work: i've backed up a bit for now to get some lexical resources together first, which is partly how i got here. I'll get a post out soon about where i'm headed, though it's still even more of a work in definition than a work in progress.

The Syntopicon as Ontology

Sat, 04 Mar 2006 16:26:54 GMT

If you're my age and your parents were upper-middle class (like mine were), you may have had a copy of the series Great Books of the Western World in your home. This collection of some 50 volumes covers many of the standards for a liberal education (in fact, our friend Jonathan attends a college based on the Great Books): i sampled many when i was a geeky kid who spent more time thinking that hanging out with friends (come to think of it, that's still the case).

One important contribution by Mortimer Adler, who oversaw the project, is called the Syntopicon (all the major topics are listed on Wikipedia). It's an overview of some 100 key concepts in the history of ideas (e.g. Wisdom), with considerable detail about sub-topics. For example, under Wisdom we have

1. The nature, origins, and kinds of wisdom
- 1a. Diverse conceptions of natural wisdom: the supreme form of human knowledge
- 1b. The distinction between speculative and practical wisdom, or between philosophical and political wisdom
- 1c. Theological and mystical wisdom: the supernatural wisdom of faith and vision; the gift of wisdom
- 1d. The wisdom of God: the defect of human wisdom compared with divine wisdom; the folly or vanity of worldly wisdom
2. Wisdom, virtue, and happiness
- 2a. Wisdom as an intellectual virtue: its relation to other intellectual virtues, especially science and understanding; the vice or sin of folly

and so forth. The first two volumes of the Great Books series consist simply of the Syntopicon, some introductory comments about each topic, and then a gigantic index to the Great Books. So under Wisdom.1d (the wisdom of God), you'll find references to Old and New Testament passages (e.g. James.3.13-18: note the Bible itself is not part of the Great Books series), as well as Aristotle, Aquinas, Melville (Moby Dick), and so on. Given this is all pre-cyberspace, the index alone takes up all kinds of space, and you've got to have the whole bookshelf to benefit from it. These days it would all be different.

I'd long since forgotten about my childhood explorations in the Great Books: but now that i've got ontological glasses on, this might be a very useful thing to convert into a formal ontology (e.g. OWL) for reference purposes. Furthermore, i didn't remember at all that there were references to the Bible embedded in it. I don't know of a machine-readable version of the Syntopicon (though surely they must sell them on CD by now), or how to get plain data out of it, and of course it would be copyright-restricted. But a index between these topics and the passages they reference would be both interesting and useful.

There's also an interesting article here by Internet luminary David Weinberger about the Great Books, Adler, and the problems of alphabetical order.

Getting Things Done with TiddlyWiki, and the Composite Gospel

Mon, 27 Feb 2006 12:40:33 GMT

I spent some time this weekend playing with TiddlyWiki, a "reusable non-linear personal web notebook" developed by Jeremy Ruston. The basic concept is pretty simple: embed enough JavaScript in a single HTML file to support a complete lightweight environment for Wiki-style authoring and hyperlinking. Rather than full-up articles like Wikipedia, TiddlyWiki is built around the concept of microcontent. Anil Dash gets significant credit for defining the microcontent idea (though i can't find the definitive reference):

"... content that conveys one primary idea or concept, is accessible through a single definitive URL or permalink, and is appropriately written and formatted for presentation in email clients, web browsers, or on handheld devices as needed. A day's weather forcast, the arrival and departure times for an airplane flight, an abstract from a long publication, or a single instant message can all be examples of microcontent."

TiddlyWiki is a great execution of a microcontent editor: just enough support to make things easy, mechanisms for tagging and searching, and open/close animations that make it fun. Because it's all in one file, it's highly portable (you can put it all on a USB drive and take it with you, a WikiOnAStick). This would be another approach to the ideas in my recent post on personal information management.

As it happened, i was trying to collect various scattered plans for SemanticBible, in the style of David Allen's Getting Things Done(known to fans simply as GTD). GTD is a popular method for time and productivity management, in particular because it's not method-heavy, but focuses on simple, high-yield approaches. The typical soil in which innovation grows is the intersection of a need and a tool: in this case, i got thinking about how easy it would be to use GTD-style planning in combination with a TiddlyWiki. After 10 minutes of trying to set a few things up myself, i realized it was likely that TWATOT: The Web Already Thought of This(tm). Sure enough, a little googling produced GTD TiddlyWiki Plus, which puts some GTD support on top of TiddlyWiki. This is my fourth or fifth time around looking for an appropriately lightweight environment for maintaining constantly changing plans, and i'm optimistic that this one might grow into a fruitful tree.

Another intersection of interests with TiddlyWiki is my constant search for innovative mechanisms for presenting and interacting with Scriptural content. There's a natural overlap between TiddlyWiki's notion of microcontent on the one hand, and the pericopes of the Composite Gospel Index on the other. Pericopes are chosen to describe a single event (expressed by one or several Gospel authors from their unique perspectives), with a consistent identifier. Hyperlinks are ideal for moving between pericopes in a variety of ways: sequence order (overall, or within an individual author's presentation), links for related content that isn't the same pericope, or top-level organization by historical period, genre, special topics like healings, miracle, parables, and so forth. All along i've had the concept that pericopes in the Composite Gospel should be "authoritatively" taggable as reference indicators of content (though i haven't gotten too far with this yet). The newer notion, coming out of some thinking about Bible study and Web 2.0, is that there's also value in the aggregated folksonomy approach of sites like del.icio.us.

Transforming the Composite Gospel content into a TiddlyWiki framework would be quite easy: each "tiddler" (as Ruston calls a piece of microcontent) has an ID, and some simple formatting. Incorporating some navigational elements that are unique to the Composite Gospel will take a little more thinking and hacking on top of the TiddlyWiki framework. For example, having a sub-title for each tiddler that describes the editor, the creation and editing date and time, isn't very relevant: properties of pericopes would be much better. But i'm excited about the possibilities: look for more, if i can find the time to Get Things Done.

Toward a Semantic Web Lexicon for the New Testament

Fri, 17 Feb 2006 13:16:19 GMT

One of my big goals with SemanticBible continues to be a freely sharable New Testament lexicon that

has a semantic organization
connects existing lexical resources in Greek and English, particularly
- Greek resources like Thayer and Louw-Nida
- English resources like WordNet
lives on the web as a resource (using URIs), freely sharable and structured to be re-usable (which to me means RDF or OWL)

In fact, i was blogging about this stuff well over two years ago: it's discouraging how little progress i've made since then!

Here's a thought experiment about how such a lexicon might be structured, starting from available resources (and happily ignoring the thorny issues of publishers and copyright!). The work involved probably exceeds what i can accomplish, at least right now: that's why it's a thought experiment. But i keep finding myself backing into such a project from different directions.

The fundamental unit of this lexicon is the assignment of a word form to a sense. For example, the verb hubrizo has at least two senses: 'insult' and 'mistreat' (the NAS lexicon includes a third, 'to be insolent'). So there would be two (or three) entries corresponding to the dictionary form hubrizo. Louw-Nida provides this kind of sense-mapping information.

Each lexical entry would have several reference properties:

the lexical form itself in Greek, as well as an English gloss (i can't render Greek adequately in my blog, unfortunately)
the part of speech (here, Verb)
the Strong's number (here, 5195): this keys the entry to several existing lexicons
the domain and sub-domain identifiers from Louw-Nida (in the case of hubrizo/insult, that's Communication (#33) and Insult, Slander (P'), as well as the sense index (33.390, which also includes another sense, enubrizo/insult: this would be a different entry, however)
the corresponding English sense from Wordnet: here, that's the verb synset 00839460, which is an instance of verb.communication, and includes the other verbs "diss" and "affront", neither of which occur (as verbs) in any of the common Bible translations.

With this much information, and a proper Semantic Web or other linkable implementation, you could connect in a number of other resources. Eventually you'd want a concordance back to the text: that takes real work, because you have to determine for each form in the text which sense it is, which can't be done automatically (though maybe it can be bootstrapped).

How is this different than what we have now? Some of it exists on-line, but for dictionary forms, without senses (Zack Hubert's lexicon includes nice RESTful URLs). The pairing of dictionary forms and senses occurs in some lexicons (like Louw-Nida or the New American Standard Greek lexicon), but either in print versions, or inside proprietary software interfaces like Libronix. Nothing exists in OWL or RDF to my knowledge.

Moving from thought experiment to realization is a big task. While the vocabulary of the New Testament is a modest 5500 words, Louw-Nida's lexicon has 25,000 senses. But much of the key information already exists: it's "just" a matter of organizing it, combining it, and transforming it to the right representation. If you're interested in pursuing such a goal (or can contribute toward it in some way), send me an email.

Bootstrapping Louw-Nida Semantics

Sat, 11 Feb 2006 17:33:24 GMT

A slightly different spin on the last post: instead of WordNet senses as the target for lexical semantic annotation, the senses from Louw and Nida's lexicon provide a semantic index and hierarchy that is less well-developed than WordNet but still has valuable information. To do this alignment, you'd have to work from the Greek terms. Following the previous example, homologeo has three senses in Louw-Nida, glossed as 'profess', 'admit', and 'declare'. All three are part of the Communication semantic domain (#33): the first falls under the sub-domain "Profess Allegiance" (33.U), the second under "Admit, Confess, Deny" (33.V), and the latter one (which seems to be the relevant one for Matt.7.23) under "Assert, Declare" (33.P). In this case, it's a slam-dunk: the English term leads you directly to the correct Louw-Nida sense. In other cases, the glosses might be slightly different, though clearly they'd have to be "close" in some sense. Ted Pedersen's WordNet-Similarity module might be useful here.

One operational problem, though: i don't have a machine-readable version of Louw-Nida. I know it's sold in electronic form with programs like Logos, Gramcord, Bibleworks, and others. But typically the data itself is locked up inside the interface: you can't get to it and re-use it for other purposes (here's some similar grumbling about the NASB plus Strongs inside e-Sword). If you know of an appropriate (i.e. still ethical) way to deal with this, please let me know.

Bootstrapping Lexical Semantics for the New Testament

Thu, 09 Feb 2006 12:23:13 GMT

It's a really long path to a semantically-annotated New Testament, and i'm not sure if we'll get there (or even that we know how). But, just as each journey begins with a single step, i've been thinking about some of the early steps it will take, and how to keep moving in the right direction.

One important milestone along the way is a sense-marked text, where each (content) word is annotated with its meaning. Let's take Matt.7.23 as an example

And then will I declare to them, 'I never knew you; depart from me, you workers of lawlessness.'(Matt.7.23, ESV)

Some of these are simple function words (like "to") that don't really carry independent meaning. "declare" and "know", on the other hand, are verbs with important semantics. This is the first step toward a sense-marked text: indicating content words, their dictionary form ("know", not "knew"), and their parts-of-speech (nouns, verb, adjectives, adverbs, etc.). No English New Testament text provides this information today, as far as i know (though something approximating it is available in texts that are indexed to Strong's Greek lexicon, namely the KJV and New American Standard Version).

The next challenge is determining the actual semantics of each (content) word. You can't just look this up in a dictionary because most words are ambiguous, some very much so. This is harder for us to grasp because our amazing ability to understand language in its context makes the process so transparent. For example, the verb "declare" has the following definitions in WordNet 2.1, a semantically-oriented lexicon of English that groups words with similar meanings:

state emphatically and authoritatively; "He declared that he needed more money to carry out the task he was charged with"
announce publicly or officially; "The President declared war"
state firmly; "He declared that he was innocent"
declare to be; "She was declared incompetent"; "judge held that the defendant was innocent"

(and a few others, like making a declaration of dutiable goods to a customs agent). It's pretty easy to eliminate senses 2 and 4. The examples don't help quite enough in distinguishing 1 and 3, but the other information in Wordnet (number of examples of usage, and the semantic hierarchy) make it pretty clear than sense 3 is a more formal kind of declaration (so sense 1 is actually a hypernym or more specific superordinate of this sense!).

This kind of annotation is a little trickier: it requires judgement about what words mean in their context (without context, any of the various sense are possible, though the earlier ones are more likely a priori). But this kind of sense tagging, particularly when coupled with a semantic hierarchy like WordNet, gives you some really interesting new capabilities. Any search engine will let you find Bible verses with the term "declare": some will let you look for wild cards, so "declar*" would find "declare", "declares", "declaring", etc. This doesn't solve the "know" vs. "knew" problem: for that, you still need the dictionary forms. This is what the Hyper-concordance does: and while i don't think it's a "killer app" for Bible search, the data underneath it really is important for these purposes.

Going beyond these cases, though, you can do much better with this kind of lexical semantic annotation: for example, you can search for instances of communication verbs (declare, announce, answer) that have the meaning of communication with words (as opposed to answer meaning the answer to a problem). Of course, we don't have this data for any English versions yet (though some Greek search tools approximate it), so we don't have this kind of search tool either. Solving this problem in a general and automatic way for unrestricted English is still a hard research problem. But we don't need that for the New Testament, because it's a closed corpus: we just need to do the work of marking it up once. That sounds very hard, but there are ways to bootstrap the process (using resources like WordNet, and the fact that we have some versions that are keyed to their Greek terms, which can help with disambiguation).

With that lengthy preamble, then, here are some thoughts on how to jump-start the process of sense-tagging the New Testament.

Take a version like the New American Standard that has annotations of the Greek dictionary forms (via Strong's numbers).
Add the annotation of the English base form (the technical term for this is lemma) so you know that "knew" is an instance of "know". I have code to do this behind the Hyper-concordance: i've only configured it for the vocabulary of the RSV and ESV at this point, and it's not perfect, but it's a pretty good start.
Assigning parts-of-speech can also be usefully approximated through a combination of several factors:
- the fact that some English words have only one part-of-speech (for example, "declare" is only a verb)
- for ambiguous words like "light" (which can be a noun, a verb, or an adjective), the mapping to Greek will often assign a correct part-of-speech, especially for more literal translations like NASB. This isn't perfect, though: in Matt.5.15, "it gives light to all who are in the house", the Greek part-of-speech is a verb, not a noun.
- there are some automated and open source approaches to part-of-speech tagging with good performance (like Eric Brill's tagger), though the New Testament has some fairly unique vocabulary that might do a little worse
Now the tricky part: how to assign senses (assuming you're using WordNet senses as the target)? There are several sources of information:

Using WordNet, you look up the possible senses (technically synsets) for the English term in its relevant part-of-speech, which are ordered by frequency of occurrence. Each synset gives you a set of one or more terms: for example declare#1 and #3 have only the term "declare", while declare#2 has ["announce", "declare"].
the NASB lexicon also has a set of English glosses for the Greek term, ordered by their frequency of use for translation: for "declare", that's Strong's ID G3670, homologeo. homologeo is variously translated throughout the NASB as "acknowledge", "admit", "confess"/"confessed"/"confesses"/"confessing", "declare", "give thanks", "made", "profess", and "promised". Each of these also have frequencies, though they need to be collapsed across variants (like in step #2 above) so you get the count for all forms of "confess".

At this point we need some metrics and heuristics to decide between the various possible WordNet senses. Two easy ones: more frequent senses (in general English) should be more likely in the NT too, and more general senses should be more likely that their subordinate ones. Is the case here with "declare", where one sense is a hypernym of other, unusual for the subset of English represented in a New Testament translation? I don't know. But just making the first choice on these grounds is likely to be right a very large percentage of the time.

There will probably be plenty of rough edges still to work out: what if a NT term isn't in WordNet at all? (but hey, they've got propitiation and leaven, so the coverage must be pretty complete) In general, i haven't had enough experience with WordNet to know how to use it most effectively: but there must be some benefit here. When the glosses from the NASB lexicon overlap with the glosses for a particular WordNet sense, that sense is more likely. The most challenging case is where the WordNet synset only provides one term (like "declare" senses 1 and 3): then what do you do? You could try the approach of taking the immediate hypernyms of each term, and measure overlap (or semantic distance) from the NASB glosses here as well.

Bottom-line: suppose the best you can do is to identify and rank the choices, and present them in an usable interface that makes it easy to manually select the right sense. I'd estimate this could be reviewed in a matter of a few person weeks by someone who understood the issues and was willing to do the work. Though that's still a fair amount of effort, it would create a valuable resource for Bible study and search (not to mention linguistic research in general).

Imperatives of Jesus at xristos.org

Sun, 05 Feb 2006 14:33:33 GMT

While looking at some other materials on Gospel harmonies, i happened to find some powerpoint presentations on the commands of Jesus (focused on Mark's Gospel) at xristos.org. The links on their page to view the slides seem to be broken, but you can download them all here.

I've been looking at this same idea for a while under the rubric of imperatives, and have recently re-started an effort to create a semantic knowledge base in OWL for the things Jesus explicitly told us to do. That's another step along the very lengthy path toward a semantic Bible: hopefully i'll get a detailed description of that project up soon (but it's still in a bit of flux). So i'll be interested to look more closely at their material.