Thursday, October 30, 2008

How-To: Determining Source Reliability On The Internet (Link List)

There are a number of great sites (mostly libraries) where there is good guidance on how to evaluate internet based sources. I intend, in this post, to list some of those sites and identify some up-and-coming tools. Finally, I want to highlight an important contribution to this literature that specifically pertains to intelligence analysis.

Virtually every good research library has a page dedicated to evaluating internet based sources. Some good examples include, among others:

Beyond these resources there are also a couple of new automated tools that are available for checking the accuracy and reliability of some internet sites. Beyond those that look for malware (such as McAfee's Site Advisor), there are two products which I have found particularly interesting as they are primarily designed to examine content.

The first is SpinSpotter (a firefox extension). SpinSpotter (which is "very" beta right now) allows you to annotate web sites for "spin" and to view other people's evaluations on websites that have already been evaluated. At some point (although it is unclear when), a computer algorithm will kick in (once it has learned enough about how to spot spin from thousands of reader's input) and begin to automatically mark up pages. This is when the tool will get really interesting...

The second project, WikiTrust, developed by the University of California, Santa Cruz WikiLab, is designed to use data from any MediaWiki based product (such as Wikipedia or Intellipedia) and, in turn, be able to automatically indicate how "trustworthy" the content of that wiki is. You can actually see a demo of it here based on 2007 data. You can also download the software that will allow you to apply the trust algorithm to any MediaWiki based wiki today. The problem is, of course, that the person applying the code also has to control the wiki (Hmmm...I wonder if Intellipedia uses this...I wonder why Wikipedia doesn't use it now...).

None of these solutions specifically had the intelligence professional in mind, however. This has changed recently with Dax Norman's recent online publication of his 2001 Joint Military Intelligence College/National Defense Intelligence College thesis, usefully titled How To Identify Credible Sources On The Web. Dax is the curriculum manager at the National Cryptologic School and one of the most intelligent and insightful people I know. Possessed of deep experience and a darn good mind, he has spent a good bit of time reflecting on how best to improve the analytic process. As a result, he is always worth listening to.

His thesis is particularly well worth the read for anyone who is interested in the subject. While much as been done in the area of assessing internet sources (see above), his take-away -- a research based checklist of key variables in assessing source reliability -- is as good today as it was in 2001.

If you are interested in the details of this scoring system, how it was derived and validated, I will have to refer you to the thesis. Using the checklist, however, is dead easy. Just check the blocks, add up the total and compare it to the scale on top. While I am virtually certain that Dax would not claim that this checklist should replace analytic judgment, I do think that it is far better than a guess-timate or, even worse, no assessment of source reliability at all.

Wednesday, October 29, 2008

Male Or Female? (GenderAnalyzer)

GenderAnalyzer is a free site that seeks to automatically determine the sex of a particular blog's author. Simply submit a blog to the site and it will come back pretty quickly with a guess about the sex of the author. Overall the site claims about 66% accuracy. UClassify, the company that sponsors the site, also has a free service called TypeAnalyzer which determines something close to a Myers Briggs Personality Type automatically as well (though there are no accuracy statistics associated with this tool).

Analyzing distinct online characteristics for identity markers is rapidly becoming a robust business. Since so much activity online is either functionally anonymous or deliberately deceptive, everyone from advertisers to law enforcement professionals to national security types are trying to figure out who is doing/watching what.

Monday, October 27, 2008

Augmented Reality For Mobile Devices: It's Here... (Dante Technologies via Kzero)

Kzero featured a series of augmented reality videos today but the one that impressed me the most was the mobile application (see video below). I tracked down the maker of the product to a Singapore based company called Dante Technologies. While the site is not very sophisticated, the wide variety of augmented reality products and demo videos is pretty impressive.

Sunday, October 26, 2008

What Is Intelligence? Let's Find Out! (Online Survey)

On two lists to which I subscribe the question has come up within the last six months, "What is Intelligence?" To be truthful, on one of those lists, it was my own series of posts on the subject that got things going, but the heated discussions on both lists suggest that a certain passion for the issue exists.

While these discussions are often eloquent, they typically rely on persuasive argumentation rather than on empirical data. I want to change all that.

I have designed (and one of our research assistants (Thanks, Kathleen!) has created) a short online survey to gather some data about what kinds of activities are "intelligence-like" and which kinds are not.

Click here to take the "What is Intelligence?" Survey


Clicking on the link above will take you to the survey which contains 14 examples of different kinds of activity. You are asked to rank how closely the described activity matches your own internal definition of intelligence on a scale from 1 to 5 (1 meaning that it does not fit your definition at all and 5 meaning that it fits it extremely well). You don't need to come up with a definition; just read the question and let your instincts guide you.

No personally identifying information will be collected. From the compiled results I expect to be able to spot areas of broad agreement and disagreement so that, in the end, we will hopefully be able to focus the debate somewhat.

What is intelligence? I say, "Let's find out!"