Citation graphing challenge

Academic literature is exploding.

Spencer Weart summarizes the problem quite well in his article "Trend-spotting: Physics in 1931 and today"

"A quick way to compare the situations then and now is to look at the Physical Review. The most obvious difference is size. Last year's volumes take up about 30 times as much shelf space as did the two 1931 volumes—not to mention that the pages have gotten bigger and the print smaller. To be sure, the 1931 student also had to read Zeitschrift für Physik and Nature. Even so, the student could have read every important article in the field. Today such breadth is out of the question; dozens of subfields each publish more than the entire physics community did back then. To get an overview of physics nowadays, you must read review journals; scan news stories in Science, Nature, and PHYSICS TODAY; and—that old standby—talk with professors."

This is a project that I hope people smarter than I will solve. My complaint about current academic search engines like NASA ADS, LANL arxive, and google scholar is that the search results are too flat. A keyword search results in a list sorted by a complicated algorithm that might take into account word frequency, seperation distance, connectedness between cited links, and a whole host of things. However what the search cant tell you is what you might be missing by using the wrong phrases. Academic literature uses citations for a reason. What I would like to see is a search engine that displays its results graphicly in a user friendly graph that allows the user to select different display modes (context grouping, co-citation grouping, author\topic highlighting, 2d\3d representation, interactive navigation).

At first I thought I could get some of this done myself. I wrote a perl script that retrieves two citation or reference lists from NASA ADS entries and finds their intersection. The answers the question "what articles did both of my papers of interest cite" or "what articles reference both of my papers of interest" . These metrics measure co-citation.

It worked well but it was an itty bitty baby step in a large problem. Maybe by next update of this page google or NASA will step up to the plate! In my estimation Tim Brody at Citebase is probably the closest, he recently added cocitation and coreferencing, just like my crummy little perl script but much nicer.