Bitwacker Associates

Since Summer 2010 I’ve had the privilege of working as a research engineer at the Tetherless World Constellation (TWC) at RPI, primarily helping the team in the execution of various projects related to their association with the Obama Administration’s Data.gov initiative. One of those projects is an applet for the Elsevier SciVerse Hub portal. The following is from the description page for our application.

Data.gov Dataset Search (Profile View)

The US Government Dataset Search application is an easy way for SciVerse users and developers to search from among over 300,000 available US government datasets at http://data.gov to automatically find matches to their queries. Based on the user’s SciVerse Hub query, searches are simultaneously made against all datasets published through Data.gov as well as the RDF-converted data and related demos at the Linking Open Government Data (LOGD) portal, created by the Tetherless World Constellation (TWC) at Rensselaer Polytechnnic Institute (RPI).

Any user with the ability to search SciVerse Hub can use the US Government Dataset Search application. The application and the government data it exposes are made available free of charge. The US Government Dataset Search application is targeted at both SciVerse end users (researchers) and application developers interested in applying government datasets to their applications. Researchers utilizing SciVerse Hub are able to discover and access contextually relevant data from the US Government. Developers may utilize SciVerse Hub to identify RDF-converted data sets based on the US Government data and access this data in their applications through SPARQL endpoints or retrieve the datasets themselves.

How the US Government Dataset Search application works: For each SciVerse query the user makes, a keyword search across all current Data.gov datasets is made via a SPARQL endpoint at the TWC LOGD portal. A summary of these results is presented on the Hub search results page. Detailed results are presented in tabular form in the ‘Canvas’ (larger) view by clicking on any link. On the canvas view links are provided directly to the Data.gov dataset description pages as well as RDF-converted versions of these datasets at the TWC LOGD portal. Note that faceted search is not available with the application and only the original query in Hub willbe submitted.

All queries are made against the LOGD SPARQL endpoint at http://logd.tw.rpi.edu/sparql The application also makes use of the Google Visualization toolkit.

This application is optimized for Firefox, Chrome and Internet Explorer 8.

For more information about creating mashups using Data.gov datasets, please check out RPI’s Linking Open Goverment Data (LOGD) Portal at http://logd.tw.rpi.edu

About the TWC Linking Open Government Data project: The TWC LOGD team investigates opening and linking government data using Semantic Web technologies. TWC LOGD actively develops tools for the large-scale translation of government-related datasets into RDF, linking them into the ‘Web of Data’ and providing demos and tutorials on various means for consuming linked government data, including creating mashups, applications and data visualizations. The TWC LOGD Portal was awarded second place (open division) at the 2010 Semantic Web Challenge, held during the 2010 International Semantic Web ConferenceISWC2010.

About the Tetherless World Constellation at RPI: The Tetherless World Constellation addresses the emerging area of Web Science, focusing on the World Wide Web and its future use. Faculty in the constellation lead explorations into the principles that underlie the Web; enhance the Web’s reach beyond the desktop and laptop computer; and develop new technologies and languages that expand the capabilities of the Web. TWC researchers use powerful scientific and mathematical techniques from many disciplines to explore the modeling of the Web from network- and information- centric views. TWC’s objectives include making the next generation web natural to use while being responsive to the growing variety of policy and social needs, whether in the area of privacy, intellectual property, general compliance, or provenance. The Tetherless World Constellation is designing new techniques to explore social, scientific, and legal impacts of the evolving technologies deployed on the Web.

News about the TWC/Elsevier US Government Dataset Search Application

1. Featured in Looking Back at 2010 at Rensselar RPI News & Events (20 Dec 2010)
2. SciVerse Hub Application Connects Researchers with U.S. Government Datasets Information Today (20 Dec 2010)
3. U.S. Government Dataset Search Opens Data.gov to Scientists Data.gov website (14 Dec 2010)
4. New Application Allows Scientists Easy Access to Important Government Data RPI News & Events (10 Dec 2010)
5. New Application Allows Scientists Easy Access to Important Government Data Lab Manager Magazine (13 Dec 2010)
6. New Application Allows Scientists Easy Access to Important Government Data EurekAlert (10 Dec 2010)
7. New Application Allows Scientists Easy Access to Important Government Data Physorg.com (10 Dec 2010)
8. New Application Allows Scientists Easy Access to Important Government Data FirstScience.com (10 Dec 2010)
9. New Application Allows Scientists Easy Access to Important Government Data NewsoDrone.com (10 Dec 2010)

UPDATE: I’m currently developing an iGoogle Gadget version of the SciVerse app, based on the same core queries. A screen shot of the “profile” view of that app appears below. In addition to enabling me to monitor the health of our systems from my desktop, it also enables me to test out possible features for the SciVerse app itself.

iGoogle Gadget version of the US Government Dataset Search app