ScraperWiki is a powerful cloud-based service that lets you scrape data from online documents and websites.
When you write a scraper – a script to pull information from a web resource and then parse out the bits you want – it will execute inside the ScraperWiki environment.
You can store the data that is scraped inside a data store and then access the data from outside the ScraperWiki environment using their API. Scrapers can be written in one of several different languages – Ruby, PHP and Python.
Your script executes inside the Tropo environment, which means you can make direct connections to external resources – like the ScraperWiki API – from within your executing script. There is no need for extra HTTP overhead, and the additional step of posting to a back end server to connect to other APIs or resources.
In the following screencast, I demonstrate how to use Tropo and ScraperWiki to quickly and easily build an airport information system for the Philadelphia International Airport.
This is still a work in progress – I’d like to run this script multiple times per day (ideally, maybe once an hour) to get updates to flight information and ensure that the app has the most up to date flight status. The voice dialog could also use a little tweaking, and I’d like to offer the option of repeating the information.
But even with these refinements aside, it is evident how combining these two powerful cloud resources can generate a pretty useful application in a very short time.
Tropo and ScraperWiki are a powerful combination. Happy flying!