YQL has attracted a large number of OpenData tables thanks to the efforts of the community. But some of these tables don’t end up working properly due to many factors, like recent changes made to the underlying API. Therefore we’ve created two new tools, YQL Table Health and YQL Lint, to help developers see and understand which tables actually work and which ones don’t.
YQL Table Health is intended to provide a quick general overview of how “healthy” the community OpenData tables are:
When you first arrive to the page, you will see a list of all the tables that can be used by YQL. Clicking on one of the entries in the list will cause it to expand and show additional information regarding where the source of the XML file is, what kind of table it is, sample query information, and lastly any errors that were encountered. You can use the controls on the left-hand side to further filter, sort, and search through all this data. If you see a table that doesn’t work, you can contact the author of the table to fix it via github. You may also fork the yql-tables from github and fix or enhance the table yourself.
YQL Table Health uses a sever-side script to iterate through all the tables. Each table’s XML file is loaded into YQL cloud storage before a series of checks are run against that XML file. The test results are then cached in a database as well as memory to serve this data as fast as possible. Updates to caches are triggered by users visiting the page and only fire if the data is older than thirty minutes; requests for an update are also synchronized to prevent a race condition occurring, where two or more requests might be made simultaneously. Last but not least, the data is served to the user through a user-interface built using HTML5 and the YUI Library.
The next tool we’re going to introduce to you is YQL Lint.
YQL Lint is essentially an XML debugger for individual YQL tables. You can enter either a URL to a XML file, or the contents of an XML file, and it will validate this against our schema for syntax flaws. Once the schema check has been passed, we will use YQL to get a description of your YQL table and check to see if it contains a sample query that returns a valid result. YQL Lint essentially relies on the same core backend as YQL Table Health.
Please experiment with these tools and send us any questions or suggestions you might have.
Daniel Park – YQL/Pipes Intern