<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>YQL Blog &#187; feature</title>
	<atom:link href="http://yqlblog.net/blog/index.php/category/feature/feed/" rel="self" type="application/rss+xml" />
	<link>http://yqlblog.net/blog</link>
	<description>Yahoo! Query Language</description>
	<lastBuildDate>Thu, 22 Jul 2010 17:19:14 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.4</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Adding value to a data feed using YQL Execute</title>
		<link>http://yqlblog.net/blog/2009/06/17/adding-value-to-a-data-feed-using-yql-execute/</link>
		<comments>http://yqlblog.net/blog/2009/06/17/adding-value-to-a-data-feed-using-yql-execute/#comments</comments>
		<pubDate>Wed, 17 Jun 2009 19:00:45 +0000</pubDate>
		<dc:creator>yqlteam</dc:creator>
				<category><![CDATA[feature]]></category>
		<category><![CDATA[tutorial]]></category>
		<category><![CDATA[earthquake]]></category>
		<category><![CDATA[usgs]]></category>
		<category><![CDATA[yahoo]]></category>
		<category><![CDATA[yql]]></category>
		<category><![CDATA[yql execute]]></category>

		<guid isPermaLink="false">http://yqlblog.net/blog/?p=92</guid>
		<description><![CDATA[I want USGS earthquake data.  More specifically, I&#8217;m interested in recent, substantial quakes.  Fortunately, data.gov makes this data easy to find.  After searching through the USGS raw data catalog for the text &#8220;earthquake&#8221;, I choose the Worldwide M2.5+ Earthquakes, Past 7 Days feed, and pull it into YQL for parsing.  It&#8217;s almost perfect, but I [...]]]></description>
			<content:encoded><![CDATA[<p>I want USGS earthquake data.  More specifically, I&#8217;m interested in recent, substantial quakes.  Fortunately, <a href="http://www.data.gov">data.gov</a> makes this data easy to find.  After <a href="http://www.data.gov/catalog/category/0/agency/30/filter/earthquakes/type/">searching through the USGS raw data catalog for the text &#8220;earthquake&#8221;</a>, I choose <a href="http://earthquake.usgs.gov/eqcenter/catalogs/7day-M2.5.xml">the <em>Worldwide M2.5+ Earthquakes, Past 7 Days</em> feed</a>, and <a href="http://developer.yahoo.com/yql/console/?q=select%20*%20from%20atom%20where%20url%3D%27http%3A%2F%2Fearthquake.usgs.gov%2Feqcenter%2Fcatalogs%2F7day-M2.5.xml%27">pull it into </a>YQL for parsing.  It&#8217;s almost perfect, but I want easy access to each quake&#8217;s magnitude, and the magnitude is buried in the &#8220;title&#8221; element.  No worries.  I&#8217;ll use <a href="http://developer.yahoo.com/yql/guide/yql-execute-chapter.html">YQL Execute</a> to split it out and give it its own element in the feed&#8217;s structure.  I can then visualize this data using something like Jon LeBlanc&#8217;s <a href="http://github.com/jonleblanc/yql-utilities/tree/master"><em>js-yql-display</em> project</a> on github.</p>
<p>Here are a few reasons why YQL is perfect for this task:<br />
1) I can take advantage of Yahoo!&#8217;s web-serving infrastructure to fetch, process, and cache the feed, reducing my server&#8217;s exposure and bandwidth costs.  My table is also cached, further reducing bandwidth usage.</p>
<p>2) Because YQL Execute employs standard <a href="http://en.wikipedia.org/wiki/E4X">E4X</a>, I am using and adding to my JavaScript skill set, instead of spending time learning a new language</p>
<p>3) E4X was built specifically for XML manipulation so it has a convenient syntax for this job</p>
<p>4) By using YQL to do the heavy lifting, I can minimize the code I send to the browser and keep it focused on the display logic.</p>
<p>Ok. Ok. Here&#8217;s the code:</p>
<pre>
<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>?</FONT>xml version<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"1.0"</FONT> encoding<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"UTF-8"</FONT><FONT COLOR=BLUE>?</FONT><FONT COLOR=BLUE>&gt;</FONT>
<FONT COLOR=BLUE>&lt;</FONT>table xmlns<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"http://query.yahooapis.com/v1/schema/table.xsd"</FONT><FONT COLOR=BLUE>&gt;</FONT>
  <FONT COLOR=BLUE>&lt;</FONT>meta<FONT COLOR=BLUE>&gt;</FONT>
        <FONT COLOR=BLUE>&lt;</FONT>description<FONT COLOR=BLUE>&gt;</FONT>Extracts magnitude from item title in atom feed and adds it as an element to the item.  We can then filter by magnitude using yql's built-in operators<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>description<FONT COLOR=BLUE>&gt;</FONT>
	<FONT COLOR=BLUE>&lt;</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>select entry from usgs<FONT COLOR=BLUE><B>.</B></FONT>earthquakes<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>
	<FONT COLOR=BLUE>&lt;</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>select entry from usgs<FONT COLOR=BLUE><B>.</B></FONT>earthquakes where entry<FONT COLOR=BLUE><B>.</B></FONT>magnitude <FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>=</FONT> <FONT COLOR=YELLOW>6.0</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>

  <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>meta<FONT COLOR=BLUE>&gt;</FONT>
  <FONT COLOR=BLUE>&lt;</FONT>bindings<FONT COLOR=BLUE>&gt;</FONT>
    <FONT COLOR=BLUE>&lt;</FONT>select itemPath<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>""</FONT> produces<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"XML"</FONT><FONT COLOR=BLUE>&gt;</FONT>
		<FONT COLOR=BLUE>&lt;</FONT>urls<FONT COLOR=BLUE>&gt;</FONT>

			<FONT COLOR=BLUE>&lt;</FONT>url<FONT COLOR=BLUE>&gt;</FONT>http<FONT COLOR=BLUE>:</FONT><FONT COLOR=GREEN><I>//earthquake.usgs.gov/eqcenter/catalogs/7day-M2.5.xml?11d&lt;/url&gt;
</I></FONT>		<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>urls<FONT COLOR=BLUE>&gt;</FONT>
		<FONT COLOR=BLUE>&lt;</FONT>execute<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>!</FONT><FONT COLOR=BLUE><B>[</B></FONT>CDATA<FONT COLOR=BLUE><B>[</B></FONT>

			default xml namespace <FONT COLOR=BLUE>=</FONT> <FONT COLOR=PURPLE>"http://www.w3.org/2005/Atom"</FONT><FONT COLOR=BLUE><B>;</B></FONT>
			<FONT COLOR=RED><B>var</B></FONT> xml <FONT COLOR=BLUE>=</FONT> request<FONT COLOR=BLUE><B>.</B></FONT>get<FONT COLOR=BLUE><B>(</B></FONT><FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>.</B></FONT>response<FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//call the url defined above

</I></FONT>			 	entries <FONT COLOR=BLUE>=</FONT> <FONT COLOR=BLUE>&lt;</FONT>entries<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>entries<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//prep the output object
</I></FONT>				entry <FONT COLOR=BLUE>=</FONT> null<FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//individual entry in xml obj. used in loop below.
</I></FONT>				magnitude <FONT COLOR=BLUE>=</FONT> null<FONT COLOR=BLUE><B>;</B></FONT><FONT COLOR=GREEN><I>//magnitude of quake.  used in loop below

</I></FONT>			<FONT COLOR=RED><B>for</B></FONT> each<FONT COLOR=BLUE><B>(</B></FONT>entry <FONT COLOR=RED><B>in</B></FONT> xml<FONT COLOR=BLUE><B>.</B></FONT>entry<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>{</B></FONT>
				magnitude <FONT COLOR=BLUE>=</FONT>
					entry<FONT COLOR=BLUE><B>.</B></FONT>title<FONT COLOR=GREEN><I>//eg M 3.0, Puerto Rico region

</I></FONT>					<FONT COLOR=BLUE><B>.</B></FONT>split<FONT COLOR=BLUE><B>(</B></FONT><FONT COLOR=PURPLE>' '</FONT><FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>1</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=GREEN><I>//eg --&gt; 3.0,
</I></FONT>					<FONT COLOR=BLUE><B>.</B></FONT>replace<FONT COLOR=BLUE><B>(</B></FONT><FONT COLOR=PURPLE>','</FONT><FONT COLOR=BLUE><B>,</B></FONT> <FONT COLOR=PURPLE>''</FONT><FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT><FONT COLOR=GREEN><I>//eg --&gt; 3.0

</I></FONT>				entry<FONT COLOR=BLUE><B>.</B></FONT>appendChild<FONT COLOR=BLUE><B>(</B></FONT> <FONT COLOR=BLUE>&lt;</FONT>magnitude<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>{</B></FONT>magnitude<FONT COLOR=BLUE><B>}</B></FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>magnitude<FONT COLOR=BLUE>&gt;</FONT> <FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT>
				entries<FONT COLOR=BLUE><B>.</B></FONT>appendChild<FONT COLOR=BLUE><B>(</B></FONT>entry<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT>

			<FONT COLOR=BLUE><B>}</B></FONT>
			response<FONT COLOR=BLUE><B>.</B></FONT>object <FONT COLOR=BLUE>=</FONT> entries<FONT COLOR=BLUE><B>;</B></FONT>
		<FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>execute<FONT COLOR=BLUE>&gt;</FONT>

    <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>select<FONT COLOR=BLUE>&gt;</FONT>
  <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>bindings<FONT COLOR=BLUE>&gt;</FONT>
<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>table<FONT COLOR=BLUE>&gt;</FONT>
</pre>
<p>Now, we can put this table on a server, <a href="http://developer.yahoo.com/yql/console/?q=use%20%27http%3A%2F%2Fgithub.com%2Fspullara%2Fyql-tables%2Fraw%2Ff196ed25002a6ad616d1c53adda5e28ea51f3fe2%2Fusgs%2Fusgs.earthquakes.xml%27%20as%20table%3B%20select%20entry%20from%20table%20where%20entry.magnitude%20%3E%3D%206.0">load it up in YQL</a>, and easily access the magnitude using YQL&#8217;s parser.</p>
<p>For those unfamiliar with E4X, it&#8217;s worth noting the namespace declaration (<code>default xml namespace = "http://www.w3.org/2005/Atom";</code>).  It tells YQL&#8217;s JavaScript engine what kind of structure to expect.  We wouldn&#8217;t be able to access the feed&#8217;s elements without it.  Find the namespaces associated with your data by looking in the xml wrapper:   The <em>Atom</em> namespace governs my feed&#8217;s structure as a whole, which is why it was convenient to declare it as a default.  For access to specific elements using another namespace, e.g. georss data, it&#8217;s be easier to define the namespace locally like this:<br />
<code>var ns = Namespace("http://www.georss.org/georss");</code><br />
and then use it like this:<br />
<code>var latitude = xml.ns::Result.ns::Latitude;</code></p>
<p>Since we&#8217;ve gone to the trouble of defining a YQL table, we may as well add parsing for the <em>summary</em> element, which also contains some useful information in an inconvenient format.  Because this content is a bit more extensive, while still being somewhat predictable, a regular expression works well.  Here&#8217;s the code:</p>
<pre>
<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>?</FONT>xml version<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"1.0"</FONT> encoding<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"UTF-8"</FONT><FONT COLOR=BLUE>?</FONT><FONT COLOR=BLUE>&gt;</FONT>
<FONT COLOR=BLUE>&lt;</FONT>table xmlns<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"http://query.yahooapis.com/v1/schema/table.xsd"</FONT><FONT COLOR=BLUE>&gt;</FONT>
  <FONT COLOR=BLUE>&lt;</FONT>meta<FONT COLOR=BLUE>&gt;</FONT>

	<FONT COLOR=BLUE>&lt;</FONT>description<FONT COLOR=BLUE>&gt;</FONT>Extracts magnitude from item title in atom feed and adds it as an element to the item.  We can then filter by magnitude using yql's built-in operators.  Additionally, it extracts summary cdata, parses it, wraps the parsed data in its own element, and adds this element to the xml output.  <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>description<FONT COLOR=BLUE>&gt;</FONT>
	<FONT COLOR=BLUE>&lt;</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>select entry from usgs<FONT COLOR=BLUE><B>.</B></FONT>earthquakes<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>

	<FONT COLOR=BLUE>&lt;</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>select entry<FONT COLOR=BLUE><B>.</B></FONT>title<FONT COLOR=BLUE><B>,</B></FONT> entry<FONT COLOR=BLUE><B>.</B></FONT>updated<FONT COLOR=BLUE><B>,</B></FONT> entry<FONT COLOR=BLUE><B>.</B></FONT>link from usgs<FONT COLOR=BLUE><B>.</B></FONT>earthquakes<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>

	<FONT COLOR=BLUE>&lt;</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>select entry<FONT COLOR=BLUE><B>.</B></FONT>summary from usgs<FONT COLOR=BLUE><B>.</B></FONT>earthquakes where entry<FONT COLOR=BLUE><B>.</B></FONT>summary<FONT COLOR=BLUE><B>.</B></FONT>type <FONT COLOR=BLUE>=</FONT> <FONT COLOR=PURPLE>"xml"</FONT> and entry<FONT COLOR=BLUE><B>.</B></FONT>summary<FONT COLOR=BLUE><B>.</B></FONT>depth<FONT COLOR=BLUE><B>.</B></FONT>km <FONT COLOR=BLUE>&gt;</FONT> <FONT COLOR=BROWN>99</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>sampleQuery<FONT COLOR=BLUE>&gt;</FONT>

  <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>meta<FONT COLOR=BLUE>&gt;</FONT>
  <FONT COLOR=BLUE>&lt;</FONT>bindings<FONT COLOR=BLUE>&gt;</FONT>
    <FONT COLOR=BLUE>&lt;</FONT>select itemPath<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>""</FONT> produces<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"XML"</FONT><FONT COLOR=BLUE>&gt;</FONT>
		<FONT COLOR=BLUE>&lt;</FONT>urls<FONT COLOR=BLUE>&gt;</FONT>

			<FONT COLOR=BLUE>&lt;</FONT>url<FONT COLOR=BLUE>&gt;</FONT>http<FONT COLOR=BLUE>:</FONT><FONT COLOR=GREEN><I>//earthquake.usgs.gov/eqcenter/catalogs/7day-M2.5.xml?11d&lt;/url&gt;
</I></FONT>		<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>urls<FONT COLOR=BLUE>&gt;</FONT>
		<FONT COLOR=BLUE>&lt;</FONT>execute<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>!</FONT><FONT COLOR=BLUE><B>[</B></FONT>CDATA<FONT COLOR=BLUE><B>[</B></FONT>

			default xml namespace <FONT COLOR=BLUE>=</FONT> <FONT COLOR=PURPLE>"http://www.w3.org/2005/Atom"</FONT><FONT COLOR=BLUE><B>;</B></FONT>

			<FONT COLOR=RED><B>var</B></FONT> xml <FONT COLOR=BLUE>=</FONT> request<FONT COLOR=BLUE><B>.</B></FONT>get<FONT COLOR=BLUE><B>(</B></FONT><FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>.</B></FONT>response<FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//call the url defined above

</I></FONT>			 	entries <FONT COLOR=BLUE>=</FONT> <FONT COLOR=BLUE>&lt;</FONT>entries<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>entries<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//prep the output object
</I></FONT>				entry <FONT COLOR=BLUE>=</FONT> null<FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//individual entry in xml obj. used in loop below.
</I></FONT>				magnitude <FONT COLOR=BLUE>=</FONT> null<FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//magnitude of quake.  used in loop below

</I></FONT>				re <FONT COLOR=BLUE>=</FONT> <FONT COLOR=PURPLE>'&lt;img '</FONT><FONT COLOR=GREEN><I>//img tag opening bracket (note: trailing spaces here and below)
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'src="(http://earthquake\\.usgs\\.gov/images/globes/[\\d_-]+\\.jpg)" '</FONT><FONT COLOR=GREEN><I>//img src - capture
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'alt="([\\d\\.]+&amp;#176;(?:N|S) [\\d\\.]+&amp;#176;(?:W|E))" '</FONT><FONT COLOR=GREEN><I>//img alt - ignore (we already have coords from georss)

</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'align="(left|right)" '</FONT><FONT COLOR=GREEN><I>//img align - ignore
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'hspace="(\\d+)" '</FONT><FONT COLOR=GREEN><I>//img hspace - ignore
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'/&gt;'</FONT><FONT COLOR=GREEN><I>//img tag closing bracket
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'&lt;p&gt;'</FONT><FONT COLOR=GREEN><I>//opening p tag

</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'(\\w+, \\w+\\s+\\d+, \\d+ [\\d:]+) UTC'</FONT><FONT COLOR=GREEN><I>//utc date - capture (note: variable amt of whitespace btwn month and day)
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'&lt;br&gt;'</FONT><FONT COLOR=GREEN><I>//br tag
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'(\\w+, \\w+\\s+\\d+, \\d+ [\\d:]+ (?:AM|PM)) at epicenter'</FONT><FONT COLOR=GREEN><I>//local date at epicenter - capture

</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'&lt;/p&gt;'</FONT><FONT COLOR=GREEN><I>//closing p tag
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'&lt;p&gt;'</FONT><FONT COLOR=GREEN><I>//opening p tag
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'&lt;strong&gt;Depth&lt;/strong&gt;: '</FONT><FONT COLOR=GREEN><I>//descriptive text w/ strong tags

</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'([\\d\\.]+) km '</FONT><FONT COLOR=GREEN><I>//depth in kilometers - capture
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'\\(([\\d\\.]+) mi\\)'</FONT><FONT COLOR=GREEN><I>//depth in miles (enclosed in parenthesis) - capture
</I></FONT>					<FONT COLOR=BLUE>+</FONT> <FONT COLOR=PURPLE>'&lt;/p&gt;'</FONT><FONT COLOR=BLUE><B>,</B></FONT><FONT COLOR=GREEN><I>//closing p tag

</I></FONT>				cdata <FONT COLOR=BLUE>=</FONT> null<FONT COLOR=BLUE><B>,</B></FONT>
				summary <FONT COLOR=BLUE>=</FONT> null<FONT COLOR=BLUE><B>;</B></FONT>

			<FONT COLOR=RED><B>for</B></FONT> each<FONT COLOR=BLUE><B>(</B></FONT>entry <FONT COLOR=RED><B>in</B></FONT> xml<FONT COLOR=BLUE><B>.</B></FONT>entry<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>{</B></FONT>

				magnitude <FONT COLOR=BLUE>=</FONT>
					entry<FONT COLOR=BLUE><B>.</B></FONT>title<FONT COLOR=GREEN><I>//eg M 3.0, Puerto Rico region
</I></FONT>					<FONT COLOR=BLUE><B>.</B></FONT>split<FONT COLOR=BLUE><B>(</B></FONT><FONT COLOR=PURPLE>' '</FONT><FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>1</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=GREEN><I>//eg --&gt; 3.0,

</I></FONT>					<FONT COLOR=BLUE><B>.</B></FONT>replace<FONT COLOR=BLUE><B>(</B></FONT><FONT COLOR=PURPLE>','</FONT><FONT COLOR=BLUE><B>,</B></FONT> <FONT COLOR=PURPLE>''</FONT><FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT><FONT COLOR=GREEN><I>//eg --&gt; 3.0
</I></FONT>				entry<FONT COLOR=BLUE><B>.</B></FONT>appendChild<FONT COLOR=BLUE><B>(</B></FONT> <FONT COLOR=BLUE>&lt;</FONT>magnitude<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>{</B></FONT>magnitude<FONT COLOR=BLUE><B>}</B></FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>magnitude<FONT COLOR=BLUE>&gt;</FONT> <FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT>

				cdata <FONT COLOR=BLUE>=</FONT> <FONT COLOR=BLUE>new</FONT> RegExp<FONT COLOR=BLUE><B>(</B></FONT>re<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>.</B></FONT>exec<FONT COLOR=BLUE><B>(</B></FONT>entry<FONT COLOR=BLUE><B>.</B></FONT>summary<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT>

				summary <FONT COLOR=BLUE>=</FONT> <FONT COLOR=BLUE>&lt;</FONT>summary type<FONT COLOR=BLUE>=</FONT><FONT COLOR=PURPLE>"xml"</FONT><FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>!</FONT><FONT COLOR=BLUE><FONT COLOR=BLUE>-</FONT><FONT COLOR=BLUE>-</FONT></FONT> differentiate <FONT COLOR=BLUE>this</FONT> summary obj from native summary obj w<FONT COLOR=BLUE>/</FONT> type <FONT COLOR=PURPLE>'html'</FONT> <FONT COLOR=BLUE><FONT COLOR=BLUE>-</FONT><FONT COLOR=BLUE>-</FONT></FONT><FONT COLOR=BLUE>&gt;</FONT>

					<FONT COLOR=BLUE>&lt;</FONT>img alt<FONT COLOR=BLUE>=</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>2</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT> align<FONT COLOR=BLUE>=</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>3</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT> hspace<FONT COLOR=BLUE>=</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>4</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT> src<FONT COLOR=BLUE>=</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>1</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT> <FONT COLOR=BLUE>/</FONT><FONT COLOR=BLUE>&gt;</FONT>

					<FONT COLOR=BLUE>&lt;</FONT>date<FONT COLOR=BLUE>&gt;</FONT>
						<FONT COLOR=BLUE>&lt;</FONT>utc<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>5</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>utc<FONT COLOR=BLUE>&gt;</FONT>
						<FONT COLOR=BLUE>&lt;</FONT>local<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>6</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>local<FONT COLOR=BLUE>&gt;</FONT>

					<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>date<FONT COLOR=BLUE>&gt;</FONT>
					<FONT COLOR=BLUE>&lt;</FONT>depth<FONT COLOR=BLUE>&gt;</FONT>
						<FONT COLOR=BLUE>&lt;</FONT>km<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>7</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>km<FONT COLOR=BLUE>&gt;</FONT>

						<FONT COLOR=BLUE>&lt;</FONT>mi<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>{</B></FONT>cdata<FONT COLOR=BLUE><B>[</B></FONT><FONT COLOR=BROWN>8</FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>}</B></FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>mi<FONT COLOR=BLUE>&gt;</FONT>
					<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>depth<FONT COLOR=BLUE>&gt;</FONT>
				<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>summary<FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE><B>;</B></FONT>

				entry<FONT COLOR=BLUE><B>.</B></FONT>appendChild<FONT COLOR=BLUE><B>(</B></FONT>summary<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT>

				entries<FONT COLOR=BLUE><B>.</B></FONT>appendChild<FONT COLOR=BLUE><B>(</B></FONT>entry<FONT COLOR=BLUE><B>)</B></FONT><FONT COLOR=BLUE><B>;</B></FONT>

			<FONT COLOR=BLUE><B>}</B></FONT>
			response<FONT COLOR=BLUE><B>.</B></FONT>object <FONT COLOR=BLUE>=</FONT> entries<FONT COLOR=BLUE><B>;</B></FONT>
		<FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE><B>]</B></FONT><FONT COLOR=BLUE>&gt;</FONT><FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>execute<FONT COLOR=BLUE>&gt;</FONT>

    <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>select<FONT COLOR=BLUE>&gt;</FONT>
  <FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>bindings<FONT COLOR=BLUE>&gt;</FONT>
<FONT COLOR=BLUE>&lt;</FONT><FONT COLOR=BLUE>/</FONT>table<FONT COLOR=BLUE>&gt;</FONT>
</pre>
<p>Now we&#8217;re talking!  <a href="http://developer.yahoo.com/yql/console/?q=use%20%27http%3A%2F%2Fgithub.com%2Fspullara%2Fyql-tables%2Fraw%2Ff196ed25002a6ad616d1c53adda5e28ea51f3fe2%2Fusgs%2Fusgs.earthquakes.xml%27%20as%20usgs.earthquakes%3B%20select%20entry.summary%20from%20usgs.earthquakes%20where%20entry.summary.type%20%3D%20%22xml%22%20and%20entry.summary.depth.km%20%3E%2099">Check it out in the console</a>.</p>
<p>Here are a couple implementation-level notes:<br />
1) this code will generate an additional <em>summary</em> object, i.e., it doesn&#8217;t replace the pre-existing one.  If the later behavior is preferred, replace<br />
<code>entry.appendChild(summary);</code><br />
with<br />
<code>entry.summary = summary;</code></p>
<p>2) the regular expression syntax used above is just the standard syntax for JavaScript, but be aware that the html is rendered using html entities, so the content I&#8217;m parsing using the regular expression looks different in the YQL console.  For example, add this as the first line inside the for loop:<br />
<code>y.log(entry.summary);</code> <br />
This will print the <em>cdata</em>-wrapped html to the <em>diagnostics</em> section of the YQL output.  Instead of &#8220;&lt;img src=&#8221;http://earthquake&#8230;-65.jpg&#8221; alt=&#8221;19.192&amp;#176;N &#8220;, as we see in the raw xml feed, it looks like &#8220;&amp;lt;img src=&#8221;http://earthquake&#8230;-65.jpg&#8221; alt=&#8221;19.192&amp;amp;#176;N &#8230;&#8221;, On the server, it actually <em>is</em> the raw html, so the regular expression must be constructed accordingly.</p>
<p>To conclude, this post presents a couple ways to restructure a USGS data feed using YQL Execute so it&#8217;s more convenient to consume.  I&#8217;ve also given a couple tips for working with E4X and YQL.  Because YQL does the fetching, processing, and caching for me, my data delivery is speedy and my client-side code is light.</p>
]]></content:encoded>
			<wfw:commentRss>http://yqlblog.net/blog/2009/06/17/adding-value-to-a-data-feed-using-yql-execute/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Getting stock information with YQL and open data tables</title>
		<link>http://yqlblog.net/blog/2009/06/02/getting-stock-information-with-yql-and-open-data-tables/</link>
		<comments>http://yqlblog.net/blog/2009/06/02/getting-stock-information-with-yql-and-open-data-tables/#comments</comments>
		<pubDate>Tue, 02 Jun 2009 16:32:23 +0000</pubDate>
		<dc:creator>yqlteam</dc:creator>
				<category><![CDATA[feature]]></category>
		<category><![CDATA[tutorial]]></category>

		<guid isPermaLink="false">http://yqlblog.net/blog/?p=79</guid>
		<description><![CDATA[One question that the YQL and Pipes teams get asked is &#8220;how can I get stock quotes? There isn&#8217;t an API for it on developer.yahoo.com&#8221;. Interestingly, while there isn&#8217;t a more traditional web service API, Yahoo finance does provide a very nice way to get a lot of well structured information on a given company. [...]]]></description>
			<content:encoded><![CDATA[<p>One question that the YQL and Pipes teams get asked is &#8220;how can I get stock quotes? There isn&#8217;t an API for it on developer.yahoo.com&#8221;. Interestingly, while there isn&#8217;t a more traditional web service API, Yahoo finance <strong>does</strong> provide a very nice way to get <strong>a lot</strong> of well structured information on a given company. For example, here&#8217;s the <a href="http://finance.yahoo.com/q?s=yhoo">Yahoo finance page on YHOO</a>:</p>
<p><a href="http://yqlblog.net.p2.hostingprod.com/blog/wp-content/uploads/2009/06/picture-91.png"><img class="aligncenter size-medium wp-image-81" title="Finance page for Yahoo" src="http://yqlblog.net.p2.hostingprod.com/blog/wp-content/uploads/2009/06/picture-91.png" alt="" width="503" height="224" /></a></p>
<p>You&#8217;ll notice that there&#8217;s a little &#8220;download data&#8221; link to the right. If you click the link, it generates a CSV file dynamically with almost all the pricing information on the page. The problem is how to understand what fields the &#8220;f&#8221; parameter actually produce in the CSV file. Luckily <a href="http://www.gummy-stuff.org/Yahoo-data.htm">someone has already done that hard work</a>. So now we have a link with a bunch of configurable parameters to get lots of lovely stock information for multiple stock symbols. It is an API of sorts, but that CSV file is still a hard to work with, somewhat cryptic to use, and the data in it is a bit messy.</p>
<p>Enter YQL open data tables. If you don&#8217;t want to know &#8220;how&#8221; this works, and just want a really cool open data table and API to give you stock quotes, then <a href="http://developer.yahoo.com/yql/console/?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22YHOO%22%2C%22AAPL%22%2C%22GOOG%22%2C%22MSFT%22)%0A%09%09&amp;env=http%3A%2F%2Fdatatables.org%2Falltables.env">give this a go in the YQL console</a>. Here&#8217;s a second example of <a href="http://developer.yahoo.com/yql/console/?q=select%20symbol%2C%20ChangeRealtime%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22YHOO%22%2C%22AAPL%22%2C%22GOOG%22%2C%22MSFT%22)%20%7C%20sort(field%3D%22ChangeRealtime%22%2C%20descending%3D%22true%22)%0A%09%09&#038;env=http%3A%2F%2Fdatatables.org%2Falltables.env">pulling out only a few fields and sorting the quotes by who has the biggest gain</a>.</p>
<p>You&#8217;ll see the query:</p>
<pre>select * from yahoo.finance.quotes where symbol in ("YHOO","AAPL","GOOG","MSFT")</pre>
<p style="text-align: left;">And the results (trimmed for this post &#8211; there&#8217;s a lot of data in the results):</p>
<pre><span style="color: #0000ff;">&lt;?</span>xml version="1.0" encoding="UTF-8"<span style="color: #0000ff;">?&gt;</span>
<span style="color: #0000ff;">&lt;</span><span style="color: #800000;">query</span> <span style="color: #ff0000;">xmlns</span>:<span style="color: #ff0000;">yahoo</span>=<span style="color: #0000ff;">"http://www.yahooapis.com/v1/base.rng"</span> <span style="color: #ff0000;">yahoo</span>:<span style="color: #ff0000;">count</span>=<span style="color: #0000ff;">"4"</span> <span style="color: #ff0000;">yahoo</span>:<span style="color: #ff0000;">created</span>=<span style="color: #0000ff;">"2009-06-01T10:40:52Z"</span> <span style="color: #ff0000;">yahoo</span>:<span style="color: #ff0000;">lang</span>=<span style="color: #0000ff;">"en-US"</span> <span style="color: #ff0000;">yahoo</span>:<span style="color: #ff0000;">updated</span>=<span style="color: #0000ff;">"2009-06-01T10:40:52Z"</span> <span style="color: #ff0000;">yahoo</span>:<span style="color: #ff0000;">uri</span>=<span style="color: #0000ff;">"http://query.yahooapis.com/v1/yql?q=select+*+from+yahoo.finance.quotes+where+symbol+in+%28%22YHOO%22%2C%22AAPL%22%2C%22GOOG%22%2C%22MSFT%22%29"</span><span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">diagnostics</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">publiclyCallable</span><span style="color: #0000ff;">&gt;</span>true<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">publiclyCallable</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">url</span> <span style="color: #ff0000;">execution</span>-<span style="color: #ff0000;">time</span>=<span style="color: #0000ff;">"2"</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;</span>![CDATA[http://datatables.org/alltables.env]]<span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">url</span> <span style="color: #ff0000;">execution</span>-<span style="color: #ff0000;">time</span>=<span style="color: #0000ff;">"55"</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;</span>![CDATA[http://www.datatables.org/yahoo/finance/yahoo.finance.quotes.xml]]<span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">url</span> <span style="color: #ff0000;">execution</span>-<span style="color: #ff0000;">time</span>=<span style="color: #0000ff;">"5"</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;</span>![CDATA[http://download.finance.yahoo.com/d/quotes.csv?s=YHOO,AAPL,GOOG,MSFT&amp;f=aa2bb2b3b4cc1c3c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy]]<span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">url</span> <span style="color: #ff0000;">execution</span>-<span style="color: #ff0000;">time</span>=<span style="color: #0000ff;">"13"</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;</span>![CDATA[select * from csv where url=@url and columns='Ask,AverageDailyVolume,Bid,AskRealtime,BidRealtime,BookValue,Change&amp;PercentChange,Change,Commission,ChangeRealtime,AfterHoursChangeRealtime,DividendShare,LastTradeDate,TradeDate,EarningsShare,ErrorIndicationreturnedforsymbolchangedinvalid,EPSEstimateCurrentYear,EPSEstimateNextYear,EPSEstimateNextQuarter,DaysLow,DaysHigh,YearLow,YearHigh,HoldingsGainPercent,AnnualizedGain,HoldingsGain,HoldingsGainPercentRealtime,HoldingsGainRealtime,MoreInfo,OrderBookRealtime,MarketCapitalization,MarketCapRealtime,EBITDA,ChangeFromYearLow,PercentChangeFromYearLow,LastTradeRealtimeWithTime,ChangePercentRealtime,ChangeFromYearHigh,PercebtChangeFromYearHigh,LastTradeWithTime,LastTradePriceOnly,HighLimit,LowLimit,DaysRange,DaysRangeRealtime,FiftydayMovingAverage,TwoHundreddayMovingAverage,ChangeFromTwoHundreddayMovingAverage,PercentChangeFromTwoHundreddayMovingAverage,ChangeFromFiftydayMovingAverage,PercentChangeFromFiftydayMovingAverage,Name,Notes,Open,PreviousClose,PricePaid,ChangeinPercent,PriceSales,PriceBook,ExDividendDate,PERatio,DividendPayDate,PERatioRealtime,PEGRatio,PriceEPSEstimateCurrentYear,PriceEPSEstimateNextYear,Symbol,SharesOwned,ShortRatio,LastTradeTime,TickerTrend,OneyrTargetPrice,Volume,HoldingsValue,HoldingsValueRealtime,YearRange,DaysValueChange,DaysValueChangeRealtime,StockExchange,DividendYield']]<span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">javascript</span> <span style="color: #ff0000;">instructions</span>-<span style="color: #ff0000;">used</span>=<span style="color: #0000ff;">"279387"</span><span style="color: #0000ff;">/&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">user</span>-<span style="color: #ff0000;">time</span><span style="color: #0000ff;">&gt;</span>313<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">user</span>-time<span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">service</span>-<span style="color: #ff0000;">time</span><span style="color: #0000ff;">&gt;</span>75<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">service</span>-time<span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">build</span>-<span style="color: #ff0000;">version</span><span style="color: #0000ff;">&gt;</span>1678<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">build</span>-version<span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">diagnostics</span><span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">results</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">quote</span> <span style="color: #ff0000;">symbol</span>=<span style="color: #0000ff;">"YHOO"</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Ask</span><span style="color: #0000ff;">&gt;</span>16.60<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Ask</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">AverageDailyVolume</span><span style="color: #0000ff;">&gt;</span>22083900<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">AverageDailyVolume</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Bid</span><span style="color: #0000ff;">&gt;</span>16.55<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Bid</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">AskRealtime</span><span style="color: #0000ff;">&gt;</span>16.60<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">AskRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">BidRealtime</span><span style="color: #0000ff;">&gt;</span>16.55<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">BidRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">BookValue</span><span style="color: #0000ff;">&gt;</span>8.30<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">BookValue</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Change</span><span style="color: #ff0000;">_PercentChange</span><span style="color: #0000ff;">&gt;</span>+0.74 - +4.67%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Change</span>_PercentChange<span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Change</span><span style="color: #0000ff;">&gt;</span>+0.74<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Change</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Commission</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ChangeRealtime</span><span style="color: #0000ff;">&gt;</span>+0.74<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ChangeRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">AfterHoursChangeRealtime</span><span style="color: #0000ff;">&gt;</span>N/A - N/A<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">AfterHoursChangeRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DividendShare</span><span style="color: #0000ff;">&gt;</span>0.00<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DividendShare</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">LastTradeDate</span><span style="color: #0000ff;">&gt;</span>6/1/2009<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">LastTradeDate</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">TradeDate</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">EarningsShare</span><span style="color: #0000ff;">&gt;</span>0.011<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">EarningsShare</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ErrorIndicationreturnedforsymbolchangedinvalid</span><span style="color: #0000ff;">&gt;</span>N/A<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ErrorIndicationreturnedforsymbolchangedinvalid</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">EPSEstimateCurrentYear</span><span style="color: #0000ff;">&gt;</span>0.36<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">EPSEstimateCurrentYear</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">EPSEstimateNextYear</span><span style="color: #0000ff;">&gt;</span>0.42<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">EPSEstimateNextYear</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">EPSEstimateNextQuarter</span><span style="color: #0000ff;">&gt;</span>0.08<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">EPSEstimateNextQuarter</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DaysLow</span><span style="color: #0000ff;">&gt;</span>16.13<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DaysLow</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DaysHigh</span><span style="color: #0000ff;">&gt;</span>16.65<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DaysHigh</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">YearLow</span><span style="color: #0000ff;">&gt;</span>8.94<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">YearLow</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">YearHigh</span><span style="color: #0000ff;">&gt;</span>27.10<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">YearHigh</span><span style="color: #0000ff;">&gt;</span>
...
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">MarketCapitalization</span><span style="color: #0000ff;">&gt;</span>23.140B<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">MarketCapitalization</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">MarketCapRealtime</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">EBITDA</span><span style="color: #0000ff;">&gt;</span>1.278B<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">EBITDA</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ChangeFromYearLow</span><span style="color: #0000ff;">&gt;</span>+7.64<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ChangeFromYearLow</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">PercentChangeFromYearLow</span><span style="color: #0000ff;">&gt;</span>+85.46%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">PercentChangeFromYearLow</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">LastTradeRealtimeWithTime</span><span style="color: #0000ff;">&gt;</span>N/A - &amp;lt;b&amp;gt;16.58&amp;lt;/b&amp;gt;<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">LastTradeRealtimeWithTime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ChangePercentRealtime</span><span style="color: #0000ff;">&gt;</span>N/A - +4.67%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ChangePercentRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ChangeFromYearHigh</span><span style="color: #0000ff;">&gt;</span>-10.52<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ChangeFromYearHigh</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">PercebtChangeFromYearHigh</span><span style="color: #0000ff;">&gt;</span>-38.82%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">PercebtChangeFromYearHigh</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">LastTradeWithTime</span><span style="color: #0000ff;">&gt;</span>4:00pm - &amp;lt;b&amp;gt;16.58&amp;lt;/b&amp;gt;<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">LastTradeWithTime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">LastTradePriceOnly</span><span style="color: #0000ff;">&gt;</span>16.58<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">LastTradePriceOnly</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">HighLimit</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">LowLimit</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DaysRange</span><span style="color: #0000ff;">&gt;</span>16.13 - 16.65<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DaysRange</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DaysRangeRealtime</span><span style="color: #0000ff;">&gt;</span>N/A - N/A<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DaysRangeRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">FiftydayMovingAverage</span><span style="color: #0000ff;">&gt;</span>14.6126<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">FiftydayMovingAverage</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">TwoHundreddayMovingAverage</span><span style="color: #0000ff;">&gt;</span>12.9096<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">TwoHundreddayMovingAverage</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ChangeFromTwoHundreddayMovingAverage</span><span style="color: #0000ff;">&gt;</span>+3.6704<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ChangeFromTwoHundreddayMovingAverage</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">PercentChangeFromTwoHundreddayMovingAverage</span><span style="color: #0000ff;">&gt;</span>+28.43%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">PercentChangeFromTwoHundreddayMovingAverage</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ChangeFromFiftydayMovingAverage</span><span style="color: #0000ff;">&gt;</span>+1.9674<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ChangeFromFiftydayMovingAverage</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">PercentChangeFromFiftydayMovingAverage</span><span style="color: #0000ff;">&gt;</span>+13.46%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">PercentChangeFromFiftydayMovingAverage</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Name</span><span style="color: #0000ff;">&gt;</span>Yahoo! Inc.<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Name</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Notes</span><span style="color: #0000ff;">&gt;</span>-<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Notes</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Open</span><span style="color: #0000ff;">&gt;</span>16.18<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Open</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">PreviousClose</span><span style="color: #0000ff;">&gt;</span>15.84<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">PreviousClose</span><span style="color: #0000ff;">&gt;</span>
...
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Symbol</span><span style="color: #0000ff;">&gt;</span>YHOO<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Symbol</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">SharesOwned</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">ShortRatio</span><span style="color: #0000ff;">&gt;</span>2.10<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">ShortRatio</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">LastTradeTime</span><span style="color: #0000ff;">&gt;</span>4:00pm<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">LastTradeTime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">TickerTrend</span><span style="color: #0000ff;">&gt;</span>&amp;amp;nbsp;======&amp;amp;nbsp;<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">TickerTrend</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">OneyrTargetPrice</span><span style="color: #0000ff;">&gt;</span>15.27<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">OneyrTargetPrice</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">Volume</span><span style="color: #0000ff;">&gt;</span>27926064<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">Volume</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">HoldingsValue</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">HoldingsValueRealtime</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">YearRange</span><span style="color: #0000ff;">&gt;</span>8.94 - 27.10<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">YearRange</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DaysValueChange</span><span style="color: #0000ff;">&gt;</span>- - +4.67%<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DaysValueChange</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">DaysValueChangeRealtime</span><span style="color: #0000ff;">&gt;</span>N/A - N/A<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">DaysValueChangeRealtime</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">StockExchange</span><span style="color: #0000ff;">&gt;</span>NasdaqNM<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">StockExchange</span><span style="color: #0000ff;">&gt;</span>
...
    <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">quote</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">quote</span> <span style="color: #ff0000;">symbol</span>=<span style="color: #0000ff;">"AAPL"</span><span style="color: #0000ff;">&gt;</span>
...
  <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">results</span><span style="color: #0000ff;">&gt;</span>
<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">query</span><span style="color: #0000ff;">&gt;</span></pre>
<p>So how did we go from that ugly looking CSV file to the lovely XML? The answer is the <a href="http://www.datatables.org/yahoo/finance/yahoo.finance.quotes.xml">yahoo.finance.quotes open data table</a>:</p>
<pre><span style="color: #0000ff;">&lt;?</span>xml version="1.0" encoding="UTF-8" <span style="color: #0000ff;">?&gt;</span>
<span style="color: #0000ff;">&lt;</span><span style="color: #800000;">table</span> <span style="color: #ff0000;">xmlns</span>=<span style="color: #0000ff;">"http://query.yahooapis.com/v1/schema/table.xsd"</span><span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">meta</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">sampleQuery</span><span style="color: #0000ff;">&gt;</span>
      select * from {table} where symbol in ("YHOO","AAPL","GOOG","MSFT")
    <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">sampleQuery</span><span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">meta</span><span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">bindings</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">select</span> <span style="color: #ff0000;">itemPath</span>=<span style="color: #0000ff;">"quotes.quote"</span> <span style="color: #ff0000;">produces</span>=<span style="color: #0000ff;">"XML"</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">urls</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span>http://download.finance.yahoo.com/d/quotes.csv?s={-listjoin|,|symbol}<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">urls</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">inputs</span><span style="color: #0000ff;">&gt;</span>
        <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">key</span> <span style="color: #ff0000;">id</span>=<span style="color: #0000ff;">'f'</span> <span style="color: #ff0000;">type</span>=<span style="color: #0000ff;">'xs:string'</span> <span style="color: #ff0000;">const</span>=<span style="color: #0000ff;">'true'</span> <span style="color: #ff0000;">default</span>=<span style="color: #0000ff;">'aa2bb2b3b4cc1c3c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy'</span> <span style="color: #ff0000;">paramType</span>=<span style="color: #0000ff;">'query'</span> <span style="color: #0000ff;">/&gt;</span>
        <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">key</span> <span style="color: #ff0000;">id</span>=<span style="color: #0000ff;">'symbol'</span> <span style="color: #ff0000;">type</span>=<span style="color: #0000ff;">'xs:string'</span> <span style="color: #ff0000;">batchable</span>=<span style="color: #0000ff;">'true'</span> <span style="color: #ff0000;">maxBatchItems</span>=<span style="color: #0000ff;">'20'</span> <span style="color: #ff0000;">paramType</span>=<span style="color: #0000ff;">'path'</span> <span style="color: #ff0000;">required</span>=<span style="color: #0000ff;">'true'</span><span style="color: #0000ff;">/&gt;</span>
      <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">inputs</span><span style="color: #0000ff;">&gt;</span>
      <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">execute</span><span style="color: #0000ff;">&gt;</span><span style="color: #0000ff;">&lt;</span>![CDATA[
        var results = y.query("select * from csv where url=@url and columns='Ask,AverageDailyVolume,Bid,AskRealtime,BidRealtime,BookValue,Change&amp;PercentChange,Change,Commission,ChangeRealtime,AfterHoursChangeRealtime,DividendShare,LastTradeDate,TradeDate,EarningsShare,ErrorIndicationreturnedforsymbolchangedinvalid,EPSEstimateCurrentYear,EPSEstimateNextYear,EPSEstimateNextQuarter,DaysLow,DaysHigh,YearLow,YearHigh,HoldingsGainPercent,AnnualizedGain,HoldingsGain,HoldingsGainPercentRealtime,HoldingsGainRealtime,MoreInfo,OrderBookRealtime,MarketCapitalization,MarketCapRealtime,EBITDA,ChangeFromYearLow,PercentChangeFromYearLow,LastTradeRealtimeWithTime,ChangePercentRealtime,ChangeFromYearHigh,PercebtChangeFromYearHigh,LastTradeWithTime,LastTradePriceOnly,HighLimit,LowLimit,DaysRange,DaysRangeRealtime,FiftydayMovingAverage,TwoHundreddayMovingAverage,ChangeFromTwoHundreddayMovingAverage,PercentChangeFromTwoHundreddayMovingAverage,ChangeFromFiftydayMovingAverage,PercentChangeFromFiftydayMovingAverage,Name,Notes,Open,PreviousClose,PricePaid,ChangeinPercent,PriceSales,PriceBook,ExDividendDate,PERatio,DividendPayDate,PERatioRealtime,PEGRatio,PriceEPSEstimateCurrentYear,PriceEPSEstimateNextYear,Symbol,SharesOwned,ShortRatio,LastTradeTime,TickerTrend,OneyrTargetPrice,Volume,HoldingsValue,HoldingsValueRealtime,YearRange,DaysValueChange,DaysValueChangeRealtime,StockExchange,DividendYield'",{url:request.url});
        var quotes = <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">quotes</span><span style="color: #0000ff;">/&gt;</span>;
        var rows=results.results.row;
        for each (var row in rows) {
          for each (var item in row.*) {
            var elname = item.localName();
            var txt = item.text().toString();
            if (txt=="N/A") txt=""; else if (txt=="-") txt=""; else {
              txt = txt.replace(/"/g, '');
            }
            row[elname]=txt;
          }
            quotes.quote += <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">quote</span> <span style="color: #ff0000;">symbol</span>=<span style="color: #0000ff;">{row.Symbol.text().toString()}</span><span style="color: #0000ff;">&gt;</span>{row.*}<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">quote</span><span style="color: #0000ff;">&gt;</span>;
        }
        response.object = quotes;
           ]]&gt;<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">execute</span><span style="color: #0000ff;">&gt;</span>
    <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">select</span><span style="color: #0000ff;">&gt;</span>
  <span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">bindings</span><span style="color: #0000ff;">&gt;</span>
<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">table</span><span style="color: #0000ff;">&gt;
</span></pre>
<p>Let&#8217;s step through the main parts of the open data table definition. First the URL that YQL builds to get the data looks like this:</p>
<pre><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span>http://download.finance.yahoo.com/d/quotes.csv?s={-listjoin|,|symbol}<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">url</span><span style="color: #0000ff;">&gt;</span></pre>
<p>The <code>listjoin</code> is a URI template instruction that creates a &#8220;comma&#8221; separated list of values for each item in the &#8220;symbol&#8221; value. Symbol itself looks like this in the inputs section:</p>
<pre><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">key</span> <span style="color: #ff0000;">id</span>=<span style="color: #0000ff;">'f'</span> <span style="color: #ff0000;">type</span>=<span style="color: #0000ff;">'xs:string'</span> <span style="color: #ff0000;">const</span>=<span style="color: #0000ff;">'true'</span> <span style="color: #ff0000;">default</span>=<span style="color: #0000ff;">'aa2bb2b3b4cc1c3c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy'</span> <span style="color: #ff0000;">paramType</span>=<span style="color: #0000ff;">'query'</span> <span style="color: #0000ff;">/&gt;</span>
<span style="color: #0000ff;">&lt;</span><span style="color: #800000;">key</span> <span style="color: #ff0000;">id</span>=<span style="color: #0000ff;">'symbol'</span> <span style="color: #ff0000;">type</span>=<span style="color: #0000ff;">'xs:string'</span> <span style="color: #ff0000;">batchable</span>=<span style="color: #0000ff;">'true'</span> <span style="color: #ff0000;">maxBatchItems</span>=<span style="color: #0000ff;">'20'</span> <span style="color: #ff0000;">paramType</span>=<span style="color: #0000ff;">'path'</span> <span style="color: #ff0000;">required</span>=<span style="color: #0000ff;">'true'</span><span style="color: #0000ff;">/&gt;</span></pre>
<p>Note the <code>batchable</code> attribute of <code>symbol</code>. This tells YQL that this parameter can accept a set of values that can be sent to the remote data provider in a <strong>single</strong> request. In this case, the finance CSV API can take a comma separated list of stock symbols and return all that information for each entry.</p>
<p>That cryptic looking <code>f</code> input key is a constant &#8211; we&#8217;re going to get all the fields we can every time, and this value holds all the short field names that the API understands.</p>
<p>The YQL <code>execute</code> section actually dispatches the request and processes the return data.</p>
<pre>var results = y.query("select * from csv where url=@url and columns='Ask,AverageDailyVolume,Bid,AskRealtime,BidRealtime,BookValue,Change&amp;PercentChange,Change,Commission,ChangeRealtime,AfterHoursChangeRealtime,DividendShare,LastTradeDate,TradeDate,EarningsShare,ErrorIndicationreturnedforsymbolchangedinvalid,EPSEstimateCurrentYear,EPSEstimateNextYear,EPSEstimateNextQuarter,DaysLow,DaysHigh,YearLow,YearHigh,HoldingsGainPercent,AnnualizedGain,HoldingsGain,HoldingsGainPercentRealtime,HoldingsGainRealtime,MoreInfo,OrderBookRealtime,MarketCapitalization,MarketCapRealtime,EBITDA,ChangeFromYearLow,PercentChangeFromYearLow,LastTradeRealtimeWithTime,ChangePercentRealtime,ChangeFromYearHigh,PercebtChangeFromYearHigh,LastTradeWithTime,LastTradePriceOnly,HighLimit,LowLimit,DaysRange,DaysRangeRealtime,FiftydayMovingAverage,TwoHundreddayMovingAverage,ChangeFromTwoHundreddayMovingAverage,PercentChangeFromTwoHundreddayMovingAverage,ChangeFromFiftydayMovingAverage,PercentChangeFromFiftydayMovingAverage,Name,Notes,Open,PreviousClose,PricePaid,ChangeinPercent,PriceSales,PriceBook,ExDividendDate,PERatio,DividendPayDate,PERatioRealtime,PEGRatio,PriceEPSEstimateCurrentYear,PriceEPSEstimateNextYear,Symbol,SharesOwned,ShortRatio,LastTradeTime,TickerTrend,OneyrTargetPrice,Volume,HoldingsValue,HoldingsValueRealtime,YearRange,DaysValueChange,DaysValueChangeRealtime,StockExchange,DividendYield'",{url:request.url});</pre>
<p>This runs another YQL select statement when the table gets invoked. It&#8217;s fetching a CSV data source from a URL and setting up the column names for each &#8220;row&#8221; that comes back. The URL that YQL would originally have fetched for this data is already created in the <code>request</code> object, containing the list of symbols expanded into the <code>s</code> parameter, so we just used the <code>@</code> substitution syntax to add that into the YQL statement.</p>
<pre>var quotes = <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">quotes</span><span style="color: #0000ff;">/&gt;</span>;
var rows=results.results.row;</pre>
<p>The next few lines create a new XML object called <code>quotes</code> which will hold our final XML document and gets an XML list to each of the rows that came back from the CSV query.</p>
<pre>for each (var row in rows) {</pre>
<p>Now we&#8217;ll loop over each of those rows (one row per stock symbol).</p>
<pre>   for each (var item in row.*) {
    var elname = item.localName();
    var txt = item.text().toString();
    if (txt=="N/A") txt=""; else if (txt=="-") txt=""; else {
      txt = txt.replace(/"/g, '');
    }
    row[elname]=txt;
  }</pre>
<p>For each element in that row (note the E4X syntax to get all the elements <code>row.*</code>) we&#8217;re going to clean up the XML somewhat. We&#8217;ll get rid of &#8220;N/A&#8221; and &#8220;-&#8221; text elements and use empty elements instead, as well as remove any quotes in the text.</p>
<pre>   quotes.quote += <span style="color: #0000ff;">&lt;</span><span style="color: #800000;">quote</span> <span style="color: #ff0000;">symbol</span>=<span style="color: #0000ff;">{row.Symbol.text().toString()}</span><span style="color: #0000ff;">&gt;</span>{row.*}<span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">quote</span><span style="color: #0000ff;">&gt;</span>;
}</pre>
<p>Finally in the main loop we&#8217;ll append a new <code>quote</code> element to our root XML <code>quotes</code> element that contains the reformatted XML elements, and an attribute called <code>symbol</code></p>
<pre>response.object = quotes;</pre>
<p>Last of all we set the response object to the document we&#8217;ve just created in the loop.</p>
<p>We&#8217;ve already added the table to the github open data table repository, so you can try this table out  <a href="http://developer.yahoo.com/yql/console/?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22YHOO%22%2C%22AAPL%22%2C%22GOOG%22%2C%22MSFT%22)%0A%09%09&amp;env=http%3A%2F%2Fdatatables.org%2Falltables.env">in the YQL console</a> just by including the community tables. And now its in YQL, you can sort, filter, project and join on any of this data that comes back! For example, you can <a href="http://developer.yahoo.com/yql/console/?q=select%20symbol%2C%20ChangeRealtime%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22YHOO%22%2C%22AAPL%22%2C%22GOOG%22%2C%22MSFT%22)%20%7C%20sort(field%3D%22ChangeRealtime%22%2C%20descending%3D%22true%22)%0A%09%09&#038;env=http%3A%2F%2Fdatatables.org%2Falltables.env">pull out only a few fields and sort the quotes by who has the biggest gain</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://yqlblog.net/blog/2009/06/02/getting-stock-information-with-yql-and-open-data-tables/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Do it your way with execute and open data tables</title>
		<link>http://yqlblog.net/blog/2009/04/29/do-it-your-way-with-execute-and-open-data-tables/</link>
		<comments>http://yqlblog.net/blog/2009/04/29/do-it-your-way-with-execute-and-open-data-tables/#comments</comments>
		<pubDate>Wed, 29 Apr 2009 20:52:45 +0000</pubDate>
		<dc:creator>yqlteam</dc:creator>
				<category><![CDATA[feature]]></category>

		<guid isPermaLink="false">http://yqlblog.net/blog/?p=53</guid>
		<description><![CDATA[Today we&#8217;ve released a significant new feature for YQL: &#60;execute&#62;.
Previously our open data tables enabled any developer to bring the power of YQL and the table model available to new APIs or data sources that we hadn&#8217;t covered. Many of these are hosted on github, and you can read (and use) about them on datatables.org. [...]]]></description>
			<content:encoded><![CDATA[<p>Today we&#8217;ve released a significant new feature for YQL: <strong>&lt;execute&gt;</strong>.</p>
<p>Previously our open data tables enabled any developer to bring the power of YQL and the table model available to new APIs or data sources that we hadn&#8217;t covered. Many of these are hosted on <a href="http://github.com/spullara/yql-tables/tree/master">github</a>, and you can read (and use) about them on <a href="http://datatables.org">datatables.org</a>. These give you access to friendfeed, google reader, guardian, lastfm, nyt, twitter, weather.com, whitepages, yelp, zillow to name just a few.<br />
<strong></strong></p>
<p><strong>YQL &lt;execute&gt;</strong> extends the open data table capabilities through server-side transformation and processing using javascript. This dramatically increases the reach and power of open data tables, for example:</p>
<ul>
<li>join data across services<br />
e.g. grab New York Times article tags and incorporate associated flickr photos</li>
<li>combine multiple searches into a single result<br />
twitter, web, news and image&#8230;</li>
<li>augment data<br />
e.g. Add city/state information to APIs that just return zip code</li>
<li>create an API from a web page<br />
e.g. celebrity birthdays scraped from imdb</li>
<li>access APIs that require authentication<br />
e.g. Netflix OAuth, FlickrAuth, Google AuthSub &#8230;</li>
<li>data transformation<br />
e.g. convert the result from xml to Google&#8217;s kml format</li>
<li>move business logic for your application&#8217;s data to the cloud</li>
<li>leverage the work of others to make access easier by sharing tables</li>
</ul>
<p>Read more about this on our <a href="http://developer.yahoo.net/blog/archives/2009/04/yql_execute.html">YDN blog post</a>, including links to several examples to wet your appetite.</p>
]]></content:encoded>
			<wfw:commentRss>http://yqlblog.net/blog/2009/04/29/do-it-your-way-with-execute-and-open-data-tables/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>YQL environment files</title>
		<link>http://yqlblog.net/blog/2009/02/20/yql-environment-files/</link>
		<comments>http://yqlblog.net/blog/2009/02/20/yql-environment-files/#comments</comments>
		<pubDate>Fri, 20 Feb 2009 19:32:56 +0000</pubDate>
		<dc:creator>yqlteam</dc:creator>
				<category><![CDATA[feature]]></category>
		<category><![CDATA[news]]></category>

		<guid isPermaLink="false">http://yqlblog.net.p2.hostingprod.com/blog/?p=38</guid>
		<description><![CDATA[We&#8217;ve introduced a new env query parameter to both the YQL console and the web service. The env parameter expects a URL value that points to a plain text file anywhere on the web (including subversion repositories, github etc). The file should contain a list of use statements that specify all the open data tables [...]]]></description>
			<content:encoded><![CDATA[<p>We&#8217;ve introduced a new <code>env</code> query parameter to both the YQL console and the web service. The <code>env</code> parameter expects a URL value that points to a plain text file anywhere on the web (including subversion repositories, github etc). The file should contain a list of <code>use</code> statements that specify all the open data tables that you&#8217;d like to be available to your YQL queries for that request. For example, if I want all of the New York Times open data tables hosted at javarants I would create a file like this:</p>
<pre><code>use 'http://www.javarants.com/nyt/nyt.article.search.xml';
use 'http://www.javarants.com/nyt/nyt.bestsellers.history.xml';
use 'http://www.javarants.com/nyt/nyt.bestsellers.search.xml';
use 'http://www.javarants.com/nyt/nyt.bestsellers.xml';
use 'http://www.javarants.com/nyt/nyt.movies.critics.xml';
use 'http://www.javarants.com/nyt/nyt.movies.picks.xml';
use 'http://www.javarants.com/nyt/nyt.movies.reviews.xml';

</code></pre>
<p>&#8230;and serve it from somewhere on the web. If you want to try this new feature on some useful open data tables in the console, you can use a few of the ones currently in the <a href="http://github.com/spullara/yql-tables/tree/master">github open data tables</a> project like this:</p>
<p><code>http://developer.yahoo.com/yql/console/?env=http://github.com/spullara/yql-tables/raw/ef685688d649a7514ebd27722366b2918d966573/alltables.env<br />
</code></p>
<p><b><a href="http://developer.yahoo.com/yql/console/?env=http://github.com/spullara/yql-tables/raw/ef685688d649a7514ebd27722366b2918d966573/alltables.env">Try it in the console!</a></b>. If you look to the &#8220;table&#8221; list on the right of the console, you&#8217;ll see an extra <b>9 APIs and 30 tables to explore</b>!</p>
]]></content:encoded>
			<wfw:commentRss>http://yqlblog.net/blog/2009/02/20/yql-environment-files/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
