Category Archives: news

y.ahoo.it URL Shortener end-of-life announcement

Filed under news

The y.ahoo.it URL shortener powered by YQL will be shutdown on November 20th 2013 – a URL shortening service is not one of Yahoo!’s core competencies supporting our users’ Daily Habits.

As of November 20th, you will not be able to create short URLs using either the form at http://y.ahoo.it or the yahoo.y.ahoo.it YQL table available on the YQL console. The page at http://y.ahoo.it will redirect to http://www.yahoo.com.

Any short links you have already created will continue to redirect to the correct locations. We will provide at least three months notice before making any further changes to the service.

Unified YQL Console

Filed under news

Introducing the new Unified YQL Console.

Unified YQL Console

This new release combines YQL developer tools under a shared header. The YQL Console and YQL Editor are now updated with a new look and feel, easier navigation to these complementing products and some new features.

YQL Console

The YQL Console has been refreshed with a new look and feel that will unify the looks of the Console and the YQL Table Editor.

We’ve made a slight adjustment to the layout in order to make it easier for developers to get working right away. The list of tables, which originally sat in the bottom-right corner, was moved over to occupy the entire left-hand side, since this is where most developers start with the console. In addition, this change gives developers a lot more space to browse and try out the different tables.

With the new console, we’ll remember how you customized the console so you can start working even faster the next time you’re back. For example, if you prefer to work with JSON over XML and have both sidebars closed, you will get this layout every time you load the
console.

Last, but not least, there have been several improvements to performance. The tables index and search data is now cached server-side, which means you can work and search
tables faster.

YQL Table Editor

The YQL Editor was rebuilt using the YUI3 Apps Framework and now allows you to
dynamically save, load, and delete your data (versus a full page reload).

We’ve also upgraded our CodeMirror library to a newer version and added two
plugins to help developers write their YQL tables.

gif_600x264_855280

These plugins will close XML tags automatically and suggest tags and attributes based on the official YQL table schema. Like the Console, the editor features some basic state persistence including a backup of your data in the event your browser crashes.

Publishing to GitHub

Filed under feature, news

A lot of hassle is involved in submitting your own table to YQL.

First, you would have to open a GitHub account. Then you would visit the YQL repository on GitHub and create a fork. Once you’ve done that, you’d have to pull the fork onto your local machine. After the fork has been pulled, you would have to sift through directories to get a feel for the table naming convention before adding your table in. Finally you’d have to make a pull request to YQL and wait until your table has been approved.

Today, we’re releasing a new feature in the Table Editor that allows you to submit tables directly to GitHub without even touching the command-line. All you have to do is press a simple button to start:

Screen Shot 2012-08-13 at 11.42.13 AM

Clicking the “Publish to GitHub” button will show you a brief disclaimer that will explain how your table will be submitted:

Screen Shot 2012-08-13 at 12.25.21 PM

Once you’ve signed in and authenticated the Table Editor with GitHub, we’ll go ahead and publish your table. In the future, you won’t have to authenticate with GitHub in order to publish a table. And if you ever wish to disassociate your GitHub account from the Table Editor, you can visit your GitHub application settings to revoke access.

Currently we will only push tables to GitHub. Environment and JS files will not be pushed at this time.

In addition to this new feature, we’ve ported all client-side code over to using the YUI 3 App Framework for better flexibility in the future.

Please try out this new feature and send us any questions or suggestions you might have.

Run Yahoo! Pipes from YQL Execute

Filed under feature, news

We recently added a new YQL execute method: y.pipe(pipeid,params)

This method will allow you to run a Yahoo Pipe within a YQL execute statement.

Accepted Parameters:
pipeid (required string)
params (optional json object)

Returns:
A response object that contains a result instance or an error object

Why would you want to do this?

1. Allows you to utilize all the benefits that a YQL execute statement gives you.
2. Higher rate limits! Benefit from YQL rate limits instead of Pipes (which is much lower).
3. Mash ups with other YQL tables
4. Extend your Pipe with server side javascripting

Example:

This is the Pipe that I want to run in YQL: http://pipes.yahoo.com/pipes/pipe.info?_id=990bf4c00040ad06ba83de9aadd6293b

Here is a simple sample YQL table that uses y.pipe. You can create one too using the YQL Table Editor.

And this is the YQL Query I would run to access my example: use “store://HuTaxj5021R7THBitCNIcJ” as ypipe_example; select * from ypipe_example

(You could always extend the sample table to accept the pipeid or pipe params as a parameter via the input keys)

Currently YQL does not produce the output formats that Pipes does (RSS, KML, ICAL, CSV).

YQL will only currently produce XML, JSON, JSONP and JSONP-X output formats.

Job posting: Sr. Software Engineer – YQL

Filed under news

The Yahoo! Query Language is an expressive SQL-like language that lets you query, filter, and join data across Web services. With YQL, apps run faster with fewer lines of code and a smaller network footprint. With YQL, developers can access and shape data across the Internet through one simple language, eliminating the need to learn how to call different APIs.

The YQL team is looking for a talented and enthusiastic software engineer who will work on extending the YQL platform. We’re looking for someone who can both solve day to day problems, work with other teams in Yahoo to use YQL, as well as bring their own ideas for further evolution of the product.

Please submit your resume at: yql-questions [at] yahoo-inc.com

Job Responsibilities:

* Develop and maintain the YQL engine and platform, evolve architecture

* Strong ability to translate functional requirements into a technical design

* Strong analytical and problem-solving skills

* Excellent communication skills

* Highly motivated to be part of an innovative and talented team

Minimum Job Qualifications:

* 5+ years of software development experience, J2SE and J2EE hands-on development

* Ability to design, develop, test and deploy multi-tier and multi-threaded distributed applications

* Significant development experience in a *nix environment, debug and diagnose production issues

* Familiarity with a wide variety of software development methodologies, tools, languages and approaches

* Working knowledge of Web Services, Design Patterns, XML, RSS and other web data formats

* Extensive knowledge of core Java libraries, design patterns, JSP and servlets

Preferable Job Qualifications:

* BS/MS in Comp Science 5+ years of relevant industry experience

* Ability to build scalable/fault tolerant systems

* Knowledge of programming language design

* Experience creating Developer tools

* Familiarity with wide variety of web APIs

* Knowledge of server-side Javascript runtime environments

Please submit your resume at: yql-questions [at] yahoo-inc.com

YQL Rate Limit Increase

Filed under feature, news

Effective immediately the YQL Rate Limits are now increased:

Public endpoint to 2k/ip/hr (previously 1k)
Oauth endpoint to 20k/ip/hr (previously 10k)

Per application limit (identified by your Access Key): 100,000 calls per day will remain the same.

With this increase you can get up to 3 million – 3.1 million requests per month. (100k cap per day)

Thank you to the YQL community – we hope you enjoy this increase.

Recent Enhancement to the HTML table

Filed under feature, news

The HTML table has recently been enhanced to support HTML version 5. The HTML table in the backend uses a parser which autocorrects malformed tags. To support HTML5 we are using a different parser then the one used previously. Because of this change, the output might be slightly different than before. To ensure backward compatibility, both the parsers are supported, with the older one being the default. The new parser can be used by just appending compat=”html5” to the query.

For Example: select * from html where url=”http://finance.yahoo.com/q?s=yhoo” and compat=”html5″

Please start using this feature and give us your feedback! Eventually the new parser which supports ‘html5’ will be made default, but that will follow an announcement. Even after the new parser is made default, the old one can still be used by having compat equals to ‘html4’.

YQL Table Health and YQL Lint

Filed under feature, news

YQL has attracted a large number of OpenData tables thanks to the efforts of the community. But some of these tables don’t end up working properly due to many factors, like recent changes made to the underlying API. Therefore we’ve created two new tools, YQL Table Health and YQL Lint, to help developers see and understand which tables actually work and which ones don’t.

YQL Table Health is intended to provide a quick general overview of how “healthy” the community OpenData tables are:

img1

When you first arrive to the page, you will see a list of all the tables that can be used by YQL. Clicking on one of the entries in the list will cause it to expand and show additional information regarding where the source of the XML file is, what kind of table it is, sample query information, and lastly any errors that were encountered. You can use the controls on the left-hand side to further filter, sort, and search through all this data. If you see a table that doesn’t work, you can contact the author of the table to fix it via github. You may also fork the yql-tables from github and fix or enhance the table yourself.

YQL Table Health uses a sever-side script to iterate through all the tables. Each table’s XML file is loaded into YQL cloud storage before a series of checks are run against that XML file. The test results are then cached in a database as well as memory to serve this data as fast as possible. Updates to caches are triggered by users visiting the page and only fire if the data is older than thirty minutes; requests for an update are also synchronized to prevent a race condition occurring, where two or more requests might be made simultaneously. Last but not least, the data is served to the user through a user-interface built using HTML5 and the YUI Library.

The next tool we’re going to introduce to you is YQL Lint.

img2

YQL Lint is essentially an XML debugger for individual YQL tables. You can enter either a URL to a XML file, or the contents of an XML file, and it will validate this against our schema for syntax flaws. Once the schema check has been passed, we will use YQL to get a description of your YQL table and check to see if it contains a sample query that returns a valid result. YQL Lint essentially relies on the same core backend as YQL Table Health.

Please experiment with these tools and send us any questions or suggestions you might have.

Daniel Park – YQL/Pipes Intern

Search tables and BOSS v1

Filed under news

We’ve removed all search tables that relied on the BOSS v1 API (search.web, search.image, and search.news) as the aforementioned BOSS v1 no longer exists as of today (http://www.ysearchblog.com/2011/06/30/you-asked-for-this-boss-v2-updates/).

For those of you relying on those tables please consider using the community BOSS v2 table (https://github.com/yql/yql-tables/blob/master/boss/boss.search.xml).

Thanks -YQL Team

1000th Community Table!

Filed under news

We started datatables.org with our first github entry back in 2009-02-05. Since then we’ve had quality contributions made by the public. We amazingly hit our 1000th table and the award goes to Carson McDonald (@casron) for submitting soundcloud.playlists.xml!! This means on average we get just over 1 new table a day. Thousands of developers benefit from using these tables and we’re impressed and thankful for all your contributions. Keep them coming!!