Lynx44’s Blog


Hulu “Plug-in” for MythTV

Update 1 (5/12/2008):

Note that it is now necessary to always use the “-a” option when using the XWFDumper utility, since the RSS feeds no longer point to the embedded video.

Contents

Backstory

The Problem

For a few months now I’ve been looking for a way to get Hulu into MythTV. Ideally, I wanted it to match the Myth interface. I had a few different ideas on how to do this.

First I thought I could just point to Firefox and map keyboard shortcuts to navigate the website. I didn’t find a good extension or program that would let me do this easily, although I’m sure it exists somewhere. Also, I figured it would be pretty clunky to load up a browser over the top of Myth – not exactly the solution I was looking for.

The next idea almost worked but fell a little short and had too many hacks. There is a program called PlayOn that runs on Windows to provide Hulu videos via UPNP (advertised to work with the PS3 and Xbox 360). I ran it inside of VirtualBox on my Myth backend server.

Unfortunately, Myth’s frontend doesn’t connect to a UPNP source, so I realized that I needed something file based. An application called DJmount exists, that mounts a UPNP server as if it were a file system. While this can work, it takes a very long time for Myth to scan the directory, as DJmount is pretty slow at discovering the videos (I think it’s done dynamically on request). Additionally, running the PlayOn software inside a VM on an AMD 3200+ wasn’t exactly speedy or reliable, as it transcodes the videos from Flash to MPEG on the fly.

The Solution

Finally, I went back to basics. I found on the Hulu website that they provide an RSS feed for each of their shows. Luckily, each RSS feed was easy to guess. It follows the format of http://www.hulu.com/feed/show/show id/episodes. The show IDs seem to be pretty random, but range from 0 to approximately 2000. For example, King of the Hill‘s RSS feed is at http://www.hulu.com/feed/show/55/episodes, so it’s ID is 55.

So I had a way to grab information about each episode, but still the problem was that the URL inside of the RSS feed wasn’t playable by any common media player, as it was an SWF object. The FLVs don’t appear to be available, which makes some sense because you could probably easily skip the commercials if that were the case. Luckily, Adobe’s standalone flash player can easily play an SWF from any website. The unfortunate part is that the application only takes one argument, which is the URL of the SWF. That turned out to be the next challenge.

There are no open source SWF player applications that can play Hulu’s videos (that I could find – if I’m wrong, please let me know because it would much improve the experience of this hack). Therefore, I had to find a way to play the video, then full screen it. I found an application called xdotool that can mimic keyboard and mouse gestures. When a Hulu video is launched in the Flash player, xdotool clicks the play button and sets the player to  full screen mode.

Additionally, I’ve set up LIRC so that it can pause and exit the player by faking a space key press and by killing the Flash player process (respectively).

Its not elegant, but it works (although it can be a bit buggy). Using MythVideo, I can now scan the list of TV shows on Hulu alphabetically and play the video – all without using a mouse or keyboard.

This solution has been fully tested on Fedora 8, and partially tested on Ubuntu 8.04.

How-To

First, you will need these applications installed on your system:

Required:

Adobe’s Standalone Flash player (if link is broken, check here, versions 9 and 10 were tested)

XDOTool

Mono (or download it from your package manager)

XWFTools, which includes:

XWFDumper

XWFReader

flashplayer.sh

Optional:

kill.sh

togglepause.sh

Quick Start

Step 1:

First, head to Adobe’s Website and download the standalone flash player player. You should be able to get it here

Step 2:

Download, compile and install xdotool.

Step 3:

Install Mono. In most distributions, you can install it through your package manager.

Step 4:

Download and extract XWFTools.

Step 5:

Run XWFDumper by using the following command:

mono XWFReader -s “hulu” -d mythvideosdirectory

For example, on my system I would use:

mono XWFReader -s “hulu” -d “/storage/media/videos/Hulu”

As /storage/media/videos is where MythVideo looks for my video collection (NOT recordings).

Step 6:

Extract XWFReader

Step 7(optional):

Test the kill.sh script

Edit the script and point the script to the path of your xdotool. For example:

path_to_xdotool=’/home/mythtv/scripts/xdotool/xdotool’

Step 8(optional):

Test the togglepause.sh script

Edit the script and point the script to the path of your xdotool. For example:

path_to_xdotool=’/home/mythtv/scripts/xdotool/xdotool’

Step 9(optional):

Add the scripts to execute through LIRC:

For example, I have in my .lircrc file:

begin
prog = irexec
button = Back
config = /home/mythtv/scripts/flashplayer/kill.sh
end
begin
prog = irexec
button = Play
config = /home/mythtv/scripts/flashplayer/togglepause.sh
end

Additionally, there is a sample .lircrc file in the XWFTools “flashplayer” folder

Step 10:

Set up the flashplayer.sh script. Update the beginning of the script to point to the correct paths to the standalone flashplayer executable (from step 1), xdotool (from step 2) and XWFReader (from step 6).

For example, I have:

flashplayer_path=’/home/mythtv/scripts/flash_10/flashplayer’
path_to_xdotool=’/home/mythtv/scripts/xdotool/xdotool’
xwfreader_path=’/home/mythtv/Applications/XWFReader/XWFReader.exe’

Test to make sure that the flashplayer.sh script works without errors on your system:

./flashplayer.sh [path to XWF file]

For example, if I wanted to launch the King of the Hill episode A Bill Full of Dollars:

./flashplayer.sh /storage/videos/Hulu/K/King of the Hill/King of the Hill – 06×13 – A Bill Full of Dollars.xwf

Make sure the player opens and eventually goes fullscreen (it may take somewhere around 30 seconds depending on the settings found at the beginning of the script). You probably want to tweak it to your likeness at this point.

Step 11:

Open up mythfrontend. I’m going to assume that you are using the default menu layout.

Navigate to Utilities->Video Manager to scan for videos.

Navigate to Utilities->Setup->Media Settings->Videos Settings->File Types then click the New button. When it prompts you, add the extension “xwf” (no “.” in front).

newvideofiletype

For the command field, point it to the location of the flashplayer.sh script on your system, and use %s for the argument. For example, mine is at /home/mythtv/scripts/flashplayer/flashplayer.sh, so I have:

/home/mythtv/scripts/flashplayer/flashplayer.sh %s

After that is done navigate back to the main menu and open your list of videos (under Media Library->Watch Videos) and verify that the files are there. Attempt to launch one.

Everything should work. If not, post in the comments section, read the In Depth Explanation section, or retrace your steps through this guide to make sure you got everything correct. Also make sure that the flashplayer.sh script works from the command line and note any errors that it may give (XDOTool errors are typically OK).

In Depth Explanation

XWFDumper

The first tool I created dumps a series of XML files that define the TV shows available on Hulu. They are dumped in a directory structure similar to how I commonly organize my video collection. I called it “XWFDumper” because I semi-randomly chose the extension “.xwf” (XML and SWF), and potentially it could be used for other streaming video websites in the future. Its written in C#, and requires mono to run (sorry, I know most Linux users aren’t a fan, but I’m a .NET developer so it was the easiest and fastest way for me to get this working).

Examples:

mono XWFReader -s “hulu” -d “/storage/videos/Hulu”

Will result in a directory tree /storage/videos/Hulu/first letter of show/Name of show/Name of episode.xwf. For example:

/storage/videos/Hulu/K/King of the Hill/King of the Hill – 06×13 – A Bill Full of Dollars.xwf

Additional arguments can be specified, for example, if you (like me) like your shows to be separated into season folders. The directory structure would then look like:

/storage/videos/Hulu/first letter of show/Name of show/Season Number/Name of episode.xwf

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -f

Would then look like:

/storage/videos/Hulu/K/King of the Hill/Season 06/King of the Hill – 06×13 – A Bill Full of Dollars.xwf

Also, you can specify to only import specific shows by ID or by their name:

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -n “King of the Hill,The Simpsons”

or

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -i “55,58”

The ID comes from the RSS feed ID number, for example King of the Hill‘s is http://www.hulu.com/feed/show/55/episodes, therefore it’s ID is 55. This is the most efficient way to do it. Doing it by name is slower because each feed still needs to be downloaded in order to figure out what the name of the show is.

Additionally, you can add the -a (all) flag if you notice a particular show is missing some episodes:

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -i “55,58” -a

Reason being is that by default, the application only grabs episodes from the Hulu RSS feed. The feed doesn’t always have every episode listed, so the -a flag tells the application to do some screen scraping in order to find every episode. This flag can only be used in conjunction with the -i flag, to limit the amount of bandwidth used on Hulu’s server. I don’t want everyone scraping each webpage on their website.

Also note that it is likely that it will not work if you include a space after each comma for each show name:

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -n “King of the Hill, The Simpsons”

– may not work.

Additionally, you can specify to exclude specific shows by ID or by their name:

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -x “King of the Hill,The Simpsons”

or

mono XWFDumper -s “hulu” -d /storage/videos/Hulu -e “55,58”

In these cases, all shows will be downloaded except for the shows specified.

XWFReader

While this program could potentially have many more uses, it is currently very simple. Its a one arg command line tool that spits out the URL for that specific episode.

For example:

mono XWFReader “/storage/videos/Hulu/K/King of the Hill/King of the Hill – 06×13 – A Bill Full of Dollars.xwf”

would simply output:

That is the location for the SWF that needs to be played.

If you for whatever reason do not want to use this tool, I’m sure there are applications that can read XML files and dump the value of a specific node. That’s all this app does, it dumps the “url” attribute of the “media:player” tag.

flashplayer.sh

This is a script that I wrote in order to open the Flash player and launch it into full screen mode. Before you use it, you’ll need to set the path to Adobe’s Standalone Flashplayer application (flashplayer_path), XDOTool (path_to_xdotool) and XWFReader (xwfreader_path).

For example, mine looks like:

flashplayer_path=’/home/mythtv/scripts/flash_10/flashplayer’path_to_xdotool=’/home/mythtv/scripts/xdotool/xdotool’xwfreader_path=’/home/mythtv/Applications/XWFReader/XWFReader.exe’

Additionally, you can tweak the behavior by modifying the values in the OPTIONAL section. I tried to make it as flexible as possible, so hopefully you won’t have to change much to make it usable on your system.

It only takes two parameters. One is a URL to an SWF object, the other is a path to an XWF object. I think you get the general idea, but here are two examples to explain the point further:

or

./flashplayer.sh “/storage/videos/Hulu/K/King of the Hill/King of the Hill – 06×13 – A Bill Full of Dollars.xwf”

kill.sh

Additionally, I created a simple script that will kill the flashplayer process when I want to exit. This is pretty crucial, since the player does not close automatically when the video ends. I mapped it in LIRC using irexec to my Back button. Unfortunately its pretty buggy. I attempted to make it so that it would kill all child processes from that script, but it doesn’t seem to work as well as I thought.  If you have a better fix, please send me the changes and I’ll gladly include them.

First, point the script to the path of your xdotool:

path_to_xdotool=’/home/mythtv/scripts/xdotool/xdotool’

Then update your .lircrc file. For example:

begin
prog = irexec
button = Back
config = /home/mythtv/scripts/flashplayer/kill.sh
end

togglepause.sh

This is very similar to the kill.sh script, except that it toggles pausing the player. It basically just uses xdotool to simulate a space press. Again, I used irexec to map it to my remote’s play button. Alternatively, you could just map the play button on your remote to simulate the space key through irxevent directly, I just wanted to keep my mappings the same.

First, point the script to the path of your xdotool:

path_to_xdotool=’/home/mythtv/scripts/xdotool/xdotool’

Then update your .lircrc file. For example:

begin
prog = irexec
button = Play
config = /home/mythtv/scripts/flashplayer/togglepause.sh
end

Disadvantages

The obvious main disadvantage is that the Flash player can only go into full screen mode through mouse commands. If Adobe would provide a few more command line arguments, or if another Flash player alternative existed that could play these videos, the solution could be much simpler. Timing issues can easily occur because of this, so the functionality can be funky sometimes.

Additionally, the content won’t always be up to date. Without specific patches I don’t believe there is  a way in Myth to scan the video section automatically. This has to be done by the user whenever they want to check for new content.

Future

The best update would be a Flash player that would have many arguments, and even an LIRC interface. I’m not sure if that will ever happen, but it would be nice.

Otherwise, it would be nice to add more sources besides Hulu TV Episodes. The XWFDumper application is already set up to easily add new sources, I just haven’t looked into doing that yet. Soon enough I’m sure I’ll update it so that it also dumps Hulu’s movie section. Other networks have chosen to keep their content off of Hulu for whatever reason, such as CBS and ABC, so those would be nice to add. Additionally, it may even be possible to add things like Netflix – if the video is streamed through Flash (I don’t know because I don’t have an account).

Also, I would like to be able to add high quality versions of these videos. At the moment, they are the low quality (360p, I believe), and there is no way that I know of to view the 480p version without visiting Hulu’s website directly.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: