Writing a web client for MP3 streaming in perl.

What is streaming?

By streaming we refer to the decoding of a file consurrently with its download, i.e. rendering the contents of the file as they become available, without having to stand by for the entire file to be downloaded. Streaming is very useful in applications that involve live content (such as readio transmissions, or newscasts) or media files that tend to be very large, that is in applications where the user cannot afford waiting for the download to be completed before the file can be decoded. Not all file formats can be streamed. Many common media fromats/compressions that we use everyday, such as GIF or WAV require the entire file to be available before the decoding can take place. Formats such as mpeg-3 or RealAudio on the other hand, can be decoded as the data comes in. Such formats are called collectively streaming formats.

Taking advantage of the streaming properties of a file.

If you start downloading an mp3 file and stop the download before it's finshed, you'll then be able to hear the song that you have downloaded perfectly ok, up to the point where the transfer was interrupted. Going a step further, if you start the download and also ask your mp3 player to play the file which is being written to right now, you'll notice that the decoder plays the file just fine, and if your network connection can maintain a sufficient throughput, you could listen to the entire song as it's being downloaded without a single jitter. Because network connections rarely are too stable, you will find that it is good practice to let the download do some progress first and then start the decoder, so that even if your network throughput loses pace for a while, there will be some data read ahead for the decoder to munch as the network catches up.. otherwise you might experience jerks and pauses. This is a common technique, similar to the lookahead buffer that car stero cd's use to ensure sound continuity even when you hit a bump on the road.

Automating the process...

While it was fun to experiment with streaming manually, it would be even better fun if we made a program to do all of it for us: download the file from the internet and redirect the incoming data stream to the decoder. Ideally, we would just provide the url of the remote mp3 file and soon we would be listening to the song hot off the network. All we need is to make a program that can open a socket to a remote host, use HTTP to request the file from the server, and than just feed the incoming data from the socket into the standard input of the decoder/player to which we will open an input pipe.

Introducing webamp...

And here's our program, webamp. Have a look at the code, which is pretty basic, and see for yourself how it works.. I will be explaining the details shortly.

0  #!/usr/bin/perl
1  use Socket;
2  
3  my $handler = 'splay';
4 
5  my $url = shift @ARGV;
6  $url=~m/http\:\/\/([^\:^\/]*)(?:\:(\d+))?\/(.*)/;
7  my $host = $1;
8  my $port = $2;
9  $port = 80 unless($port);
10  my $file = '/'.$3;
11 
12  my $proto = getprotobyname('tcp');
13  socket(SOCK, PF_INET, SOCK_STREAM, $proto);
14  print "Looking up $host..\n";
15  my $sin = sockaddr_in($port, inet_aton($host));
16  print "Connecting to $host:$port..\n";
17  connect(SOCK, $sin) || die "Connect failed: $!\n";
18 
19  my $old_fh = select(SOCK);
20  $|=1;
21  select($old_fh);
22 
23  print "Requesting $file..\n";
24  print SOCK "GET $file HTTP/1.0\n";
25  print SOCK "Accept: */*\n";
26  print SOCK "User-Agent: webamp $version\n\n";
27  print "Waiting for reply..\n";
28  my $header = <SOCK>;
29  print "$header\n";
30  exit unless($header=~m/200|OK/);
31
32  while($header = <SOCK>) {
33         chomp;
34         last unless(m/\S/);
35  }
36 
37  my $content; 
38  open(HANDLER, "|$handler") or die "Cannot pipe input to $handler: $!\n";
39  print "Redirecting HTTP filestream to $handler..\n";
40  while(read(SOCK, $content, 512))
41  {
42          print HANDLER $content;
43  }
44  close SOCK;

How it works.

Line 3

This is just the name of the program that we will use as a decoder. It is important to use a decoder that is capable of accepting standard input as we are going to be piping the data stream to it.

Lines 5-10

Here we parse the URL that was passed as an argument from the command line. The URL consists of three components of interest to us. The hostname, the port and the filepath relatiev to the server's root. A simple regular expression (line 6) helps us identify each and store them in variables $host, $port and $file. Note that if no port is specified we assume port 80, the default HTTP server port.

Lines 12-17

Here we create a socket and try to connect to the remote server/port. If all goes well, we wnd up with a filhandle SOCK bound to the socket. We will be using this filhandle to communicate with the remote program. (the web server in that case) Printing to the filhandle sends data and reading from the filhandle blocks waiting to receive until incoming data is available to satisfy the read request.

Lines 19-21

In order for this to work we must turn off the buffering of the stream. Line 20 turns off buffering of the selected stream. We first select (in line 19) the socket filhandle, then switch off the bufferring, and then restore the previously selected filhandle (good housekeeping rules dictate leaving things as you found them, unless there's good reason for not doing so) which we had saved on our first select().

Lines 23-30

Now that our communication channel is set up, we may send our HTTP request (lines 24-26) to the remote web server. Then we must wait for the server to respond. We read the first line from the response which contains the status code and message. If it's 200 OK, this means that our request can be satisfied and that headers and content follow... otherwise we abort. (This is a far too simplistic approach at interpreting server responses, but our main concern here is to provide a demonstration rather than a sophisticated web client that can deal with redirects, caching etc)

Lines 32-35

Here we just loop over the headers ignoring them to get to the actual content. Again this is probably not an appropriate tactic for a serious web client, but we just want to provide a working demonstration at the moment.

Lines 37-44

Finally we ger to the fun part. In line 38 we open a pipe to the inpu of the handler program, and right after that, we start reading through the incoming data stream and redirecting all data to the handler which will decode and play the music.

Further Improvement

There is a lot of room for improvement in this program. First of all one desirable feature would be to also store the incoming data into a file as well as passing it along to the handler. So if at the end of the day we like the song, we have it stored somewhere to play it again later. Another obsious enhancement would be to change the last part of the program to buffer the incoming stream and keep a lookahead of a few Kbytes so that we reduce the chance of jitters in the decoding. Finally, note that this is a generic streamer application after all.. we can use any program as a handler, so with appropriate handler programs for various formats we could implement streaming of any other file type. We could even provide the ability to choose the handler from the command line, so the same program can be applied to all streaming uses we wish.

Online Documentation/Tutorials

We have a short introductory article on socket programming with perl, that you might find interesting
We also have an introductory article on HTTP that provide some ground for understanding how the web works behind the scenes.
Have a look at LWP, a powerful package that provides you with all the facilities you could possibly dream of for creating web clients.

Save This Page

Comments


Mike	Posted at 10:12pm on Saturday, April 28th, 2007
Nice simple tutorial. Submitted in queue @ tweako ( http://www.tweako.com )
Pete	Posted at 9:08am on Friday, May 23rd, 2008
Yup, very nice, simple example. Thanks!