My Recent Tunes Code

Here is the code that I use to power my Recent Tunes list seen at right. It’s a little messy, but at least commented so you have some clue what each part is doing. The cover_update.php script is what does most of the work. You shouldn’t have to change anything in the amazon folder – that’s all there to talk to the Amazon.com API, which you will need access to. I also recommend Recent Tunes for updating your tracks from iTunes. Note that I use the current tune, not the list of recent tunes, as it makes managing the MySQL database easier and lessens the load put on Amazon’s API (which I imagine has a ~1000 query per day limit). If you have questions about using it, post a comment so that others can benefit from the conversation as well.

My Recent Tunes Code

BitTorrent Info Scraping

After using TorrentSpy on Windows for a while, I wanted a Mac version to do some of the same functions. TorrentSpy’s main feature is that it allows you to see the current number of users on a .torrent file in real-time. I was curious how it worked, so with a little traffic capturing with Ethereal, I was able to see that TorrentSpy was contacting the tracker listed inside the .torrent file at a specific address that was similar to the actual web address of the tracker. This special address, I figured, must hand real-time data back to TorrentSpy. After a minute or two of wandering around on Google (have I mentioned yet how much I love Google?), I came up with the protocol to be followed for transforming torrent announce URLs into “scrape” URLs on a Yahoo groups post. With this information, I started doing some coding in PHP to see what I could come up with.

The next problem presented was that of decoding a .torrent file so the information could be pulled out of it. A great open source project called TorrentParse came to the rescue here, as it provides PHP code for encoding and decoding BitTorrent metainfo files. As I would find out a short while later, these libraries are also pivotal in decoding the real-time information retrieved from the tracker server.

The code I wrote has only a small few parts. The first handles the uploading of a torrent file and checks to make sure it is of the required file type. The next partdecodes the given .torrent file with TorrentParse and extracts the announce URL (the address that the BitTorrent client notifies to show the new seed on the tracker). I wrote a small function called makescrape() that transforms the announce URL into a scrape URL per Bram’s post, linked to above. Upon requesting this new scrape URL, you are returned a BitTorrent-encoded chunk of data containing the real-time information about the torrent. Run that through TorrentParse’s decode function, and pull out the data you want to display.

I would post a demo of the code, but I’m afraid it would be automated and/or abused. However you’re free to download the code and try it out. It still needs some work, especially in handling and managing uploads, but it all works well enough to display basic real-time torrent information. It’s not exactly the Mac version I had in mind from the start, but it could be adapted to a Mac OS X widget with a little work. Get the code here. Enjoy.

BitTorrent Info Scraping

My Recent Tunes

A number of people have asked me how my Recent Tunes list works. It’s a little complicated, but here’s how.

First, you need to get track info from iTunes to your website in a fairly timely basis to keep the information accurate. Freshly Squeezed Software, makers of the popular Mac newsreader PulpFiction, has a nice freeware application called Recent Tunes which does exactly that. It can upload a list of recent tracks as well as the current track, all formatted according to a template you design. I don’t use the recent tracks list, and you’ll see why very shortly.

With the current track info on the website, it’s time to actually do something with that data. You could stop here and simply include the HTML file in your design with a small PHP command, and the current tune info would show up, but with no artwork or links. The next step is to create a timed or invoke-able script that will open up that current track HTML file, split apart the various chunks of information (artist, album, title, etc.), and go from there. Now, I should mention at this point that the current track HTML file that Recent Tunes uploads does not have to be a fully formatted HTML file, by which I mean all the standard tags can be left out — it will make your life easier while writing the script that splits up the data.

In Recent Tunes, I set my template to show the album, artist, title, and rating, each separated with a simple few asterisks *****. This string of asterisks is what your code will look for when splitting up song info. Do note that if one of your current tracks has five asterisks in one of those four pieces of information, your script will break unless that possibility is accounted for. PHP has a wonderful and amusing function called explode(), which separates data however many times is necessary, given a string to split at. In this example, you would do list($artist, $album, $title, $rating) = explode(“*****”,$trackinfo); After that, you could do some error checking to make sure none of those fields are empty. If one or more is empty, it might not be worth your while to even bother continuing with the script, so have it exit cleanly now.

Now that you have the song info from your computer, to your website, read in with PHP, and split apart into usable bits, you are ready to start looking up artwork, links, and more. Amazon.com has a very nice web service interface that provides just about every bit of info they have on a product, including artwork. This is where the album images will come from. The only problem presented, aside from the task of writing the code that will be communicating with Amazon’s infrastructure, is that their artwork isn’t a consistent size. They offer “small”, “medium”, and “large”, but the sizes in each category vary slightly. If we’re going to go to the trouble of doing all this, it should look good. The obvious answer is to make the album artwork the size you want.

Since the “small” artwork is already low in detail, it would look even worse if resized — any manipulations you make will lessen the quality each time. The best way to approach this would be to use the biggest available image and scale it to the size you want in one shot. Enter GD, an open source image library capable of handling common image formats. PHP must be compiled to use GD, but it’s quite common. I’ve found that most web hosts provide it. It is not built into PHP that comes with Mac OS X, however it is available as an installer package from Marc Liyanage‘s site. I always use his releases for local development and never have problems with them.

Once you retrieve the artwork from Amazon by matching artists and album titles, use GD to scale the image to 50×50 pixels and save it (just write it to a file with a unique name, say, an MD5 hash of the song info with “.jpg” tacked on). Amazon’s API also provides the product URL too, so hang onto that as well. The iTunes Music Store addresses are relatively easy to create. Using information I dug up on NSLog();, I was able to create a small function that returns an iTunes link. Once you’ve collected all this information and media, you’re ready to save it all to a database. MySQL runs WordPress, so it was only fitting that I use the same database only a different table. Do a MySQL “INSERT” command and log the song title, artist, album, rating, time (if you want), cover artwork path, Amazon product URL, and iTunes Music Store URL.

A good chunk of code to add here is one that will read the ID of the tenth newest row, and remove all rows and images that were before it. I found a good way to do it was read out the rows first, unlink the images using the paths set in the image path column, then MySQL “DELETE” that row, and continue until there are no more rows older than that tenth entry. In this way, you will keep your list of songs down and not gradually run yourself out of disk space by keeping ancient artwork files around.

Finally, when a visitor comes by, read out the last 5 or so rows (no more than your limit set in the previous paragraph) from the table and format them using some CSS. And last but not least — error checking everywhere. If artwork is not found or broken, or if the scaled image file doesn’t exist, or GD chokes on a corrupted Amazon image, simply don’t add that tune to the database and skip to the next one. It’s just a song.

I would release the code I wrote now, but it’s a little sloppy. I hacked it together in two days time, so I can’t guarantee anything! Let me know if you see any problems. If it continues to work well, I’ll surely hand it out.

Known Bugs: Albums with single quotes in any of the info cause the iTunes link to break. They are stored using the HTML entities ON for quotes, yet that seems to stop iTunes from understanding them. Not yet sure why that is.

My Recent Tunes

RCDefaultApp

I recently discovered RCDefaultApp, a Mac OS X preference pane that gives you incredible control over some things that are hidden away in the system. For example, it lets you set the default application for handling URLs with different prefixes such as feed://, irc://, news://, etc. It also controls every file extension and type handled by Mac OS X, which is nice because sometimes the “Change All” button in Finder info windows doesn’t always do what it should. There are a few other things that you can use it for, like setting handlers for different MIME types and URLs. If you need very specific control over different file types, prefixes, extensions, and handler applications, this prefpane is exactly what you need. It, too, is freeware. I’m also impressed at how small its file size is – 98 KB. What’s not to love?

RCDefaultApp

PlaintextPaste

PlaintextPaste is a small InputManager plugin that adds “Paste Plain Text” to the Edit menu of all OS X native applications. It’s useful for several reasons. When you copy text in Mac OS X, the formatting options such as font, size, and color come along with the text. I find that, most of the time, I don’t need or want anything but the text, especially when pasting into iChat. With PlaintextPaste installed, you can hit Command-C to copy text like normal, then hit Command-Shift-V to paste only the text, or Command-V to paste it with formatting. The second reason I like PlaintextPaste is that it has the option to swap the current selection with the clipboard contents – handy for programming. On top all of that, it’s free.

PlaintextPaste

iPod Super

What is this about?

Ultimately, I’d like to get a regular 3.5″ hard drive working with an iPod. The fact that it would be huge will simply be a novelty.

Why do this?

This project came about after I dropped my 40 GB 3rd generation iPod and killed the hard drive in it. I decided to open up the iPod and see what I could do with it. I could do so without fear of breaking it, since I’d already broken the most expensive part in it.

Disassembling the iPod

It’s not terribly hard to open the iPod if you know how. Since I had owned my iPod for some time, it had a little wear, and the seam where plastic and metal meet was open just enough to push my fingernail into. Sliding it down, I was able to stick in a nylon pry tool like the ones from RadTech. After getting the tool in there, you can pop the plastic catches and the top comes loose. OWC suggests squeezing the two halves of the iPod together at the side edge until it clicks. It looks a little scary to me, but I’ve never tried their method. I’m sure it’s perfectly valid, too. After that, detach the small white connector opposite the hold switch, and the two iPod halves separate. After moving all the loose items like the hard drive out of the way, they all disconnect without much of a problem — just be careful with those thin ribbon cables. Work slowly and don’t force things to come apart. If something’s resisting, it’s most likely because there’s a catch or screw that you overlooked.

Hard Drive Connections

After opening the iPod, I took a look at the hard drive that no longer worked. I had a 40GB Toshiba MK4004GAH. I looked up the drive on Toshiba’s site and found a pin diagram of the connector. The top-right pin in the following image is pin 1, and below it is pin 2 — the pattern continues in an up-and-down fashion. I remembered reading somewhere that 1.8″ hard drives used an ATA interface, but I wanted to confirm this. Another search yielded a diagram of a standard ATA/IDE connector. Except for four pins dedicated to logic and power, the two diagrams had the same connections.

Building the Adapter

Now that I knew the hardware would match up fairly well, I needed a way for the iPod to connect to a bigger drive. A post on the ipodhacks.com forums revealed a foreign company that sold an adapter similar to what I needed, but I couldn’t read German to figure out the site, much less order from them. Examinging the ribbon cable that connects the 1.8″ hard drive to the iPod, I saw that there was an open area at each pin where it connected to the actual pins that went into the hard drive. I spent the next several minutes cutting and stripping rather thin 30 gauge wrapping wire. I was going to simply solder wires onto each of those points and run them to a regular IDE connector. With the wire cut, I decided I should look into finding the other end of the connector before I began the tedious soldering task ahead of me.

It didn’t take long to dig up an old dead hard drive whose S.M.A.R.T. status indicated it was ready to go belly-up. I unscrewed the Torx screws on the drive’s circuit board and separated it from the metal housing. Desoldering all 44 pins would be a chore, so I took the easier route and just ripped through the circuit board with a Dremel. I also considered using an IDC connector from an IDE ribbon cable, but I figured that a female IDE connector would allow me more options when it came time to attach the iPod to other devices — I could use an IDE cable and run it to a hard drive, instead of requiring that two parts sit so close together. The next step was to spend some time soldering all 44 tiny wires and testing the connections with a continuity tester. A bead of hot glue was added to hold the wires more securely, and it was finished. Though not part of my plan, this adapter can also connect an iPod hard drive to an IDE bus if necessary.

Preparing the Hard Drive

At this point I read up on how the iPod’s hard drive is formatted. There are basically three partitions: the partition map, a 32 MB firmware partition, and the rest of the disk is where your music is kept. However, the first two aren’t normally visible. Even Disk Utility will only show you the third partition. I found a lot of good info on this page, including this explanation of the partitions:

1: The first partition of the hard drive (partition no. 1 above) is necessary to make the hard drive mountable, it contains the partition map for the disk. It’s size is 63-1 = 62 in blocks which equals 32 KB. This partition is known as ‘master boot record’.

2: The second partition ‘firmware’ from block 63 to 65599, 65536 blocks in total (equals exactly 32 MB), holds the firmware of your iPod. The type is ‘Apple_MDFW’.

3: Finally, the partition ‘disk’ is of type ‘Apple_HFS’ and is keeping the data on your iPod. The size of the last partition is [number of blocks] – [base of partition ‘disk’] which is 4.74 GB of the iPod used in this example.

Formatting the New Disk

Using the information about pdisk and dd on the page mentioned above, I was able to format a 3.5″ drive as the iPod would expect to see it.

Booted

With a 2.5 inch HD

Update:

Here are some iPod articles from Apple that might be of use:

iPod Super