Moving iTunes libraries from one hard drive to another

For the love of me, I don’t know how I ended up here again. The last time I moved my iTunes library to a bigger disk, I was able to use the Consolidate Library command and let iTunes do all the file moves. This time… not so much.

I was trying to do two things at once with this move: put all my music onto a larger external drive (a 1TB MiniStack from OtherWorld Computing. Can’t recommend the enclosure enough for form factor, ports, and reliability; my older 500 MB drive is in the same enclosure), and move to the new iTunes library layout, where there are separate top level folders for music, movies, podcasts, etc. This didn’t work, because partway through the process writing to the disk errored out. I quickly realized that the problem was that I was doing it over the network (the new 1TB drive and my older drive were both connected to my AirPort Extreme). So I directly connected the new drive via FireWire, left the old one on the AirPort, and tried to consolidate again. Only this time, it told me I didn’t have enough disk space. On the new 1TB drive.

What?

Then I realized what was going on. The restructuring of the library wasn’t moving files, it was creating another copy of the files on the same drive. I had started consolidating before I did the library restructuring, so now it was trying to write a second copy of all my music on the drive. Since I probably have about 515 GB of music to work with (some resident on my MacBook Pro’s internal drive), two copies weren’t going to fit on the new drive.

So now I had: a full copy of my library spread across the old drive and my hard drive; one partial copy at the root of my new drive; and another partial copy in the proper location in a Music subdirectory. I didn’t want to delete either of the partial copies because songs in my library were mapped to both locations; I couldn’t reconsolidate on the old drive for lack of space. But I still had over 160 GB free on the new drive, so I could probably copy over the missing files by hand.

So I’m going the manual route to clean up. First, I went to the Terminal and ran the comm command, which compares two text files line by line; I fed it the directory listings of the old external drive and the new final destination, and it spit out about 165 differences, directories that didn’t get copied from the old drive to the new one.

Second, I’m going line by line through the results of the comm listing. For each line, I:

1. Copy the missing files from the old drive to the new one.
2. Delete the old files from the old drive.
3. Go to the matching tracks in iTunes and do a Get Info. Amazingly, since I’m copying the new files into the correct directory structure, a lot of the time I don’t have to do any more work and the library automatically finds the files in the new drive. Sometimes I have to browse one file at a time to link up to the new files, which is a drag.

I expect I’ll be done with the process sometime next week. Painful, but at least no data is lost. Then I can repeat with any files from my laptop’s disk, a much shorter list.

The final cleanup may take some XSLT fu. I will need to triple check the final library to make sure no tracks point to locations on the old drives. I’m going to try using XSLT on the iTunes Library.xml file to see if I can cull out the problem tracks that way; if not, it’ll be a matter of trial and error, because there’s no convenient way to find the file system locations of iTunes tracks from within iTunes itself, other than one at a time.

I’d love to have this be  a more error-free process. I’m beginning to think that iTunes libraries on external drives simply isn’t a well tested scenario by Apple.