Still going after all these years.
First Panther note: some of my scripts don’t seem to work under Mac OS X 10.3. Specifically, Look Up Current Track in iTunes doesn’t work and iTunes2Manila fails because it can’t connect to the Internet. Wonder if Apple changed how SOAP calls work…
Update: Changed or more precisely broke, it looks like, based on evidence from this MacScripter thread. Stay tuned…
As I mentioned earlier this week, I’m running an AppleScript once an hour to download data from Weblogs.com. For anyone who may be interested in seeing how it’s done, the code is below and is easily adaptable to other scripting languages, including DOS Batch (I know, I’ve done it). This is first pass code, not optimized, warts and all. Enjoy.
copy getWeblogsChanges() to {success, theFile}if not success then
display dialog "Could not get the changes file from Weblogs.com."
return
end if
processWeblogsChanges(theFile)
on getWeblogsChanges()
set changesSource to "http://www.weblogs.com/changes.xml"
set theDate to current date
set theYear to year of theDate as string
set theMonth to month of theDate as string
set theDay to day of theDate as string
set theTime to time of theDate as string
-- need to prompt for the user's directory instead
set theFileName to "~/" & theYear & theMonth & theDay & theTime & "_changes.xml"
set theShellCommand to "curl " & changesSource & " > " & theFileName
--need error handling
do shell script theShellCommand
return {true, theFileName}
end getWeblogsChanges
on processWeblogsChanges(theFile)
set theShellCommand to "gzip -c " & theFile & " >>changesxml.gz"
do shell script theShellCommand
end processWeblogsChanges
I noted that I had let a new high water mark go by last week; the weblogs.com data watch has been updated and you can download the new data set here. The new high water mark is amazing, too: out of nowhere, it jumped from under 3000 to above 4000 weblogs in a three-hour period. Anyone know what was happening on Friday to drive that much traffic???
Anyway, this was the kick in the butt I needed to look at my cron script that I set up to download the Weblogs.com changes file. I had set cron to run a custom AppleScript (source to be shared shortly) to download changes.xml every two hours and gzip it, or so I thought. Looking at it today, the first day I left the machine on overnight since adding the cron setting, I realized I had asked it to download the file once a minute during the 2 am hour instead. Oops. Sorry about the bandwidth, Userland.
So why is this important? As I’ve been saying for a bit, I want to understand the dynamics of a day and a week in terms of blog posting frequency. Which are the high traffic days? What percentage of blog users post more than once a day? More than once every few days? Just how many unique blogs ping Weblogs.com in a two week period?
Starting today, I’ll be working on finding out. My cron script is now working (it’s amazing the difference between * 2 * * and * /2 * *). My machine won’t be sleeping or shut down for the next two weeks. I’ll make my summary data available at the end of the experiment and see if I can draw some conclusions about the meaning of the high water marks we’ve been seeing. Hopefully, if I’m successful with the project, this can be a longer term study. But for that to be true, I’ll have to automate the process of importing the data file and aggregating the statistics, and that may be too much to get done right now.
Is anyone else engaging with the changes data in this way? Are there any questions about the weblog population that two weeks of granular update data would provide?
As I promised a few weeks ago, I’ve spent a little more time working on AmazonHandler. The biggest problems people have had with it are that
Number one is fairly trivial: you can either put the supporting script, SOAPXMLRPCHandler.scpt, in the standard location, which is in the computer’s /Library/Scripts directory—the one for all users, not your local version, though it should probably check both—or you can edit the InitSOAP subroutine to tell it to look for the script elsewhere.
Number two is harder, and I spent some time working through it last night. The trick is in knowing that various parts of the return have to be explicitly transformed either to lists or records before their parameters can be read. Once you do that, it’s fairly simple to parse the output.
I’ve hacked a quick demo script that is actually somewhat useful. Until I work out all the bugs, it will be available as a separate download; after that, I’ll probably bundle it with AmazonHandler. The script, Look Up Current Track in Amazon, talks to iTunes to get information about the currently playing track, then looks up the track’s album (or artist, if no album title is available) at Amazon, tells you what Amazon’s current price is, and offers you the option to go to Amazon’s page for the product should you wish to consider the option of purchasing it. So (buzzword version) this script integrates iTunes and Amazon using SOAP-based web services.
The script is available for download on my Scripts page.
I’ve had no less than four people contact me within the last few weeks to tell me that AmazonHandler, my AppleScript glue to allow calling the Amazon Web Services API, isn’t working for them. The most common error is “Can’t make ‘http://soap.amazon.com/onca/soap’ into a record” but others have been reported as well, including problems with finding and loading the support scripts to do SOAP.
I think the first problem is caused by some code I put into SOAPXMLRPCHandler to work around a bug Apple introduced in the underlying SOAP layer sometime in the late 10.1.x/10.2 timeframe. The support script problem is tricky. The point of scripts is that they’re supposed to be quick and easy, so making the user install support scripts seems dumb. But it’s quicker and easier for me to release scripts that use common technologies (including the Manila API and SOAP generally) if I keep the scripts separate.
I promise I’m looking at it, if only because having this script would allow me to automate some of the workflow around “Current Listening” and “Current Reading.” But be patient—I have a day job, after all.
Dave continues to explore the joys of getting Userland’s flagship workgroup blogging solution, Manila, up and functional at Harvard. Today’s story captures him learning about a potential conflict between Frontier’s built-in web server and Microsoft IIS, if both are running on the same box.
This is exactly the sort of thing that more software developers, and CEOs, should do—get out into the wild, outside the company IT environment, on the other coast or in another part of the world, and try to install and operate your product. Does it work like you expected? Uh oh, it doesn’t! Have you documented the problem? Is it a bug or a compatibility issue? Has anyone else had the same problem? Is your website any help? Kudos to Dave for using his new job as a way to improve Userland’s products and support.
Brent points out that there’s a non-trivial cost to supporting multiple kinds of weblogs in a blog front end tool like NetNewsWire or Manila Envelope. Weblogs may look alike on the surface (and to an RSS aggregator), but they’re all different software platforms underneath and accordingly have different information needs. This is fine as long you don’t have to expose the mess to the user.
This is one of the reasons, I think, why the MetaWeblog API came about. Abstracting the common elements of the data elements into a new API layer is smart. Unfortunately not all the platforms support it…
… with being a part-time hobbyist developer. I’d love to implement trackback pinging in Manila Envelope, but I really, really don’t have the time right now.
more…
Sincere apologies for the temporary triple posts. It looks like I got false error messages from Manila Envelope making me think I had to repost. Ah well. Another day, another bug.
After the big release of 1.0.3a on Tuesday, a reader in England found some bugs I had missed in the first go-round with Manila Envelope on Jaguar. Specifically, people using the software for the first time can’t post!
The problem was a result of two things. First, the problem with Jaguar requiring plain text for XML-RPC extended into an additional function call that I hadn’t investigated, one that passed a URL as one of its parameters. This particular call only happens if no successful calls to the server have been made before, so I didn’t catch it in my haste to release 1.0.3a. Once again the importance of thorough testing…
Second, I made a big fuss about the importance of separating user interaction from business logic back in February, but didn’t catch all the problem areas. This particular problem was caused by a function that returned a string value. In the first implementation, I had the calling functions make the assumption that an empty string meant an error had occurred. Only problem: if the calling function in the UI is handling error notification and I’m returning a single empty string parameter, there’s no way to provide detailed error information.
This is where I miss “pass by reference” in other programming languages. It then becomes simple: you pass in a variable to be transformed by the method, and you check a return value for success or failure. Instead, I am using multiple return values:
return [Macro error: Can't compile this script because of a syntax error.]
which means I have to use AppleScript’s copy syntax instead of a simple set. Ah, growing pains.
The release notes are here. This version adds Jaguar compatibility. It has not been tested with pre-10.2.1 versions of Mac OS X; feedback is appreciated. Also noteworthy: this version now uses XMLRPC exclusively. You can download the software here; source code is also available.
I’ve deliberately avoided adding new features in this release since it took me so long to get the Jaguar bugfixes figured out. But there may be a couple of new features coming soon.
…I got iTunes2Manila working again. The problems appear to be rooted in changes to Apple’s implementation of XML-RPC and SOAP in Jaguar:
set s to (s as record)'s «class ktxt»
I am testing my other scripts with these fixes (which reach into the supporting library scripts ManilaHandler and SOAPXMLRPCHandler) and will post fixes soon.
Sigh. Not yet. I just tried my modification of the iTunes2Manila script that encodes the HTML tags as entities. No joy…still posts an empty message body. Time for some serious debugging.
Apologies for not blogging sooner. Between slow network connectivity here and the generally fragile state of editthispage.com, I’ve been unable to get to my site all day.
I had some insights on the problems with my scripts on the flight back. By reviewing the Apple-published scripting list archives, I realized that others were having problems publishing via XML-RPC (and probably SOAP) any text that contained unescaped HTML tags (e.g. < instead of <). This explains why my iTunes2Manila script was publishing with an empty body and should be easy to fix. There were also suggestions that processing the arguments to an XML-RPC or SOAP call to use plaintext instead of Unicode (or vice versa) would result in successful RPC calls. I may have some head down time on that shortly and hope to be able to bring my scripts up to Jaguar compatibility soon.
more…