Using cURL to retrieve multiple URLs from a text file

Posted Tuesday July 22nd 2008 from 53.3123, -6.2335

curl -K params.txt

params.txt:

url = http://tomholland.co.uk/blog
url = http://tomholland.co.uk/about
url = http://tomholland.co.uk/publications
output = file1.html
output = file2.html
output = file3.html

curl -K params.txt

params.txt:

url = http://tomholland.co.uk/{blog}
url = http://tomholland.co.uk/{about}
url = http://tomholland.co.uk/{publications}
output = #1.html
output = #1.html
output = #1.html

Will create files named:

blog.html
about.html
publications.html

Update:

An alternative to the second method:

xargs -n1 curl -o "#1.html" < urls.txt

urls.txt:

http://tomholland.co.uk/{blog}
http://tomholland.co.uk/{about}
http://tomholland.co.uk/{publications}

Will create files named:

blog.html
about.html
publications.html

If you're running Tiger (Mac OS 10.4) or Leopard (Mac OS 10.5) then curl and xargs are there by default (I'm unsure about Jaguar or Panther).

Tagged , and .

Codebase updates

Posted Sunday July 20th 2008 from 53.3127, -6.2468

Ben has recently published some code for routing requests to the correct controllers and views in a PHP MVC framework, so I've spent some of my weekend integrating it into my codebase (yep, this is the sort of thing I do on weekends).

Mapping URLs to site functionality has always seemed quite clumsy in my projects to date; Ben's solution has quite a Rails feel to it and is definitely the most elegant method I have encountered to date.

Tagged , , and .

Dropping support for IE 6

Posted Sunday July 6th 2008 from 53.3123, -6.2335

Regarding phasing out support for IE 6 across all 37signals products.

When I made the latest set of cosmetic changes to the blog, I decided I would no longer test in IE 6. If you are using Windows then you really should be using Firefox (or if you're one of those people who don't like v3, perhaps Opera). If using Internet Explorer is "using the Internet" to you, then you should accept the default choices you are offered when accessing Windows Update (which will include IE 7).

When I first started working as a Web Developer I was still testing in Netscape Navigator 4 and Internet Explorer 4. IE 5 was painful to support and IE 6 followed in the footsteps of its parent. IE 7 isn't perfect, but at least there are less bugs for developers to deal with.

Now if only the rest of the world would follow suit, then I could stop supporting it when developing sites for others.

Tagged , , and .

How not to build an unsubscribe system

Posted Tuesday May 13th 2008 from 53.3123, -6.2335

Apparently the concept of a 1-click unsubscribe mechanism is completely lost on those responsible for the Tesco website. Here are the unsubscribe steps required to stop e-mails from the Tesco online store:

Once you have logged in to www.tesco.ie follow these 4 steps to unsubscribe.

  1. Select the "log out/my details" tab from the top menu.
  2. Go to the "Update My Details" section and click on "click here to update your details.
  3. Scroll down to the "Data Protection" section and click on the link "click here to receive information about Tesco products and services".
  4. Check the box after the sentence "I do NOT want to receive offers and information about Tesco products and/or services." Press Submit.

It may take up to five working days to unsubscribe.

Sometimes I worry that I'm pretty thick and that one day I won't be able to find employment. Then I realise that some company is still paying the people who build these systems a wage, despite the utter complete mess they make of it - and I feel a bit more confident that I won't be jobless. I mean someone who built this system must have looked at the above and thought "yea - this is terrible".

Tagged , and .

Subversion hosting

Posted Thursday April 24th 2008 from 53.3127, -6.2468

I came across Beanstalk a few months ago, so I finally decided to stop sponging my Subversion hosting off Geffy and give it a try.

Tagged , , and .

Uploading files with ActionScript 2.0 & PHP

Posted Monday July 9th 2007 from 51.4598, 0.0021

If you have to write some server side code (such as PHP) to receive a file being uploaded through HTTP POST from ActionScript 2.0 (such as in Flash 8 - here is a good tutorial) - the $_FILES['Filedata'] variable which PHP can access is something defined in the Flash internals - not something a Flash developer sets.

Hopefully this will save some PHP and Flash developers a few hours of headache.

Tagged , , and .

Subversion and Collaboa

Posted Monday June 4th 2007 from 51.4598, 0.0021

Thanks to Geffy, I've been able to start using Subversion to manage my source code - this little blog may not seem like much, but with its MVC structure, is composed of over 50 PHP and JavaScript files.

As part of this, you can now access the source code using the Collaboa tool. Obviously, things such as usernames and passwords have been removed; but the potential for finding logic flaws still remains. I won't pretend I understood everything in the book (Ok - probably about 5%), but a paragraph from Bruce Schneier's Applied Cryptography sticks in my mind:

"If I take a letter, lock it in a safe, hide the safe somewhere in New York, and then tell you to read the letter, that's not security. That's obscurity. On the other hand, if I take a letter and lock it in a safe, and then give you the safe along with the design specifications of the safe and a hundred identical safes with their combinations so that you and the world's best safecrackers can study the locking mechanism--and you still can't open the safe and read the letter, that's security."

Tagged , , , and .

Location tagging blog entries

Posted Sunday March 25th 2007 from 51.4598, 0.0021

As some of you may have already noticed, I've added the ability to tag each blog post with a latitude and longitude; on the user front-end you will see a "Blogged from" link, which, when clicked, shows the location of the blog entry in a popup using Google Maps (if you're reading this via a feed, then you'll need to view blog entries on the site to see this). You can also view the location of all blog entries I've made; when clicked, the red markers will tell you the latitude and longitude, the nickname I gave that location (if it's a location I regularly blog from) and the number of posts made from that location.

The coolest bit is actually on the admin front-end, where you use Google Maps and a drag-and-drop marker to select the location you're blogging from. I hope to add location tagging to blog comments too, so you guys will get to use this functionality when you're leaving a comment on a blog entry (and then I'll be able to map a blog entry and where the comments on it came from).

I'm after some feedback on improvments that could be made, ideas on other ways to map the location data, etc. Matt and Geffy have already mentioned a few things (such as making the the "x blog posts made from here" on the "All blog posts" tab link to a page listing all the blog posts from that location). I'm also planning to add the ability to upload a photo for each location, which would appear in the map marker balloon.

Drop me an e-mail, let me know what you think.

Tagged and .

Tim Berners-Lee BCS Lovelace lecture

Posted Wednesday March 14th 2007 from 51.5167, -0.1120

Tim Berners-Lee gave the British Computer Society's 2007 Lovelace Lecture last night at the Congress Centre, on Great Russell Street in London.

It was free to attend (you just had to book in advance) and it was only about 15 minutes walk from where I work. The lecture content was similar to some of his other lectures I've seen webcasts of, but it was worthwhile going to a lecture in person, even if the majority of the audience weren't quite the young Web coding types I'd expected.

Update:
Lecture slides (Thanks Jean-Gui!)
Lecture on Google Video

Tagged and .

Gravatar vs Pavatar

Posted Wednesday February 21st 2007 from 51.5167, -0.1120

I'm looking to implement an avatar solution for comments; something more than each user just uploading an image.

A while back Geffy mentioned Gravatar, but there have been some reports of outages and closure to new signups. Pavatar looks like an interesting idea - but I'm not sure if caching the avatars (as per the Pavatar Candidate Recommendaton) limits the flexibility of the idea.

What do you guys think - Gravatars or Pavatars?

Tagged , , , and .

Firefox 1.5 and Atom 1.0 issue

Posted Tuesday February 20th 2007 from 51.5167, -0.1120

I've found an odd issue when using the "Live Bookmark" feature in Firefox 1.5 to subscribe to Atom 1.0 feeds.

Below are extracts from 2 example feeds (both of which validate against RFC 4287 using feedvalidator); one uses type="html" for each entry title, the other type="xhtml" (and a child div element with xmlns attribute, as required by the RFC):

<entry>
...
<title type="html">entry one</title>
...
</entry>

<entry>
...
<title type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">entry one</div>
</title>
...
</entry>

The latter example (using type="xhtml") is valid and works in Safari - but Firefox 1.5 does not display the entry titles in its "Live Bookmarks" feature. If you remove the child div element, the titles then appear in Firefox - but it invalidates the feed.

Upgrading to Firefox 2 fixes this issue; if you're stuck with 1.5 then RSS 2.0 would be the better feed choice.

Tagged , , , and .

Can you see the difference?

Posted Monday February 19th 2007 from 51.5167, -0.1120

Ok, you might not be able to tell, but I've rewritten most of the core blog code to use OO under PHP 5 - if everything is working correctly, it should appear almost exactly the same client side, with one or two additions.

It might seem kind of fruitless right now, but it makes the site much easier to maintain and more flexible when it comes to adding new features (yes - including the ever delayed comments).

Both the Atom and RSS blog feeds have moved, in preperation for new feeds which the new features will bring, so please update your feed subscriptions (I'll keep the old feed URLs functioning for some time, whilst everyone updates).

Drop me an e-mail if you spot any issues!

I don't want to promise too much, but I'm currently developing more location based features, including some Google maps integration.

Tagged and .

Fixed the missing blogs issue!

Posted Monday February 12th 2007 from 51.5167, -0.1120

I noticed at the weekend that some of my archive blog posts have been going missing - they were still in the database, but the site was reporting them as post not found.

It turns out that becuase the server time was set to UTC, when using PHP's strtotime function to create a timestamp that crossed a BST boundary, it would not take the hour difference into account (meaning the timestamp was exactly 3600 seconds out). I've managed to rectify this by using the date_default_timezone_set function, passing it the string 'Europe/London' and everything seems to be retrieved fine now.

Tagged and .

Playing SSL on Mac OS X

Posted Tuesday January 16th 2007 from 51.5167, -0.1120

Having had to setup SSL on a Linux hosting server, I decided it would be a good idea to have SSL using a self-signed certificate set up on my Mac for testing purposes.

If you find yourself with the same inkling, take a look at Using mod_ssl on Mac OS X.

Tagged , , and .

RSS 2.0

Posted Monday September 19th 2005 from 54.2811, -0.3994

After much whinging from the likes of Geffy, Matt & Luke, I have added an RSS 2.0 feed to the site. This represents about 75% of my readship. Aren't I a considerate ranter.

Tagged , and .