BT Again

The pipex adsl saga continues.

Fortunately(?) I am at home, ill, today. Got a call from a BT engineer to say he would be calling with me.

So he called, ran his tests, and surprise surprise, the connection was working 100% all the time he was here. It has actually been better over the last few days, but still dropped out over the weekend.

As an aside, check out these sets of stats :
http://ccso.co.uk/stats/1/
http://ccso.co.uk/stats/2/
One is for us, the other is for a neighbour about one mile away. Surely that looks like a significant pattern.....

Fun in the snow

I usually try to get away from work on a Friday at about midday, as that's what it says in my contract (I like to try to stick to my contract where possible ;) )

Today I had special reasons to leave at exactly 12:00 -
1) Snow on the roads would slow my journey home
2) The kids had to be picked up by 1pm
3) (most important) I had to get home to play in the snow

Managed to get home just before 1pm, so spot on timing, which is more than could be expected as there were various reports of journey time multiplication factors of over 6, so 30 minute journeys were taking several hours. My 35-45 minute journey took 55 minutes. Not bad really.

A fresh fall had started, and the wind was blowing snow from fields through gates onto the road. The roads were mostly passable, although "the mountain" was a bit mad.

Anyhow, picked up the kids, got home, and straight out into the snow. Matthew and I were hardly inside all afternoon. Kate popped in and out. We made snowmen, used fertiliser bags to toboggan down the hill, and generally had a lot of fun. Great!

Had to make the most of it today, as apparently its all going to go away tomorrow.... at least that's what the forecasters say.... although they are the same forecasters who predicted that we wouldn't actually see any snow here (we had 5" which is the most I can remember)

Photo organisation

I have a system for storing my photos that I'm largely pleased with - I simply put them in folders that allow alpha-numeric sorting by date - e.g. "2007-03-22 Visit to Spain"

However, with 400 such folders containing almost 20,000 photos, and a small number of folders in my old system, like "Beach" or "Kids Photos" or even "Unsorted" it was time to do something about it.

I like the cygwin toolset, which is basically a full Unix toolset (grep, vi, sed, etc) for windows. It also comes with perl, and a program called 'exif'. The exif program spits out all the data for an image to the console as shown at the bottom of this post. This allows me to simply get the date/time of the photo, allowing me to rename by date and sort by time.

I wrote a small perl script to handle this for me, and now I can quickly take a set of photos in a folder called "Kids Photos" and have the script drop them into one folder per day. Then I can rename the folders how it suits. Should have done this a long time ago.

After renaming all my non-conforming photos and folders, I made top level folders called 2002, 2003... etc to limit the number of folders in the top level. It's an aweful lot tidier now.

Disclaimer - if you find anything useful here, that's great, but it's all at your own risk. If you don't understand it, don't run it! Backup your photos before you start ;)

Script - should anyone want it:

#always
use strict;
use warnings;
use Getopt::Long;

# option for just stating what should be done,
# but not actually doing it
my $print_only = 0;
GetOptions( "p" => \$print_only );
if ($print_only)
{
 print STDERR "Running in print only mode.\n";
}

#first scan all data
my %file_data;
my %dates;

print STDERR "Reading file info...";
foreach my $file (@ARGV)
{
 next if ( $file !~ /jpg$/i );
 print STDERR ".";
 my $exif = `exif '$file'`;
 my ( $date, $time ) =
   $exif =~ /Date and Time.*\(20\d\d:\d\d:\d\d)\s(\d\d:\d\d:\d\d)/;
 if ( !$date )
 {
  print "date : $date [$exif]\n\n";
  print "Not enough exif info for file $file\n";
  next;
 }

 #replace : with -
 $date =~ s/\:/-/g;

 #store the file data for later
 $file_data{$file}->{'date'}      = $date;
 $file_data{$file}->{'time'}      = $time;
 $file_data{$file}->{'date_time'} = $date . " " . $time;
 $dates{$date}++;

}
print STDERR "Done\n";

#create the directories
if ( !$print_only )
{
 foreach my $date ( keys %dates )
 {
  mkdir($date);
 }
}

my $counter = 0;
foreach my $file (
 sort { $file_data{$a}->{'date_time'} cmp $file_data{$b}->{'date_time'} }
 keys(%file_data)
  )
{
 $counter++;

 my $date        = $file_data{$file}->{'date'};
 my $print_count = sprintf( "%03d", $counter );
 my $target      = "$date/$date $print_count.jpg";
 if ( -f $target )
 {
  die "cannot overwrite target $target";
 }

 my $cmd = "mv '$file' '$target'";
 print "$cmd\n";
 if ( !$print_only )
 {
  `$cmd`;
 }

}
__DATA__

Exif example output:

EXIF tags in '2007-05-03 Coastal Drive 001.jpg' ('Intel' byte order):
--------------------+----------------------------------------------------------
Tag Value
--------------------+----------------------------------------------------------
Manufacturer KONICA MINOLTA CAMERA, Inc.
Model DiMAGE G400
Orientation top - left
x-Resolution 72.00
y-Resolution 72.00
Resolution Unit Inch
YCbCr Positioning centered
Compression JPEG compression
Orientation top - left
x-Resolution 72.00
y-Resolution 72.00
Resolution Unit Inch
Exposure Time 1/13 sec.
FNumber f/4.7
ISO Speed Ratings 50
Exif Version Exif Version 2.2
Date and Time (origi2007:05:03 19:43:16
Date and Time (digit2007:05:03 19:43:16
ComponentsConfiguratY Cb Cr -
Compressed Bits per 3.40
Brightness 31/10
Exposure Bias 0.0
MaxApertureValue 3.00
Metering Mode Center-Weighted Average
Light Source 0
Flash Flash did not fire, compulsatory flash mode.
Focal Length 5.6 mm
Maker Note 688 bytes unknown data
FlashPixVersion FlashPix Version 1.0
Color Space sRGB
PixelXDimension 2272
PixelYDimension 1704
Custom Rendered Normal process
Exposure Mode Auto exposure
White Balance Auto white balance
Digital Zoom Ratio 0.00
Focal Length In 35mm34
Scene Capture Type Night scene
Gain Control Normal
Contrast Normal
Saturation Normal
Sharpness Normal
Subject Distance RanUnknown
InteroperabilityIndeR98
InteroperabilityVers
--------------------+----------------------------------------------------------
EXIF data contains a thumbnail (3168 bytes).

Snow!

After a nice white frosty Christmas day I was hopeful that we would see some more snow. Well, tonight we got it! We must have had a good 2 inches fall over the last 4-5 hours, and now its freezing hard.

According to the BBC news there is havoc across the country, as people abandon their cars! They need 4wd :)

I spent quite a while digging to get the Passat up the drive, and just then guests arrived. We hoped they could get up the same tracks, but an hour later the tracks had

gone.... so back to the digging and brushing, and after a few attempts they were up. Of course the scooby just walked up up... :)

Getting to work tomorrow will be fun.

Christmas... again!

We hadn't managed to see my Mum and Dad over Christmas, so they came over today for Sunday dinner. We had another super feed, and the kids enjoyed seeing their grandparents again too.

The present-of-the-year appeared in the form of a bow and arrow set hand crafted by Granda, which pleased the kids no end. To be honest, it amused everyone greatly, as during the first demonstration one of our fine glasses bit the dust. Or rather it became dust! It was good that everyone saw the funny side, in fact Susan and I could hardly stop sniggering and laughing.

After dinner we got all the other presents opened, and did Christmas all over again. Great! Again!

Happy Christmas!!

We awoke this morning to a beautiful, frosty winters morning. The kids did what they were told, and waited until 7am before retrieving us from our slumber.

Downstairs, and it turns out that we must have been on Santa's good list, and we also haven't reached the age threshold for Santa gifts either. Phew.

Lots of happy children, then off to church to meet the person its all about. After that we had a great family meal, followed by opening yet more presents, and yet more food. Great!

Moon and Fire

In preparation for the arrival of our Christmas guests we did quite a bit of tidying up, and as a result we had a lot of cardboard and paper to burn. This afternoon I lit a fire beside the house, and as with everything recently I used it as an opportunity to get the new camera out. Not very exciting, but little amuses the Simon.

While I was outside I noticed that it was a fantastically clear day, and the moon and stars could be seen clearly. Again, another experiment ensued, and I got the 55-200 lens out, pointed at the moon, and after a few settings, clicked, and viola! instantly a better photo than I had managed with my compact... I think I'm going to enjoy this new camera...

Gatorville again....

This week I was back in flat Florida again with my work. The downside of this all is that I am without family, and this time I am missing switching on of the Christmas tree lights, and my work Christmas night out among other things.

The upside is that it is sunny. It was between 20-24 degrees today and sunny. Due to last minute changes in plans I was left with a 200 mile drive, from near Miami to Orlando International Airport. As I was not really under time pressure, I took the "scenic" route, driving up the West side of lake Okeechbee, and following the 27 North through Lake Placid, and Dundee.

It was a nice drive, although I think if I was doing it again, I would leave even earlier, and use more minor roads, and 20 mile straights can be monotonous.

Of course I got to the flight on time, checked in, security, etc, but Air Traffic Control decided to be silly, and we were extremely late. The 1/2 hours just kept adding to the departure time, until we were so late I knew we would miss my connection to Belfast. I had hoped that the connection would be late too, but wouldn't you know, for once it was on time! Another day to spend in NY. And this time it's freezing!

Self Help

So after not having used the perl module Petal for a few years, I find I'm a bit rusty. Rusty in the software industry means using google a lot to find answers. And when I searched to find how to iterate over an array, and use the array as hash keys or sub keys, I was interested to find a question that I had posted to the Petal mailing list 2 years ago.

Unfortunately, at that time no-one helped me, but I figured it out. Fortunately I posted the answer myself.

Just goes to show how important it is to fill out an answer on a forum/list. How many times can you find people with the same problems as you, but no answers....?

WRC - Rally Ireland 2007

This year the World Rally Championship came to Ireland for the first time. Last night we set off in a camper van to Magheralough Primary School, near Trillick, Co Tyrone. This is part of the country I'm not very familiar with, but it certainly seems to be a very scenic place.

We arrived last night, parked up and had a poor nights sleep, before the alarm woke us at 7am. After a quick breakfast we ventured outside into the cold to pick out a spot to stand. The roads had been closed at 6am, and were not due to open until 6pm. We managed to get a good place, behind a fence where we could see the cars approach, then come down a small hill, cross the bridge, and then past us.

It turned out to be great, we didn't move a lot for the first stage (stage 12). The stage started at around 9:15am, and cars continued until about 11am. The drivers were clearly timid of the greasy roads, but there was plenty of quality entertainment.

After 3pm another stage (15) was run round the same course. After the first few cars we decided to take a wander up the fields a bit, and found a few other good places to watch the group 'N' and other trailing cars from. We didn't get as wet as expected, fortunately it seemed to rain it all away during lunch, when we were safely inside the motor-home.

All in all a great day.

See the stage, and school here: http://maps.live.com/?v=2&cid=EAF1783E98F25503!162&encType=1

Some more pictures here: http://picasaweb.google.co.uk/simonmcc/20071117WRCRallyIreland

Other info here : http://www.rallyireland.org

MySql Performance

After disappointment with how Oracle developer edition runs on my PC I decided to give an old acquaintance a whirl. I downloaded the latest MySql source, and built it in the cygwin environment (using the instructions in the DBD::mysql perl package).

It all installed nicely, and I downloaded TOAD to help set up my tables - another great (and free) tool.

My requirement is a simple lookup, so I set up a simple table, and a perl script to loop and insert values into it. I was reading a bit about optimising inserts, and found that dropping the index before all the inserts, and then adding it again is significantly quicker than keeping it enabled during the inserts.

Then I discovered that it is possible to insert multiple records at once. So I started to play around with this, and made the number of rows inserted at a time a configuration variable. The sql syntax is like "insert into table_name values(?,?,?), (?,?,?)" etc... where there are 3 columns in the table.

Not surprisingly inserting 10 at once rather than one speeded the times up from 2000/sec to 5000 a sec with significantly less trips to the database. Continuing the trend, I discovered that about 10,000 at a time was quite a good number, with around 130,000 inserts per second! Now that's pretty good!

After adding a single index on the 5 parameters I needed to search on, I was able to run well over 1000 queries per second, which is pretty good whenever there is over 11 million records in the database.

Another notable difference using MySql was that there seems to be no slowdown in inserts when the dataset grows.

Malin Head

For some time I have wanted to drive the Inishowen 100, a 100 mile tour round the Inishowen peninsula. Today we set out to drive round it, but we took a couple of shortcuts (i.e. got lost), and instead arrived at "Five Finger Strand", which the kids absolutely adored. There are some great sand hills which they enjoyed descending at great speed.

A fantastic beach, well worth a visit if you're in the area. Apparently we missed the coolest thing though, at low tide you can see the wreck of the Twilight which sank in 1889, on route from Newfoundland to Derry, so we'll have to make a return visit!

The other striking thing about the countryside was the sheer number of animals in the road!

Malin Head itself wasn't too exciting, especially as it was blowing a gale, and my company just wanted to "go home".

On the way home we happened to arrive at Greencastle just in time for the ferry. This gave us a quick route home.

Fireworks!

Tonight I took the kids into Coleraine for the local firework display. This year it seemed particularly amateur, but that didn't stop the little ones from enjoying it.

We were unsure of the start time, but we figured out that last year started at 8:30. We were there just before 8:30, only to hear several folks telling us that they just announced it was starting at 9pm. An extra long wait, especially with two tired children.

Walking on the beach

One of our favorite local walks is along Castlerock beach. I sometimes wonder if we should spread our wings, explore more, and not stick to the same old walks over and over. But on a night like tonight I remember why we visit so often... never fails to impress.

Downhill forest

This evening we went for a walk in Downhill Forest. It was a beautiful autumn evening, perfect for the walk, and the changing leaves made it just fantastic.

We spent quite a while rummaging around in the undergrowth too, looking for conkers, but we only found a couple. Seems like all of the village had been there before us. Maybe next year....

Pacific Coast Highway

Today it was time to go home. Rather than hang around we decided to go to the airport via the coastal route, and headed slightly south, and west to the coast. We followed the Pacific Coast Highway for the miles, stopping briefly at things that were interesting or cool.

One of the most unusual things were the hundreds of oil pumps draining the black gold from the soil, and beaches with oil tankers and oil rigs on the horizon. We also passed by an industrial estate with what seemed to be miles of refinery. Very interesting to see.

With every beach came more great swell, and surfers keenly waiting on the next big wave. All looked very nice, unfortunately we had no time (or equipment) to join in. Made the trip to the airport a lot more interesting though, I'd highly recommend this route to LAX, if you're ever going there.

Mountains

This afternoon we had some time off, so decided to do a bit of exploring. Firstly we went South to Cleveland National Forest, where you could drive through a bit of the mountainside, which was very cool. We thoroughly enjoyed the breathtaking roads and views, coupled with the beautiful sunshine.

After stopping off for some lunch, and having acquired a taste for the mountains we drove North towards "Big Bear Lake" - the local winter sports venue of choice. The cool thing about this part of California is that while the flat plains are only a few hundred feet above sea level they are consistently surrounded by mountains that soar into the clouds. Now, we all love mountains, and we all love a nice stroll in the hills, but we had no time for that. Fortunately, neither do most of the folks from LA, and so they've built great big roads that we can power up in our gas-guzzling V6 trucks.

So, up we went, 1000ft, 2000ft, the markers at the side of the road rolled by. We made our way up to Rolling Springs, the first settlement on the hill, just after the 6000ft marker, and it was cold. Unfortunately our relatively tight schedule didn't permit us to go any further, and I suspect it would have gotten even more spectacular beyond that.

Overall we rarely strayed more than 30-40 miles from the hotel, but we got to see a lot. And the best bit was that although we used thousands of gallons of 'gas', it only cost a few cents...

Silverado - Real America

So, I finally found it. This is what I wanted to see in the US. Silverado, a small village in the hills outside the main LA area in Orange county, California

This town represents a welcome escape from the over-urbanised generic America that we've come to recognise from modern TV.

On the way up here we got lost, and stopped with a guy who seems to live in the most ideal place in the word. He was almost apologetic, that he only lived in a "trailer", but what was cool was that his trailer was in a lovely scenic valley, bounded by conservation areas and national parkland. This means that he is assured that no one else will build around him. Fantastic.

Sunset over the Pacific

After work today we had asked a colleague for recommendations of places to visit. A coastal town called Laguna Beach came highly recommended, and so we decided to go in that direction. Due to the limited daylight we had to get wherever we were going as quickly as possible, but unfortunately we picked the road with an accident, and subsequent tailback. Anyhow, we arrived just in time for the sunset, which I have to say was particularly beautiful, and well worth the wait in the traffic.

Work Travel - LA, California

Once again it falls to me to travel for work. Today I travelled to LA, California in preparation for a weeks work, and I have to say the 11 hour flight is one I could have done without.

Aer Lingus made the flight all the more enjoyable with a personal TV system available, offering a selection of TV programmes, films and music.

It's a bit annoying travelling at this time of year, as the daylight disappears so soon, but I'm informed that it's never that brilliant in LA anyway - not like midsummer here when it hardly gets dark. But, no doubt I will try to make the most of it over the incoming week....