Wednesday, May 31, 2017

Connecting to a Raspberry Pi behind a 3G Router

Experimenting with 3G router and '3' free 200MB/Mth data SIM and of course there is the NAT problem, so I did this:

/usr/bin/sshpass -p "password" ssh -o ServerAliveInterval=60 -o ServerAliveCountMax=2 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=15 -N -R 22222:localhost:22 -p 1234

Then I can "ssh localhost -p 22222" on my local pi and get connected to the one behind the NAT. Excellent.

Saturday, April 01, 2017

Raspberry pi eGalax touchscreen calibration on zero / Raspberry pi 3 / PIXEL

Software updates bring compatibility problems, that's just how it is. Although the eGalax touch screen drivers are now in the kernel, and the touch screen functionality works out of the box, I couldn't get the touch calibrated.

Long story short, this is what I ended up with. Sometime xinput_calibrator came out with a section like this :

 Section "InputClass"
        Identifier      "calibration"
        MatchProduct    "eGalax Inc. USB TouchController"
        Option  "MinX"  "2881"
        Option  "MaxX"  "62736"
        Option  "MinY"  "60381"
        Option  "MaxY"  "5495"
        Option  "SwapXY"        "0" # unless it was already set to 1
        Option  "InvertX"       "0"  # unless it was already set
        Option  "InvertY"       "0"  # unless it was already set

but that was no good!

Then, finally, it came out with a section like this:
 Section "InputClass"
        Identifier      "calibration"
        MatchProduct    "eGalax Inc. USB TouchController"
        Option  "Calibration"   "103 1962 1864 158"
        Option  "SwapAxes"      "0"

with the "Option Calibration stuff on one line, and this seemed to work. I put it in /usr/share/X11/xorg.conf.d/99-calibration.conf

Hope that saves someone else the time :)

Wednesday, March 29, 2017

moving from inittab to systemd

For better or for worse raspbian has moved from the old System V init process control to the newer systemd version. I liked my inittab file, because it was quite simple and extremely easy to use, but it is no longer a choice.

So, I had this old line in inittab:
Gar:2345:respawn:start-stop-daemon --start -p /tmp/  -m --exec /usr/bin/java  -v -d /home/pi/builds -- -jar garageController.jar

And now I want to run it on a new Raspberry Pi.

So, we need to create a service, its actually quite simple. I created the file /etc/systemd/system/garage.service with this text:
Description=Garage Door Control Software

ExecStart=/usr/bin/java -jar garageDoor.jar gpio


Then we need to run a few commands to make sure our new service auto-starts.

sudo systemctl daemon-reload

sudo systemctl enable garage.service
sudo systemctl start garage.service

sudo systemctl --all | grep garage
##this should show that the service is started.

That's it. 

(I used instructions from

Monday, March 13, 2017

Raspberry pi wont update time on NTP

Weird problem with a pi zero-w, where when I put it on my friend's network it gets the wrong time.

Bit of a workaround, rather than a fix, but it got me up and running, and should be reliable...

  1. Open rc.local file
    sudo nano /etc/rc.local
  2. add following line to the rc.local file
    sudo date -s "$(wget -qSO- --max-redirect=0 2>&1 | grep Date: | cut -d' ' -f5-8)Z"
  3. reboot the Rpi
    sudo reboot
Basically, it gets the HTTP headers from the server at and sets the pi's time to that.


Monday, January 02, 2017

Using a Raspberry Pi to fix a broken Ethernet mains switch

Energenie LAN Switcher

My father-in-law uses an Energenie LAN switch to remotely manage the lighting in a shed where animals are kept. This means that the animals can be monitored remotely day and night by CCTV, and the lighting can be controlled either via the local web server on the LAN Switch, or via Energenie's internet web interface.

However, it stopped working. Time to call in the Raspberry Pi.

I already control my central heating using a Raspberry Pi and an off-the-shelf relay module, so I wondered if the exact same solution would work for the shed lights. I also wondered if I could shoehorn a Pi and the relay module into the energenie casing, so I opened it up.

Inside with the old electronics - you can see scorch marks from the old board failing

It looked like there was enough space there, so I removed the old control board with the burnt out components, and the next stage was to get the software and hardware working. I had an 8 channel relay board and an old Raspberry Pi lying around, so I decided to use those.

It is a while since I installed or updated Raspbian, so I downloaded the latest version, and I have to say, it seems that there is a lot more there by default now, which is helpful. I quickly got my heating server installed and configured, and connected the relay board, and it all worked easily.

The Pi with a relay module - I'm only using 4 relays

Using this I was able to connect via the PHP web interface, or via the android app I wrote for it. It's strangely satisfying to hear relays click on and off when you press buttons on your phone.

As this is really just a quick fix I wasn't going to change the app at all, so it still gets the title "heating controls" and the "on" command is "boost" like it is for the heating.

I set the boost to put the lights on for 6 hours, so that is effectively as long as would be required in one go. The heating controller backend has no concept of "on" or "off", only timed programs.

With this all working, it was time to box it all up. This was probably the trickiest part of the project, but I managed to get the relay module and Pi into the box. I had to do a bit of filing, but it turned out to fit quite snugly allowing the Ethernet connection out the end for easy access.

Relay board wired in

Everything fitting snugly

And some photos with the cover back on

The system has been in the shed for a few weeks now, and is working perfectly. The app allows connection over the local LAN or over the internet seamlessly, depending on which network is available, and the text based protocol means that with even with only GPRS or Edge the switching is very fast.

Friday, December 30, 2016

ATEM Television Studio - live copies of the output file

We recently installed a ATEM Television Studio to mix cameras to an MP4 file in our church building. The idea originally was that we would burn this to DVD, and augment our CD ministry to include video distribution on DVD.
ATEM Television Studio
Since the days when we used cassette tapes, we have always distributed the media directly after the service - so people waiting 5 minutes or so afterwards could take copies of the service to deliver to those who can't make it out.

We wanted to keep this up, and so we looked into a DVD recorder to record the output of the Television Studio module directly to DVD, which could then be duplicated after the service. However, we hit several technical hurdles with this, and had to give up - mainly around aspect ratio.

So we were stuck with the MP4 file on disk. The process of converting this to DVD video, and burning it is slow. Even just burning the 2-3GB MP4 file directly to a DVD for duplicating is a slow process.

After a lot of head scratching we decided to distribute copies of our services on USB memory stick. Many people have a modern TV with a USB input, and we were surprised how many of our elderly people actually had such a device. With this, it is relatively easy to play the video file in superb quality on a TV.

But, we are still left with a problem - how do we write ~3GB of data to multiple USB sticks after the service? Writing to a stick could take 11 minutes, or about 3 minutes for USB 3.0 - but even at 3 minutes, if we have multiple copies to produce it could take quite a while.

Again, more head scratching, investigations into hugely expensive USB duplicators, and USB 3 hubs, but no satisfactory solution. Then the idea struck - why don't we write the data to the USB stick while the service is running - so we tail the file as it grows on the PC, and write the data out to each pen. The bandwidth, even in USB 2 would be more than sufficient to make quite a few simultaneous copies.

So I set to work on a piece of software to do this. I got over the problem of reading from a locked file quite easily, and had a prototype. However, when I ran it, the copy wouldn't play. By looking at binary diffs of the files I discovered that MP4 files have header information at the beginning, which is actually written at the very end of the process. I also discovered the ATEM seems to write large portions of the file filled with zeros, and then fills up the zeros with actual data.

Long story short, after many iterations of my software I got it working. Two strategies helped me:

  1. I don't read to the end of the file until it has been completely written - I stay about 5MB back from the end - this way I don't read any zeros.
  2. As I read each chunk and write it out to the USB sticks, I keep the offset, size and checksum of the data in an list. After the writing is complete, I re-read the source file from the start, and recalculate the checksum. If the new sum is different, I seek to the stored location in the file and rewrite the specified chunk. Typically it only rewrites the first chunk.
I added an option into the program to check the checksum, but since it is working properly, there is now no need to read the entire written data from the stick. 

Copies of the service are available about a minute after the service ends. Actually, these are available long before the CDs of the service become available now! And of course, the same software can be used to make copies of any service, not just the one we just recorded. It means you can make many unattended copies, only limited by the number of ports you have.

Why not YouTube?
Some people have suggested distributing service content via YouTube, but I'm not sure that fits with what we are trying to do: 
  1. It would take a long time to upload and process - some of our elderly people are watching their copy about an hour after the end of the service
  2. We could put internet access into the church building and stream live
    1. that would have associated ongoing costs
    2. our target audience for this ministry generally don't have easy access to the internet
    3. publishing online brings other concerns about who can see the content
  3. Each USB stick is brought into the home of someone who may be housebound, and not see very many visitors during the week. This contact is costly (time/effort) for the person who delivers the media, but incredibly valuable to the person who receives it..
YouTube may be something we think about for the future, but we don't see it as a suitable extension of our current ministry.

Tuesday, December 13, 2016

Screen Blanking On Raspberry Pi With Official Touch Screen

On the official RPi touch screen the screen goes blank after 30 minutes, and at first glance this seemed to do what I spent a lot of time scripting for an HDMI screen.

However, I noticed that using this default screen blanking the touch which unblanks the screen also gets sent to whatever program is below it, so if you happen to touch a button on the 'blank' screen, the button gets a press event.

So, in order to fix this I installed the xscreensaver
sudo apt-get install xscreensaver
and then configure it using
which allows me to set it to a black screen, and I also noticed there is the option there to have it activate the power management of the screen to turn it off. For this application I want the screen to go off after 1 minute, so I set that, and very easily I have the functionality I need.

The screen goes off after 1 minute, and any touch instantly brings the screen on, but doesn't get passed through to the underlying application. Perfect.

I also removed the default screen blanking, as I suspect there is a bit of an issue sometimes
edit /etc/lightdm/lightdm.conf
xserver-command=X -s 0 -dpms