May 2013 - A Latent Existence

ZFS and Ubuntu Home Server howto

A while ago I bought myself a HP Microserver - a cheap, low power box which has four 3.5″ drive bays and an optical drive bay. I bought it to run as a home server which would back up all my data as well as serve up video, music and photos around the house. I had decided before buying that I wanted to store my data using the ZFS filesystem since ZFS was the only filesystem at the time which offered guaranteed data integrity. (It still is the only filesystem of release quality which offers this, although BTRFS is catching up.) I have become almost obsessed with ZFS because of the overwhelming benefits it offers but I won’t go into them here. Instead I recommend watching this talk by the creators of ZFS (Part 1, part 2, part 3) or read through the accompanying slides. [PDF]

I meant at the time to write about how I set up my system but never did get around to it, so here is what I did in the end. The server arrived with 2GB of ECC RAM and a 250GB hard disk. I eventually upgraded this to 8GB RAM and added two 2TB hard disks, although I started with one 2TB disk and added the second as a mirror when finances allowed. ZFS checks the integrity of the stored data through checksums and so it can always tell you when there is data corruption but it can only silently heal the problem if it has either a mirror or a RAID-Z/Z2 (Equivalent to RAID 5 or 6.)

ZFS is available as part of FreeNAS, FreeBSD, Solaris, and a number of Solaris derivatives. I initially installed FreeNAS 8. FreeNAS runs from a USB stick which I put in the handy internal USB socket, but while that was great for storing and sharing files it was not so good for running bittorrent on or using SSH to connect from out of the house. I also tried Solaris but I ended up going back to what I know and using Ubuntu Linux 12.04 LTS. Although licensing prevents ZFS from being included with Linux it is trivial to add it yourself.

I have assumed a certain level of knowledge on the reader’s part. If it doesn’t make much sense to you then you might be better off with FreeNAS or an off-the-shelf NAS box.

After installing Ubuntu and fully updating it I did the following:

sudo add-apt-repository ppa:zfs-native/stable

sudo apt-get update

sudo apt-get install ubuntu-zfs

…and that was it. It is a lot more complicated to use ZFS as your root filesystem on Linux, so I don’t.

Update: as of Ubuntu 16.04 ZFS will be supported directly. You will be able to install ZFS with the following rather than adding a third-party repository:

sudo apt-get install zfsutils-linux

Next, I had to set up the ZFS storage pool. The creators of ZFS on Linux recommend that you use disk names starting with /dev/disk/by-id/ rather than /dev/sda, /dev/sdb etc as they are more consistent (particularly the wwn identifier) so look in that folder to see what disk names you have.

ls -l /dev/disk/by-id/

The example pool name given is tank but I strongly recommend that you use something else. To create a single disk storage pool with no mirror:

sudo zpool create tank /dev/disk/by-id/wwn-0x5000c5004f14aa06

To add a mirror to that later you would type:

sudo zpool attach tank /dev/disk/by-id/wwn-0x5000c5004f14aa06 /dev/disk/by-id/wwn-0x5000c500400303dd

Or if starting with two disks to put in a mirror, your initial command would be:

sudo zpool create tank mirror /dev/disk/by-id/wwn-0x5000c5004f14aa06 /dev/disk/by-id/wwn-0x5000c500400303dd

I prefer to use mirrors as they are generally faster, however if you want a RAID5-type setup use:

sudo zpool create tank raidz1 … … … (3 or more disk identifiers)

The system will create your storage pool, create a filesystem of the same name and automatically mount it, in this case under /tank.

“sudo zpool list” will show you that a pool has been created as well as the raw space in the pool and the space available.

“sudo zpool status” will show you the disks that make up the pool.

While you can just start storing data in your newly-created filesystem (in /tank in our example) that isn’t the best way to use ZFS. Instead you should create additional filesystems within your storage pool to hold different types of data. This will allow you to do things like set compression, deduplication, quotas and snapshots differently for each set of data or backup an individual filesystem with zfs send. You use the zfs command to create your filesystems. Some examples:

sudo zfs create tank/music

sudo zfs create tank/videos

sudo zfs create tank/backups

The above examples will create filesystems in the pool and will automatically mount them as subfolders of the main filesystem. Note that the name is in the format pool / filesystem name and there is no leading slash on the pool name.

Check that your filesystems have been created:

sudo zfs list

Now we need to share the data, otherwise it’s not much of a server. ZFS will automatically manage sharing through NFS (Unix/Linux) or SMB (Windows) but you must first install the server software. For sharing to Windows clients use:

sudo apt-get install samba

To add NFS use:

sudo apt-get install nfs-kernel-server

You don’t need to configure much because ZFS handles most settings for you, but you might wish to change the workgroup name for Samba in /etc/samba/smb.conf.

To share a ZFS filesystem you change a property using the zfs command. For Windows clients:

sudo zfs set sharesmb=on tank/music

sudo zfs set sharesmb=on tank/videos

For Unix / Linux clients:

sudo zfs set sharenfs=on tank/backups

Or you can share the whole lot at once by sharing the main pool. The sub-filesystems will inherit the sharing property unless you turn them off:

sudo zfs set sharesmb=on tank

sudo zfs set sharesmb=off tank/music

You can check whether your filesystems are shared or not:

sudo zfs get sharesmb,sharenfs

At this point you should be able to see your shares from other computers on the network but you probably won’t have permission to access them. You will need to ensure that the file permissions and owners are set correctly, and you will also have to add an account and set a password for use when connecting through Samba. If your username is ella then use:

sudo smbpasswd -a ella

to set your Samba password, and make sure that ella has permission to access all the files in your shared folders:

sudo chown -R ella:ella /tank/videos

Other useful features of ZFS that you should look up include snapshots and zfs send/recieve. I hope this short guide has been helpful if you are trying to set up a ZFS server. Let me know in the comments.

Updated 29/02/2016 to remove some personal details, add information about ZFS support in Ubuntu and add some explanations noted in the comments.

Whistleblower says Atos Work Capability Assessments are unfair

A doctor who worked for Atos carrying out Work Capability Assessments has resigned and has told the BBC that the assessments are unfair. He says that pressure was put on Atos staff to change the outcome of the reports.

But Dr Wood has criticised some of the tests which he says contain “dubious concepts and shaky reasoning”.

He claims assessors are told that if a claimant can walk from the kitchen to the sitting room, it proves they can walk 200m (650ft); and if a person can dress themselves once during the day that is proof they have enough concentration and motivation to hold down a job.

He insists these rules are not published in handbooks and guides, instead they are simply spoken about in training sessions.

Dr Wood, who was given special responsibility to champion mental health at Atos, said: “I was instructed to change my reports, to reduce the number of points that might be awarded to the claimants. I felt that was wrong professionally and ethically.

Watch the BBC Six O’Clock News report:

BBC News talked about this some more an hour later:

Tom Greatrex MP has written to David Cameron to urge him to order an investigation.

BBC News: Disability benefit assessments ‘unfair’, says ex-worker

The Guardian: Atos benefit claimants face biased medical assessments, doctor alleges

Benefit Scrounging Scum - Q: When is a target not a target? A: When its a statistical norm

Sue Marsh - What’s the state of ESA?

I hate telephones

I have always had a problem with talking on the phone, long before I ever encountered depression or anxiety as long term problems. People who know me might find that funny because I used to sell mobile phones for a living and I have had a smartphone since the days of the Nokia communicator. What they may not realise is that for me a smartphone is a pocket computer with an internet connection. I regard its ability to receive voice calls as an unwanted extra.

Unfortunately society demands that I do actually talk on the phone, so what is the problem?

Interestingly I actually find making calls for business or calls to services much easier than calls to friends or to people who I know outside of their job. When I worked as an IT technician, and before that selling mobile phones, I could quite happily phone someone to answer or ask technical questions, or to organise something related to work. I think this is because such calls have a defined protocol, a script. I know that I will go through certain phrases and steps in order. I can also call people who I talk to a lot such as my parents or my wife. The problem for me comes when I have to call a friend where there is no script or protocol. In such situations I don’t know what will happen or what I should say and my anxiety kicks in. This gets worse when I am unsure how well a person knows me, or if a person will recognise me or remember me, or will want to speak to me. Then my anxiety gets very bad and I am most unlikely to actually make the phone call at all.

I also hate answerphones. My mind tells me that this is silly, because at least answerphones are machines not people and so I should be less anxious about the whole thing. What actually happens is that I prepare myself to talk to a person, go through the script in my head, only to get a message demanding that I explain myself RIGHT NOW. The result is that I panic and say something stupid, or I hang up.

This whole problem is complicated even more right now by the physical health problems that I have and by the anxiety and mental health problems that I have suffered from for the last three years. On days when my fatigue and pain or cognition are very bad I just cannot speak on the phone. I may lack the energy to speak, or be unable to make the connection from thoughts to voice, or unable to complete my thoughts. (Which means I might be happily tweeting or chatting online but unable to talk on the phone.) Holding the phone for too long can cause extra pain in my hands and arms and back, and talking for more than a few minutes is exhausting. On top of all that, now I also get anxious about being anxious, and receiving a phone call can send me into a panic, which is why I often ignore calls from numbers that I don’t know. Unfortunately this can lead to things like me ignoring calls from my wife when her bike has broken down and she has borrowed someone else’s phone!

If it’s not a bedroom tax then it’s not a spare room subsidy policy either

This is a clip from the Six O’Clock News on Radio 4 today. It is about a woman who killed herself and left a note blaming the government cuts to her housing benefit.

I suppose I should be grateful that the BBC are reporting this story at all, because most people who get their news from the BBC would hardly know that cuts to welfare are even happening. However, I am furious about this story because of the way they phrased the report. Here’s how they referred to the cuts:

“her benefits were being cut as a result of the coalition’s spare room subsidy policy.

…she was facing financial difficulty because of what critics have called the bedroom tax”

This is repetition of government propaganda. True, the cut is not called the bedroom tax. Nor is it the removal of a spare room subsidy, because there never was any such thing, merely people receiving enough housing benefit to cover their rent in the available social housing. The official name of this cut in the legislation is the under-occupancy penalty. Because that’s what it is - a penalty for having a spare room, even if you had no choice about the number of rooms in the home you were allocated or if you need that room for medical equipment or numerous other reasons. It was never, ever a subsidy in the first place to remove. “Removal of the spare room subsidy” was a name given to the cut by a panicking government because people were calling it a bedroom tax and that was too close to the truth.

That the BBC repeated the official government line and referred to a spare room subsidy when reporting a suicide is a serious problem. Of course I don’t believe the BBC is unbiased any more; if anything the BBC follow a pro-government line no matter who is in power. But if the BBC won’t call it a bedroom tax then they shouldn’t call it a spare room subsidy either. They should use the official name of under-occupancy penalty, but they won’t because the word penalty is too revealing about this government’s actions when they are still claiming that this is not about money and that they are protecting the most vulnerable.

PIP Judicial Review given the go ahead

The court gave the go ahead on Friday the 3rd of May for the legal challenge against PIP to proceed to a full judicial review.

This decision represents a big step in the process as it means that a judge has heard from both my own lawyers and the Department of Work and Pensions and has decided that the case has enough merit to proceed to a full hearing in July.

Although the case was initially brought by three people through two different solicitors (Two people in conjunction with We Are Spartacus and one separately) the DWP have convinced the court to put two of the cases on hold and to proceed with only one case. Our lawyers have agreed that my case is the best one to proceed to a full judicial review. Solicitors for the other two cases will be watching my case and will request to rejoin the proceedings should they have something different to add.

We Are Spartacus: High Court gives green light to PIP legal challenge

A Latent Existence: Why I am suing the government

Tumblr: Why I am suing the government (Quick version)