I am running as a VAT registered Ltd.
I left university in 2002, and have worked as a Development Engineer in Medical devices, in that time I have been made redundant 3 times, but not gone without a job for more than 2 weeks at any point. In my line of work, when money gets tight, development is one of the first areas to be cut, hence the 3 redundancies, but there are plenty of small-medium companies with work that needs doing, so as a contractor, it is fairly good going.
Indeed I have not looked back since May 2011. My last PAYE job was one of a kind in the center of London, and that paid well, about as much as I could possibly expect to get at this point in my career without going into the dull world of management. Now as a limited I take home between 2-3 times that per year. Work is currently plentiful, indeed I recently had to choose between 3 contracts to take on, all equally good.
It is true that despite the best intentions, you will work longer, the money urges you to. Likewise you will holiday less. It is important to keep in mind that time off is valuable too!
My ltd was setup for me and all the admin such as invoices, returns etc is handled by my accountants,
http://www.danbro.co.uk. They are probably not the cheapest, and I must confess I didn't really shop around, but they do make things easy. I hope one day to be able to grow my business, take on a graduate, pay a fair wage and get them the experience they need to go on in medical devices (a particularly hard sector to get a foot into), then perhaps I won't have to be in a position of turning down 2 out of 3 contracts!
Good luck with whatever you decide, but going it alone has certainly worked for me.