I am in the process of being made redundant, if I started my own business could I claim back the PAYE I have paid in the past?

There are two main ways in which you could claim back PAYE paid in the past. It seems from your question that you are ceasing employment in the middle of a tax year, and starting your own business. If so it may be that your earnings from your new business may be small, and when you get to the end of the tax year you may find that the tax deducted while in employment (based on your PAYE notice of coding) was too high, taking into account your personal allowance. If so you could claim back the overpaid PAYE tax of this tax year. Secondly, the tax legislation (Income Tax Act 2007, section 72) says that where a loss occurs in any of the first four tax years of a new sole trade, or of a new partners’ membership of a partnership, relief may be claimed against that person’s general income of the three previous tax years, earliest first.