FLR(M) refused..right to appeal given but need advice please

[RustyDug]

My partner arrived in the UK 7 years ago after marrying a British woman. The marriage didn't last and he applied for ILR on the basis of domestic violence in 2007, but was refused and became an overstayer (not a good idea, I know, but nothing we can do to turn back time). We met a while after that and became a couple though we each had our own homes and didn't live together. Last September, we decided the time had come for us to marry and become a family (I have two kids at home) but he had no passort since his ILR application as it was retained by UKBA. We contacted them and told them that we wanted to apply for COA, and outlined the situation. They wrote and asked him to send a photo to prove that he was indeed who he said he was (presumably they wanted to compare it to his passport pic) but then we heard nothing until the end of January 2011 when they arrested him and informed him that he was liable to administrative removal on the basis of his overstay. He has to report fortnightly and is no longer allowed to work.
Left with no income, he gave up his home and, together, we moved into our present home (I was previously in social housing but gave it up to be sure UKBA wouldn't say he wasn't allowed to live there).
Anyway, we couldn't marry because of the COA but did apply for FLR(M) based on our relationship. Then, when the COA was abolished, we got married in the registry office (July).
Today, he was told that his application has been refused because they don't believe we've been together long enough and he has 14 days to appeal. We don't have a solicitor and have been getting advice from CAB who made our original application for us and chased UKBA for updates.
Now, we know we have to appeal but I'm freaking out, convinced that my husband is going to end up back in Nigeria!! Has he any chance of a successful appeal? Can anyone advise me on how best to make the appeal and what angle we need to go from please? (I work and support him, all our household bills are in joint names, my daughters accept him as their step-dad and, unlike when we applied for flr, we are now married.)
Oh, and I'm British. BTW, I've seen claims of human rights breaches in similar cases but they always seem to be about the rights of the person seeking to remain here - what rights do I have as a British citizen?

Thanks for any advice, Rusty