Hi, have a good read through
this webpage on the website of the BHC in Nigeria, and also the pages linked to from there.
Make sure you download a form VAF2 and also the INF4 information leaflet.
Clearly you will complete the application form honestly and therefore, when you read that form you will appreciate that the ECO dealing with your application will of course know about your time in the UK.
Also, have a good look around this Board. You will see quite a few messages from people who were in similar circumstances to yourself, or were failed asylum seekers. After going back to their country and applying for a spouse visa they tend to get the visa they are seeking. As said in an earlier message in the topic this is not a risk-free strategy but hopefully it will work out.
Before you go ahead and return to Nigeria, can I ask, has any contact been made with your MP? If not it might be worth trying to get the MP to contact IND and apply pressure to get the application decided sooner rather than later. After all, after three years, they really should know how they are going to respond.