Question on Breach of Immigration rules - Naturalisation

[Naks]

I have ILR and am looking to apply for Nationality in November. However, we got our ILR after a successful ruling on the 10 years rule.

This is the time line

1995 arrived to the UK

2001 - Visa renewal refused based on student
Appealed in time
2002 - changed address and notified the HO
2004 - appeal refused and sent to old address
same year passports got lost in the HO
2005 - applied for extension of student visa not knowing appeal had been refused
2006 - visa given discretion exercised - our lost passports returned
2007 - applied for ilr based on 10years
Still not knowing 2004 appeal was refused
May 2007 got ILR refused based on the 2004 appeal refused although at the time we had already clocked 10years.
July 2007 - appeal hearing AIT - it was successful
Nov 2007 - got our ILR


My question is you should not be in breach of immigration within the 5 years to be naturalised- will we be affected because of the appeal refusal 2004 that we didn't know about?