I know how many people there are who worry about this on here, so thought I would share my experience.
Every time I deal with the Home Office I expect the worst, but to be fair to them they have always been pretty efficient.
Background:
October 2003 - July 2005, working holiday visa (note this time does not count for ILR purposes)
July 2005 - July 2006, one year HSMP visa
July 2006 - July 2010, four year HSMP extension.
I applied for ILR by post in early April 2010 and received passport and visa back today (1 May 2010). I know I could have applied in July 2009 but with my absences I wanted to do some more time and not push my luck.
None of my absences were work related.
Since 2005 I have been out of the UK for a total of 265 days. Removing travel days, the number was around 235. Removing weekend/Bank Holiday trips it was around 220.
My longest absence was 106 days, of which just over 90 was unpaid leave.
I guess I have been lucky, but I did work for the same employer (a well known university) for my whole time in the UK and I submitted full payslips for 52 of the last 55 months plus an employer letter confirming my service.
Good luck.