Hey I'm new to this so help or advice would be much appreciated.

I own a 2004 525d and have had some break problems recently.
My break pedal was going very stiff and hard to press when my car was in an idle position or when trying to park. Fine when I'm driving at 20 mph plus but almost impossible to stop when I'm slowing rite down to a stop position, the more i pump the pedal, the harder it gets.After changing vacuum pump, brake pipe and changing the break servo unit, (albeit a second hand unit), the problem still persists. My mechanic told me that the only other alternative maybe to replace the brake servo unit again but this time with a brand new one. I'm afraid to do this in case I'm still left with the same problem and gone to the expense of replacing 2 servo units when all along it mite not even be the problem!
Can anybody help or provide any solutions?