An adviser has warned that the occasional flutter, no matter how harmless, could be the difference between whether lenders will offer mortgages or not.