Allsopp defends Tories' work and record spending as opponents point to lack of access and system in crisis