Winds of more than 100km/h have felled trees and damaged buildings across Victoria overnight, as emergency services say some people should reconsider unnecessary travel.