Karnataka Chief Minister Siddaramaiah on Wednesday expressed confidence that Congress party would win by-elections in all three constituencies of the state.