Concerns have been raised by some as plans to use a historic Huddersfield town centre building as a takeaway are submitted to Kirklees Council.