Police said other shoppers in the pharmacy reported seeing a man with a knife