James Brogan's family said "none of us will see him pop his head around our doors and crack a joke making us laugh"