The actress and TV presenter made a vow not to let this "take her away" from her children and grandchildren