Burrowed between mangroves and a bustling skyline, 70-year-old Rajan, who only uses one name, reminisces about his old home