Under fire for the remark, Vijayvargiya, a BJP General Secretary, today withdrew his controversial tweets but asserted that had there been intolerance in India, Shah Rukh would not have been the most popular actor after Amitabh Bachchan. He, however, refused to apologise for his remarks.