The two top Democrats in New York’s legislature withdrew their support for Gov. Andrew Cuomo on Sunday amid mounting allegations of sexual harassment and undercounting COVID-19 deaths in nursing homes.

Senate Majority Leader Andrea Stewart-Cousins became the first senior Democrat in the state to say the three-term governor should resign. Assembly Speaker Carl Heastie stopped short of demanding that Cuomo quit, but said in a statement that “it is time for the Governor to seriously consider whether he can effectively meet the needs of the people of New York.”

On Saturday, another woman who worked for Cuomo publicly accused him of inappropriate behavior, on the heels of other allegations in recent weeks.

“Every day there is another account that is drawing away from the business of government,” Stewart-Cousins said in a statement. “New York is still in the midst of this pandemic and is still facing the societal, health and economic impacts of it. We need to govern without daily distraction. For the good of the state Governor Cuomo must resign.”

Her push for his resignation came shortly after a Sunday news conference where Cuomo said it would be “anti-democratic” for him to step down.

“They don’t override the people’s will, they don’t get to override elections,” Cuomo said during a conference call with reporters when asked about members of his own party calling for him to step down. “I was elected by the people of New York state. I wasn’t elected by politicians.”

In a brief conversation Sunday prior to the press conference, Cuomo told Stewart-Cousins that he wouldn’t quit and they would have to impeach him if they wanted him out of office, according to a person who was briefed by someone who was on the call. The person spoke to The Associated Press on condition of anonymity because the call was intended to be private.

Cuomo said the next six months will determine how successfully New York emerges from the coronavirus pandemic.

“I’m not going to be distracted because there is too much to do for the people,” he said, noting that the state must pass a budget within three weeks and administer 15 million more COVID-19 vaccines.

Asked about Ana Liss, who told The Wall Street Journal in a story published Saturday that when she worked as a policy aide to the governor between 2013 and 2015, Cuomo called her “sweetheart,” kissed her hand and asked personal questions. Cuomo said such talk was “my way of doing friendly banter.”

He acknowledged that societal norms have evolved and noted: “I never meant to make anyone feel any uncomfortable.”

Liss told the Journal she initially thought of Cuomo’s behavior as harmless and never made a formal complaint about it, but it increasingly bothered her and she felt it was patronizing.

Karen Hinton, a former press aide to Cuomo when he served as the federal housing secretary under President Bill Clinton, detailed an uncomfortable hotel room interaction she had with Cuomo in a story published Saturday in The Washington Post.

Hinton said that as she got up to leave, he gave her a hug that was “very long, too long, too tight, too intimate.”

Asked Sunday about Hinton’s account, Cuomo said it was “not true.”

Cuomo’s workplace conduct has been under intense scrutiny in recent days as several women have publicly told of feeling sexually harassed, or at least made to feel demeaned and uncomfortable by him.

The state’s attorney general is investigating.