The editor of The Mail on Sunday has refused to attend a meeting with Commons Speaker, Sir Lindsay Hoyle, following the paper’s controversial report about Angela Rayner.

Chorley MP Sir Lindsay told MPs on Monday that he had arranged for David Dillon to meet with him following the negative response to the paper’s claims the Labour deputy leader crossed and uncrossed her legs during Prime Minister’s Questions to distract Boris Johnson.

In his response to the Speaker, published in the Daily Mail, Dillon said he would not be attending as journalists should “not take instruction from officials of the House of Commons, however, august they may be”.

In his letter, he wrote: “The Mail on Sunday deplores sexism and misogyny in all its forms. However, journalists must be free to report what they are told by MPs about conversations which take place in the House of Commons, however unpalatable some may find them.”

He said that while The Mail on Sunday had “the greatest possible respect both for your office and for Parliament [which] along with a free press they are the foundation stones of British democracy”, the invitation would be declined.

Earlier, in a statement, Sir Lindsay said he wanted to use the meeting to ask that “we are all a little kinder”, issuing a plea to reporters to consider the feelings of MPs and their families when covering stories in Parliament.

He made the point that he had only recently rejected calls to remove the parliamentary pass from another journalist after some MPs called for The Mail on Sunday’s political editor Glen Owen – who wrote the report about Ms Rayner – to have his pass removed.

He said: “I am a staunch believer and protector of press freedom, which is why when an MP asked me to remove the pass of a sketch writer last week for something he had written, I said ‘no’.

“I firmly believe in the duty of reporters to cover Parliament, but I would also make a plea – nothing more – for the feelings of all MPs and their families to be considered, and the impact on their safety, when articles are written. I would just ask that we are all a little kinder.

“That is what I wanted to talk about at tomorrow’s meeting.”

Rayner appeared on ITV’s Lorraine on Tuesday morning, saying she had appealed to the paper not to run the story, based on claims by an unnamed Tory MP.

“When I heard the story was coming out and we rebutted it instantly… like this is disgusting, it’s completely untrue, please don’t run a story like that,” she said.

“All I worry about when I’m at the despatch box is doing a good job and being able to do justice to my constituents and the work I’m doing, so I was just really crestfallen that somebody had said that to a paper and a paper was reporting that.”