The spokesman said the couple had stopped from the UK en route to The Hague to watch the Invictus Games, adding that Harry had previously stated that he hoped to see his grandmother.
Buckingham Palace declined to comment.
The Sun reported that Harry and Megan also saw Prince Charles during a visit to Windsor Castle, which he said was their first joint trip to Britain since resigning from their royal duties in March 2020.
Harry, who moved to the United States with Megan, did not attend last month’s memorial service for his grandfather, Prince Philip, Duke of Edinburgh, who died in April last year.
His non-participation followed a dispute with the government over security protection.
The queen, who turns 96 later this month, had mobility problems and did not attend the annual Maundy service at Windsor’s St George’s Castle chapel on Thursday. He will also not be attending an Easter Sunday service at Windsor.
(Report by Michael Holden, screenplay by James Davey, Edited by Sandra Maler)