Kate Middleton, Prince William and Kids Surprise Passengers on Flight to Scotland to Visit Queen

The royal family is ending their holiday with a big get-together at the Queen’s Scottish castle!

After a fun-filled summer that saw polo dates and a relaxing trip to Mustique, Kate Middleton and Prince William touched down in Scotland on Thursday with their three children to close out the summer with Queen Elizabeth at Balmoral Castle.

The family of five, including Prince George, Princess Charlotte and Prince Louis, took the short flight from Norwich to Aberdeen on the economy FlyBe plane. Surprised passengers saw the royal parents disembark from the plane with their children before they they got into awaiting cars to make their way to Balmoral. William carried a few bags with a little help from George, while Charlotte excitedly trailed behind them. Kate carried Louis in her arms as she walked beside the family’s trusty nanny, Maria Teresa Turrion Borrallo.

”Earlier in the flight, I’d had the feeling that someone had come on the plane that I’d recognized – but I thought it was just a footballer,” a passenger told The Sun. “We landed, and people then saw two black Range Rovers on the tarmac.

“Suddenly, Will, Kate, and the kids all got up, and left. They weren’t harassed by anyone at all as they got off the plane. They just seemed like a family traveling together to be honest. No one saw them get on the plane either. It was amazing how they did it — they must have slipped on quietly, after people were already seated.”

The family’s trip comes days after Meghan Markle and Prince Harry’s seaside vacation in Nice in the south of France with their 3-month-old son, Archie. The couple received criticism for their use of a private jet to get to their vacation destination, as some claimed it seemed to contradict their stance on protecting the environment. Singer and family friend Elton John spoke out in their defense, revealing that the Duke and Duchess of Sussex stayed at his home in Nice and that he provided the plane — which was made carbon neutral. (This typically involves calculating how much your flight generates in greenhouse gas emissions and financially contributing to a project that aims to reduce emissions by that same amount.)

“I am deeply distressed by today’s distorted and malicious account in the press surrounding the Duke and Duchess of Sussex’s private stay at my home in Nice last week,” he wrote. “Prince Harry’s Mother, Diana Princess Of Wales was one of my dearest friends. I feel a profound sense of obligation to protect Harry and his family from the unnecessary press intrusion that contributed to Diana’s untimely death.”

Celebrities and friends of the couple soon followed John’s lead with Pink, Ellen DeGeneres, Jessica Mulroney, Jameela Jamil and Nacho Figueras all speaking out in defense of the couple.

Meghan and Harry are expected to spend part of their summer with the Queen in Scotland as well before they take off for their fall tour in Africa.

For more than 150 years, the Scottish castle has served as a private retreat for the British royals. In the highlands, the royals like to entertain friends, hold dances, picnic and go on shooting parties – all the while battling the nibbling little flies that stalk the heather-covered mountain sides.

• Can’t get enough of PEOPLE’s Royals coverage? Sign up for our newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

The expansive Scottish property has been in the royal family since 1845, when Queen Victoria’s husband, Prince Albert, purchased the castle and the surrounding 7,000 acre estate.