A ChatGPT-researched guide provides too much wrong information to be trusted.
The unforgivable sin?
A lot of travel writers are going to hate me for this. Indeed, I kinda hate me for this, but I did it anyway.
I got ChatGPT to write a travel article for me.
ChatGPT vs travel writers
Treat this in the spirit of knowing your enemy. I’m deeply sceptical of what Large Language Models and generative AI can do.
I’ve also seen plenty of travel articles that have clearly been written by ChatGPT, and they are usually dreadful.
But I also have to face reality. ChatGPT may churn out cliché-ridden copy riddled with generic platitudes, but several publications have filled travel sections with this sort of dismal slop for years anyway.
In some instances, ChatGPT can do the job roughly as well as a staff writer being paid to knock up a listicle about places they’ve never been to.
Public take-up of AI technology
I also have to concede that the general public is using AI as a matter of course.
It is slowly replacing search on the web, and making search much worse in the process, as every tech company races to embrace AI, often in circumstances where it is detrimental to the user experience.
Can ChatGPT be useful for writers?
More controversially, I think writers need to concede ChatGPT and other LLMs can be useful on occasion. You can either rage against the dying of the light or accept that ChatGPT can be handy for brainstorming story ideas.
It can also be useful for restructuring an existing article, suggesting extra information to include in a story and giving tips for making the piece more search engine-friendly.
I generally hate the writing it comes out with, but think it has a place for the fiddly technical stuff, improving what you’ve 75% completed and blurting out lists of potential inclusions.
Using AI for travel information
For the average traveller, though, is ChatGPT helpful and trustworthy? I decided to give it a test.
One thing I thought would be useful is a list of which key museums and attractions in a city are open and closed on Mondays.
It’s the sort of thing that travellers would find handy, and it’s basically tedious information-checking that doesn’t require much writing skill.
I picked Oslo as the city, as I know it fairly well, and it has a lot of interesting museums.
This was my prompt:
Write a practical guide called “Which Oslo museums and attractions are open on Mondays?”
In it, list the key Oslo tourist attractions that are open on Mondays, with a one sentence description of what the attraction does and its highlights.
After each one, include a “See opening times or book tickets” call to action.
Bold up “see opening times” and “book tickets”, but not “or”. Embed a link to the opening times section/ page of the attraction’s website for “see opening times”.
Embed a link to the ticket booking page or entry prices page for “book tickets”.
Then do exactly the same for the attractions that are not open on Mondays.
The end result
I’ve published the article written as a result of this experiment – Which Oslo museums and attractions are open on Mondays? – but I think you should know what changes I had to make before it was acceptable for publication.
First of all, I had to check all opening times. I clicked through on each “see opening times” link that ChatGPT provided.
Around 60% went to a dead page, which is useless to any reader clicking through.
It was even worse with ticket booking – only around 25% of the links took me to a page where I could book tickets.
I had to manually find the correct links, and replace them, which is the sort of chore AI is supposed to be eliminating.
Wrong information from ChatGPT
Perhaps worse, some of the information on whether attractions open on Mondays was flat wrong.
ChatGPT’s guide said the Astrup Fearnley Museum and Ibsen Museum are closed on Mondays, which is not the case.
It also got the dates wrong for when the Norsk Folkemuseum is seasonally closed on Mondays.
I was also surprised by some of the omissions. I would argue that the Kon-Tiki Museum, Nobel Peace Center and Holmenkollen Ski Jump are distinctive, key attractions in Oslo. Although I can concede this is maybe a matter of taste rather than being a clear error.
Anyway, I researched these three attractions myself and added them to the list.
Should you use AI to plan your holiday?
Back to the clear errors, though. There’s a difference between the odd mistake creeping in, and a success rate this poor.
On the basis of this experiment, any traveller relying on ChatGPT to plan their itinerary is going to hit a snag fairly quickly.
I asked ChatGPT to list easily-findable information that is already on the web, and link to it. In several instances, it failed, either getting the answer wrong, inventing false information or putting in links to non-existent pages.
It is likely this will improve over time, but I think the limitations of the technology will always make it an unreliable researcher.
There’s nothing I, or any other travel writer, can do to stop holidaymakers banging queries into ChatGPT rather than reading our lovingly-crafted and researched articles. However, relying on AI is not a recipe for a hitch-free holiday.
