A Digital Compendium of Hotel Mock Data

Stay in Chicago this summer, where The Bean comes alive.  Literally.

So, it's happened.  I've teased before that I'd completed my arsenal of mock data, but now I've compiled images for each representative city, hotel, and room rate.

Here are a few key stats:

• 90 representative locations with such awe-inspiring images as the one above.
• 1803 hotels.  It even did a great job of counting for an LLM.  It was only off by one from its target of 20 hotels 16% of the time.  That would be an astronomical batting average!  Even toddlers can't count to 20 this consistently!
• 7212 room rates.
• 1766 unique room rates.
• 936MB worth of images.
• $9.55 of hard-earned money to generate my iconic collection.

If you're interested in perusing the data, it's available in my Redesigned Adventure repo on GitHub.  It's open source (MIT-licensed), so if you need (or want) a gig's worth of fake travel data, feel free to use it to your heart's content.  If you do, I'd be very happy if you contributed any new or modified content back to the repo or drop me a line telling me how you're using it.  If you're perusing the code itself, it's a bit messy and will not be deterministic in creating your own data, but, if you do some detective work (or if I clean everything up in the near future), you can certainly roll your own.

What I've Learned

GitHub shows I made my initial commit on May 2nd, so I've been at this for about 2.5 months.  I took a brief pause here and there to work on my blog or concentrate on a side project or two, so the total time is probably closer to 2 months.  Given that I was working on this part-time, it's reasonable to assume that, under a deadline, I could've comfortably completed this within 1 month as a full-time pursuit.

That's not too shabby for the sheer volume of data I was able to generate.  Though there are often problems with accuracy, sparsity, and creativity in real content data, the reliability of AI-generated content is far below what's cultivated by human sources.  

If I were focusing on accuracy, I'd do more to scour the internet for data and rely on API calls to knowledge bases like Wikipedia or Unsplash.  The trade-off for fidelity is, of course, time.  In my case, I was more interested in realistic - but still fake - data that could pose for travel site data, but isn't fraudulent in its representation of reality.

As I've mentioned before, as long as you're not relying on the responses to be factual, this is a great use for an LLM and image generator (I used Llama3 for the text.  I used ByteDance's SDXL-Lightning for the image generation).

The majority of the LLM responses easily fall into the boring, but reliable, category.

This is Paris's description:

Indulge in the City of Love and Light: Visit Paris, France! Experience the magic of the Eiffel Tower, the world's most iconic landmark, and stroll along the Seine River, taking in the breathtaking views of the city. Discover the rich history and artistry of the Louvre Museum, home to the Mona Lisa, and wander through the charming streets of Montmartre, where artists and writers once found inspiration. Savor the flavors of French cuisine, from croissants to escargots, and soak up the romantic atmosphere of the most visited city in the world. Let Paris capture your heart!

It's not horrible, but there's nothing interesting about the writing.  Once you start reading other city descriptions, they're all structured similarly.  If you couldn't pick out that the description above was AI-generated, it's readily apparent after reading 3 of them.

If you couple that with one of the original images I rejected for Paris, though, you get an entirely different vibe:

Ah, I love seeing both Eiffel Towers at sunrise.

I remember reading an article recently indicating that AI image generators favor repetition.  Sprinkling Eiffel Towers throughout the image really means it's Paris.

I also learned that you don't want to get caught up too much in prompt engineering.  I've seen people write 1000-word prompts to give the machine precise instructions for newsletter generation without understanding the mechanism that makes them work.  While 1-shot or few-shot prompting (a jargon-encoded way of saying "give the LLM some examples" in addition to basic instructions) does appear to increase the reliability of the results, AI generators have, at best, the attention span of a distracted toddler, so don't expect them to listen to you (sorry for pretending it's a human, but some guarded anthropomorphism can be more precise) for more than a sentence or two.

I have been literal in my direction to them to specify an exact format with opening and closing brackets and the machines will still fail at least 1% of the time (even more often for less powerful models).  I told the image generator to create an exterior image of the hotel or an interior image of a common room, and only succeeded around 90% of the time.

It's possible that the AI generation tools I used may not be as advanced and that others will provide better results.  However, I don't think that's the case, based on output I've seen from other people, some of whom are unapologetic AI fanboys who would try to present the outcomes in the best possible light and still smugly tout the models' potential instead of their current capabilities.

So, skip the detailed instructions to an unreasoning object that doesn't care.  Include a few examples to make its responses more consistent, but don't waste your time trying to make it understand you via prompting alone.

If you're expecting accuracy, you'll have to do the hard work.  Whether or not that's tuning the model to your specifics, implementing Retrieval Augmentation Generation (RAG), or falling back to the tried and true methods of the late 2010s and early 2020s and relying on good ol' machine learning and in-house neural nets, that's up to you, but don't expect a free ride.

After this exercise, I'm still a firm advocate for the use of a code assistant.  I'm using Codeium (as opposed to Codium) for my assistant, and it does a solid job of helping me in its capacity as a front-line API looker-upper.  By solid, I'd say it's reliable 70% of the time.  Another 20% of the time, it gives me the wrong advice, but is something I can immediately recognize as wrong and either correct or ignore, so it still provides value.  The last 10% is when it tries to send me down the rabbit hole.

Keeping that in mind, I'd advocate what I'll deem the Rule of 3 - 

If you ask a coding assistant to generate code and it's unable to provide a coherent answer in 3 attempts, it's time to go hunting the old-fashion way.  LLMs don't have critical thinking skills, so they'll just double down on an optimistic, erroneous answer.  Often, they'll loop back to a previous erroneous answer, starting what's likely an infinite, unhelpful loop.

I faced this when attempting to use the Python library celery and when attempting to render macro templates using Jinja.  

In the former,  I wanted to use a simple in-memory message bus to back celery's queueing system.  Several LLMs provided actual code (thus enhancing the verisimilitude of the response) to connect the two systems, but after a lot of back and forth where I thought I was battling my own sloppy syntax issues, I discovered that the message bus and celery aren't compatible.

In the latter, the same problem existed - you can't render macro templates using Jinja, but I needed to ask the literal question (with the odd negative inversion to generate the appropriate response) "can I not render macros using Jinja" to understand it wasn't possible.  The LLM kept trying to help me render something that was impossible.

Perhaps the best use of a coding assistant - a surprising one - is that it helped dust off skills I haven't used for a while, particularly around SQL.  I grew tired of asking the assistant the same basic questions on JOINs, GROUP BYs, and DISTINCTs, so I spent the extra effort to recall my SQL syntax (which is still very rudimentary, but matches the height of my DB skills).  Now, I'll proudly state that I can almost describe from memory what a LEFT OUTER JOIN is.

And those are my primary thoughts during this whole exercise, I'm certain I'll recall other nuances or rehash key themes later, but I hope the bolded highlights above help you maintain expectations - and your sanity - if you go down the Gen AI route.

I'll leave you with one final piece of advice:  if you stay at the Jade Emperor Hotel in Shanghai, beware the Dragon's Den Room.

This room can have up to a 75% survival rate for guests.

Until next time, my human and robot friends.