Building an Agentic Prompt Experiment Loop

Place2Page's landing page generation was already publishable. The question for this experiment was not "does it work?" It was "can we improve place-specific structure and visual rhythm without breaking the production baseline?" I fixed the actual production baseline instead of relying on a prompt file in the repo, then used external writing and design standards as experiment criteria rather than as prompt dependencies.

Summary

• Problem: Place2Page generated publishable landing pages, but some outputs stayed close to a safe baseline structure.
• Decision: The experiment baseline had to be the current production baseline, not a local prompt file.
• Approach: Split structure, anti-slop, and responsive rules into candidates, then repeated them against a fixed input set.
• Result: The combined candidate was the most practical direction because it preserved publish stability while improving structural variation.
• Lesson: By "agentic research," I do not mean handing judgment to AI. I mean having a person define the problem and evaluation criteria, then using AI agents to run repeatable experiment loops faster.

Context

Place2Page generates landing pages from Google Maps or Naver Map URLs. It gathers data such as place names, addresses, photos, opening hours, reviews, menus, and booking links, then turns them into HTML.

The feature was already usable. The problem was not whether it worked, but whether the result felt specific enough from place to place.

Local business landing pages easily converge to the same structure: hero, a few photos, menu or service cards, reviews, map, and CTA. That structure is not wrong. But if every result feels similar, a generative product starts to lose the expectation that it can adapt.

I wanted to answer two questions.

• Can the same place produce a better information flow and visual rhythm than the baseline?
• Can a more aggressive prompt stay publishable without mobile overflow or unsupported AI marketing copy?

I Started With The Production Baseline

The first step was to confirm what production was actually using as the baseline. Looking only at prompt files in the repo was not enough because the active production default can differ from the file default.

So the experiment started only after I matched the local baseline to the production baseline, not to the local file state.

This baseline was not bad. It included the basic information, used photos, and had reviews and a map. But the result still looked close to a safe restaurant landing page. The photos appeared placed inside an already-decided structure rather than strongly driving the page structure.

That created an important rule.

The experiment should ask whether the result improved over the current production baseline, not whether my local prompt became better.

I Used External Standards As Criteria, Not Dependencies

Next, I read external writing and design criteria. What I took from them was not a package or runtime dependency, but a set of prompt quality standards.

The initial candidates were split like this.

control-production
structure-first
anti-slop
responsive-hardening
combined

Each candidate had a different role.

• structure-first: prioritize section hierarchy and information flow instead of listing cards.
• anti-slop: reduce unsupported AI marketing phrases such as "premium" and "special experience."
• responsive-hardening: prevent mobile overflow, overlapping elements, and fragile grids at the prompt stage.
• combined: merge the three directions into one profile.

The experiment was repeated across four fixed places.

• Google restaurant
• Google fitness studio
• Naver workshop
• Naver beauty

The reason for using a fixed input set was simple. If every run used a different place, it would be hard to tell whether the result improved because of the prompt or because of the place data.

Before And After: Baseline To Combined

Applying combined to the same restaurant input made the structure clearer.

Compared with the baseline, several things changed.

• The hero became more concise.
• The photo grid looked more like visual evidence than a simple gallery.
• The menu section connected better with the food photos.
• Reviews and visit information landed in a steadier rhythm.
• The CTA and map area felt less detached than in the baseline.

The important point is not that it became flashier. It tried a more varied structure while still preserving publishable stability.

I also checked whether combined became too aggressive in other domains.

For the fitness studio, the structure differed from the restaurant result. Photos centered on the workout space and class scenes, and service cards did not look like restaurant menu cards. The same prompt candidate produced a different landing rhythm based on the business type.

I also tested a Korean place from Naver data.

This result showed an important point for local business generation. The same prompt quality standard worked across Korean, Naver data, and a workshop/craft domain. In other words, combined was not overfitted to one restaurant sample.

Stability Was Not Just An Automated Score

The representative full run summary looked like this.

Both control-production and combined scored 100. So it would be wrong to say that combined won by automated score.

The more accurate interpretation was this.

• anti-slop alone helped copy quality, but produced mobile overflow review cases.
• responsive-hardening strengthened layout guardrails.
• combined merged structure, copy, and responsive rules while keeping 4/4 published and 4/4 pass.
• In human review, its information flow and visual variation were better than the baseline.

The automated score was a guardrail. It helped catch publish failures, overflow, and obvious quality issues. Deciding which page was better still required human review.

The assets from this experiment were not only generated page screenshots. I also kept dashboard.html and variant_graph.html outputs so each run could show its candidates, scores, publish state, and prompt lineage in one place. After a later DB reset, some older prompt diffs became difficult to fully reconstruct, but these HTML artifacts still preserved the experiment flow.

The more important feature in that graph is prompt diff. The control has no diff, while experiment variants can open the exact prompt slots they changed. Below is the diff for the LOGI Korean few-shot candidate. It separates changes by slots such as freeform_html.map_instruction_suffix.v1, meta_prompt.contextual.v1, and meta_prompt.fixed.v1, which gives a human reviewer a way to inspect why a result changed.

The per-run dashboard separates automated guardrails from the parts that still require human review. In the English hero-copy run below, publish state, score, screenshot status, and variant rows are visible together, which makes it easier to spot copy that technically passed but still needs editorial judgment.

For a concrete output from this line of work, I can point to the LOGI Hannam landing page. I would frame it as a reference output from the experiment loop, not as a claim that this exact variant became the permanent production default.

I Also Tested A More Aggressive Dynamic Branch

Once the experiment loop existed, it became easy to ask the next question.

If combined was stable, what would happen if I pushed macrostructure and theme variation further?

That question led to the dynamic candidate.

This direction clearly created larger structural variation. But it was not a straightforward runtime default candidate. Place2Page's core promise is stable local business landing page generation. The dynamic branch was interesting, but depending on the business type and data quality, it could make the information flow less reliable.

So the conclusion of the loop was not "choose the boldest structure."

combined is more practical as a product default, while dynamic remains a candidate for the next experiment.

What I Mean By Agentic Research

I would call this work "agentic research."

By that, I do not mean asking AI to "research this for me" and accepting whatever it produces. It is closer to a workflow where a person defines the problem and judgment criteria, and an AI agent executes a repeatable experiment loop quickly.

This loop looked like this.

1. A person defines the problem
   - The page is publishable, but place-specific variation and visual specificity are weak.

2. The current production baseline is fixed
   - Confirm the actual production default, not just the repo file.

3. The agent creates the experiment surface
   - Experiment group, candidate, screenshot, score, and human judgment record.

4. Generation is repeated on a fixed input set
   - Google restaurant, Google fitness, Naver workshop, Naver beauty.

5. Automated checks are used as guardrails
   - Published status, score, overflow flag, pass/review judgment.

6. A person reviews the result
   - A score of 100 does not mean "best," so design and product review still matter.

7. Good directions and risky directions are separated
   - Combined becomes a production-default candidate, dynamic stays as a future experiment.

8. The result is documented and handed to the next product question
   - Record which prompt insight may become product behavior.

The point is not that AI writes a lot of code. The point is using AI as an experiment runner, observer, and repetitive-work assistant so that a person can spend more attention on higher-level judgment.

Follow-Up: Korean Hero Copy Felt More Obvious

When I revisited the experiment, one more thing stood out. Awkward Korean copy felt much more obvious to me than awkward English copy. That may be because Korean is my native language, but phrases like "special experience," "sensory space," or "the pinnacle of dining" expose the AI's atmospheric filler before they expose the actual place.

So I also looked at a 2026-06-05 LOGI Hannam hero-copy run that still existed in local artifacts. I did not touch the DB for this update; I used the remaining HTML and result JSON. The copy criteria were simple: as Shopify's landing copy guide puts it, the first screen should quickly tell readers who you are, what you offer, why they should care, and what to do next. Live product pages such as Stripe and Linear also make category, value, audience, and action readable immediately.

The few-shot copy editor candidate showed both progress and limits. It moved away from inflated control copy and brought source-grounded words such as fire, smoke, and charcoal into the first viewport. But using only LOGI as the H1 is too thin for a product default; a more concrete promise such as "Hannam wood-fire dining" still needs to sit next to the brand.

Lessons

First, a prompt experiment is meaningless if the baseline is wrong.

Because I matched the production baseline first, the before/after comparison was explainable. If I had started from the repo file only, I might have been experimenting against an already-drifted baseline.

Second, variation and stability have to be evaluated together.

New structure is valuable, but if it causes mobile overflow or weak information delivery, it is hard to use as a product default. On the other hand, if the output is only stable and always looks the same, the generative product loses part of its value.

Third, evidence still matters when working with AI.

Without experiment groups, candidates, screenshots, scores, and human review, a fast experiment becomes fast memory loss. What mattered here was not that AI generated many outputs quickly, but that the loop left repeatable evidence.

Fourth, a good experiment creates the next product question.

The next question after the combined experiment was, "Should I tighten the prompt so it uses photos better?" In practice, a better direction appeared: analyze the photos first, then pass that information into generation input. That leads to the separate note on the photo intelligence pipeline.

How I Will Judge This Next Time

When I run another prompt experiment, I will fix the baseline first. Repo files, local overrides, and the production baseline can differ, and if the comparison point is wrong, the result cannot be explained.

I will use automated scores as guardrails for whether a candidate can be promoted as a product default. Whether a page is better still needs human design and product review. When scores tie, I will compare structural variation, information flow, and failure risk together.

References

• Shopify - Landing Page Copy: Essential Elements + Tips
• Stripe homepage
• Linear homepage