To run market research with Claude without made-up quotes, split the work into four jobs. Scripts collect public posts through APIs (Reddit via the Arctic Shift archive, Hacker News, Stack Exchange, Discourse forums, App Store reviews). Claude reads each post and labels it: relevant or not, who is speaking, which pain point. A Python script then computes every number, and a final script finds each quote word for word in the stored corpus before letting it into the report.
The model never types a number and never copies a quote. We use this approach in fog's open source market-signal agent, on public data with no API key. Its reference run on October 12, 2026, on pre-seed fundraising, collected 65 posts, kept 30 after reading them and verified 21 quotes.
Market research with Claude: where a single prompt falls short
Most guides offer one long prompt that asks the model for the persona, the competitors, the trends and the customer reviews. The model answers fast and with confidence, and you have no idea where its sentences come from.
The typical answer to "do a market study for me"
Without a corpus, the model writes "representative" quotes with no source, gives sentiment percentages with no count behind them and cites real articles without dates, so you can't tell whether they describe this year's market or the one from eight years ago.
With web search turned on, the model reads snippets and paraphrases them. A paraphrased quote can't go on a landing page, in an ad or in a deck.
Our April 2026 run: one quote in twelve held up
On April 28, 2026, we ran an early version of the agent on accelerators in France. The synthesis relied on Exa and Firecrawl: 15 scraped sources, then a report written by the model. The report showed sentiment "estimated from text and framing": 27% positive, 53% negative, 20% mixed or neutral. Nobody had counted anything.
We replayed its quotes against its own scrapes using the verification script. Result: 11 blocked out of 12. Several quotes came from texts written in 2018 and 2020, even though the window covered 90 days. Others had lost a capital letter, stitched two passages together with an ellipsis, or appeared on no page at all. The report's "Frustrations (their language)" line matched none of the collected text.
That run is now a regression test: the verification script has to block it in every new version of the agent.
The six-step method
Each step produces a file, and each file has exactly one author: the model or a script.
- Collection plan (
plan.json), written by Claude: the sources, the queries, the window and a reason for each query. - Collection (
corpus.jsonl,health.json), done by Python scripts that call public APIs. - Labeling (
labels.jsonl), done by Claude, which reads every post. - Counting (
stats.json), done bycompute_signals.py. - Quote verification (
quotes-check.json), done byverify_quotes.py. - Report and handoff file (
.mdand.json), done bybuild_handoff.py.
The scripts only use the Python 3.9 standard library: nothing to install, no API key, no paid calls. Before your first run, launch the offline test from the agent's folder:
cd agents/market-signal
python3 scripts/selftest.py
# selftest: 50/50 checks pass (7.9 seconds on our server on Oct 12, 2026)
Step 1: write the collection plan
You give Claude the market in one sentence, plus the language and region. Claude writes the plan, and you review it before any collection starts, because a badly aimed query costs you a whole wave.
Target the forums where your customer asks for help
Look for the places where your persona asks questions, on top of the community for your category. For an invoicing SaaS aimed at freelancers, the freelancers subreddit matters more than the invoicing software one.
Cover four families of queries: words that describe the pain, tool switching ("switched", "alternative", "instead of"), hesitation ("I tried but", "what stopped me") and reviews of competing tools.
One line per source and per query
A real line from the October 12, 2026 plan:
{
"id": "reddit-startups-warm-intro",
"source": "reddit",
"why": "access to investors through introductions",
"args": {"subreddit": "startups", "query": "warm intro", "kind": "comments"},
"limit": 12
}
The why field forces Claude to justify every query. Query words never count as market language: compute_signals.py removes them from the phrase bank, otherwise your own keywords would show up in the report.
A 90-day window and caps
The default window covers 90 days, and you only keep quotes dated inside it. Three depth settings set the per-query caps: quick (10 posts, 2 pages), standard (25 posts, 3 pages), deep (50 posts, 5 pages). A capped collector keeps the most recent posts. The report then shows the volume as a minimum (lower_bound) and the trend as unknown.
Step 2: collect through APIs instead of scraping
Run the collection with one command: one process per query, sources in parallel, queries for the same source one after another.
python3 scripts/collect.py --plan $RUN/plan.json --out $RUN
The five free sources
| Source | Access | Limit to know |
|---|---|---|
| Arctic Shift archive | a few hours to a few days behind, deleted text missing | |
| Hacker News | Algolia API | no public score on comments |
| Stack Exchange | official API | 300 requests per day per IP, several sites closed |
| Discourse forums | the forum's /search.json |
keyword search, lots of noise |
| App Store | review RSS feed | no date filter, at most 10 pages of 50 reviews |
Reddit goes through the public Arctic Shift archive, Hacker News through the Algolia search API. For G2 or Capterra reviews, export a CSV yourself (url,date,text at minimum) and import it:
python3 scripts/import_records.py --input g2-export.csv --format csv --source g2
Sources blocked from a server
Reddit returns 403 to cloud and datacenter IP addresses, for JSON and RSS alike. We saw it again on October 12, 2026 from our own server. G2, Capterra, Trustpilot and Product Hunt also return 403 to a curl. On a remote server, a badly built agent then hands you a thinner report and flags nothing.
Two other traps come up often. Arctic Shift replies "Timeout. Maybe slow down a bit" to heavy queries on large subreddits, so keep one query per job and small caps. The Stack Exchange startups site no longer exists and returns a 400 error.
Check each source's status before reading a single review
The collector writes a status for each query in health.json: ok, empty, error or quota. Open this file before the corpus.
Treat an empty status as a warning: an empty source doesn't prove the market is silent. Change the wording or the community once, and if the source stays empty, log it in sources-rejected.md with the date and the reason. On October 12, the App Store review feed came back empty, as it does from time to time. Stop after two collection waves and list what's missing in the report's limitations.
Step 3: have Claude label every record
A corpus of a few hundred posts fits in Claude's context. It reads everything before naming a single pain point, then labels each post.
The pain codebook
Claude first writes the list of pain points in analysis.json, each with a stable ID (pain_investor_access), a one-sentence description with no numbers, a JTBD force (push, pull, anxiety, habit) and an awareness stage. Split two pains when people would act differently on each. Merge them when the same people say both in the same sentence.
The labeling prompt
The labeling instructions come down to a few rules, taken from the agent's skill:
Read every record in corpus.jsonl, then write one line per
record in labels.jsonl:
- relevant: true only if the post is about the market under study.
- speaker: user (talks about their own situation), practitioner (advice
from someone who has done it), vendor (selling their product), press, other.
- pains: only the pains the post expresses (a passing mention doesn't
count).
- sentiment, forces, awareness, competitors (named tools and status quo,
always spelled the same way).
Write no counts, no shares, no trends.
A real label from October 12 (ID masked): {"record_id": "hn:…", "relevant": true, "speaker": "vendor", "note": "tool launch: warm intro investor map"}. This post is relevant, but its author is promoting their own tool, so it doesn't count as a voice. Only user and practitioner count.
False positives to exclude
In a September 2026 study on fundraising, we coded 655 posts: 432 relevant, 216 off-topic, 7 duplicates. The off-topic posts came from keyword collisions: "VC" meaning voice chat, "series" meaning TV series, video game threads. On October 12, the six posts from a no-code forum surfaced by "funding" had nothing to do with fundraising, and Claude marked them all irrelevant.
Never edit corpus.jsonl by hand: a wrong post drops out through its relevant: false label, and your decision stays on record.
Step 4: let a script compute the numbers
Claude writes no numbers. You run:
python3 scripts/compute_signals.py --corpus $RUN/corpus.jsonl \
--labels $RUN/labels.jsonl --analysis $RUN/analysis.json --out $RUN/stats.json
For each pain point you get distinct authors, communities, share by source, earliest and latest dates, trend and escalation.
The threshold: 5 authors across 2 communities
A pain point makes it into the conclusions once it reaches 5 distinct authors across at least 2 communities, and an author who posts five times counts once. If you change the threshold (--min-authors, --min-communities), keep the same value from one run to the next.
In the October 12 run, two pain points clear the threshold: not knowing how much traction is enough for pre-seed (10 authors, 3 communities) and reaching investors without a network (9 authors, 2 communities). Three others stay as weak signals, at 4, 4 and 3 authors, shown separately. The report also lists one rejected pain: "warm intros" for finding customers or hiring, which uses the same words for a different market.
Capped volumes and unknown trends
A capped collector kept the recent posts and dropped the older ones, so comparing the two halves of the window would skew the trend. That's why it shows unknown. On October 12, five of the nine queries hit their cap, on purpose, to keep the example light. For volumes, switch to deep.
Look at the split by source too. On October 12, Reddit made up 60% of posts and 88% of counted voices. A pain point carried by a single family of sources deserves a second wave elsewhere before you trust it.
Step 5: verify every quote word for word
Claude picks the quotes, and a script finds each one in the corpus.
Cite by ID and passage
A quote is written with the post's ID (masked here) and a passage copied exactly:
{
"quote_id": "q_no_interviews",
"record_id": "reddit:t1_…",
"passage": "My start-up has a real product with real revenue and I am having trouble even getting VC interviews.",
"pain_id": "pain_investor_access",
"usable_as": ["headline", "hook"]
}
The passage keeps the original's capitalization, punctuation and typos. No ellipses, no [...]: a long post becomes several quotes. Quotes stay in their original language, and you translate them when you use them.
python3 scripts/verify_quotes.py --corpus $RUN/corpus.jsonl --quotes $RUN/analysis.json
The ten blocking reasons
The script tolerates three differences: whitespace, straight or curly quotation marks, and Unicode normalization. Everything else gets blocked:
no-recordandrecord-not-in-corpus: the quote has no source in the corpus.not-found: the passage doesn't exist in the post.case-changed: the passage only matches if you ignore capitalization.elided: the passage stitches two fragments together with an ellipsis.out-of-windowandno-date: the post falls outside the window or has no date.no-url,removed,metadata-mismatch: no URL, deleted text, or a URL and date that differ from the post's.
Fix a block with a different passage or a different post, never by editing the corpus, and drop the quote after three unsuccessful rounds. You can quote a vendor or a journalist as long as you present them as such, never as the voice of your customer.
Step 6: ship a report nobody touches by hand
python3 scripts/build_handoff.py --run $RUN
The script validates the files, recomputes stats.json, re-verifies the quotes and looks for bare numbers in Claude's prose. If anything is wrong, it prints every reason and writes no file: fix analysis.json or labels.jsonl, then run it again.
The report contains source status, the pain points with their counts, verified quotes with date and link, objections, a persona described by a job title (never an invented person) and recommendations, each paired with the observation that would disprove it. Your copywriting agents then read the JSON handoff file. To turn these pain points into positioning, our product market fit guide covers the next steps, and the one on your first 10 customers shows how to reach out to them.
In monitoring mode, you rerun the same plan at regular intervals: diff_runs.py lists the pain points that are new, rising, falling or gone since the previous run.
Market research with ChatGPT: the rules to keep
You can run market research with ChatGPT if you give it the corpus and take the numbers away from it. In the browser, ChatGPT can't run your local scripts, so you collect and verify separately and hand it the reading.
A market research prompt for ChatGPT
Paste your posts with an ID, a date and a URL on each line, then give it these instructions:
Here are N public posts, one per line: [id] date url text.
1. Classify each post: relevant or not to [market], and who is speaking
(user, practitioner, vendor, press).
2. List the pain points expressed, each with the ids of its posts.
3. For each pain point, give at most three quotes in the format
[id] "passage copied without any change".
Write no percentages and no totals. Do not rephrase any quote.
Then count distinct authors per pain point in a spreadsheet and search for each passage in your files with Ctrl+F. That check takes a long time by hand, so we gave it to a script.
ChatGPT, Claude.ai or Claude Code
ChatGPT and Claude.ai are enough to label a corpus you already have. Claude Code, Codex CLI, Cursor or Gemini CLI can also run the collectors and the verification in the same session, and the market-signal agent works on all four. A chatbot's web search leaves you no dated file to verify.
Build a Claude skill for market research
A Claude Code skill is a folder containing a SKILL.md file: a YAML header that tells Claude when to use it, followed by the instructions. Put it in .claude/skills/<name>/ for a single project or in ~/.claude/skills/<name>/ for all your projects, and call it with /name. The official Claude Code skills documentation covers the options, and our Claude Code skills guide shows how to organize them.
The market-signal agent splits its work into four skills, one for each step where the model is involved:
skills/
scanning-market-signals/SKILL.md # plan + collection (steps 1 and 2)
coding-the-corpus/SKILL.md # codebook + labels + counting
extracting-psychographic-profile/SKILL.md # quotes, persona, angles
watching-the-market/SKILL.md # monitoring mode
The labeling skill's header looks like this:
---
name: coding-the-corpus
description: Reads every record of the market-signal corpus and labels it,
so scripts can count. [...] The model never writes a count.
allowed-tools: Read Write Bash
---
Keep this split in your own skills: the instructions tell Claude what to read and which file to write, and every number comes out of a command.
Pitfalls from our ten studies
Between March and September 2026, we ran ten studies with an older agent: the model counted mentions and picked 15 to 20 quotes per stream from Exa searches. Their mistakes shaped the method above.
- Estimated percentages. In June 2026, a study for a functional beverage brand labeled its sentiment column "Estimated percentage", with no count behind it.
- A sample built to find unhappy people. In August, a study on AI agent tools counted 80 negative observations out of 108. That corpus didn't measure customer satisfaction, and every report should state this limitation.
- The sample versus the public distribution. For a food e-shop in September, hand counts on 70 reviews painted a softer picture than the public distribution of ratings on Trustpilot. Always compare your sample with the full distribution when it exists.
- Exa without Reddit or X in September 2026. By then, Exa no longer indexed Reddit and had removed its tweet category: the 43 "tweets" in one study were all LinkedIn posts.
- The year in the query. Queries containing "2026" inflated the share of 2026 posts and skewed the trend. Read trends on Reddit and Hacker News, which date every post.
- Keyword collisions. "VC", "series" or "funding" pull in off-market noise, so plan for the
relevant: falselabel from the first pass.
One last habit applies to every study: fifteen good quotes beat thirty weak ones.
The market-signal agent's code is open source and available to download from fog-agents.com. Our article on fog and its AI marketing agents covers the rest of the stack.
FAQ
What is the best AI for market research?
Pick a tool that can run scripts and read files: Claude Code, Codex CLI, Cursor or Gemini CLI. In it, you collect posts through APIs, have them labeled, then run the counting and verification scripts. A chatbot on its own is good for labeling a corpus you provide, never for producing the numbers.
Does a Claude prompt work for market research?
A prompt on its own produces quotes and percentages you can't verify. In our April 2026 run, built on a model-written synthesis, 1 quote in 12 passed word-for-word verification. A prompt becomes effective when it works on a stored corpus, with quotes cited by ID and numbers computed by script.
Can you do free market research with AI?
Yes. The market-signal agent's collectors use public APIs with no key (Hacker News Algolia, Arctic Shift, Stack Exchange, Discourse, App Store feed) and the scripts run on standard Python. You only pay for the subscription to the AI tool that reads the corpus.
Can you quote Reddit posts in market research?
Yes, as long as they are public, dated and quoted word for word. Keep the passage in its original language, never attribute it to a person or a company, and don't present it as a testimonial about your product. In a public document, remove the username and the link to the thread.
What are the 3 types of market research?
The usual split is desk research (existing data: reports, statistics, articles), qualitative research (interviews, discussions, verbatim quotes) and quantitative research (surveys and counts on a sample). The method in this tutorial is qualitative. It counts distinct authors and doesn't measure market size.
How many posts do you need for reliable market research?
Count distinct authors rather than posts. A pain point makes it into our conclusions once it reaches 5 authors across 2 communities. On October 12, 2026, 65 posts collected over 91 days were enough to surface two pain points.
