Documentation
Crowdflow backtests cold-outreach pitches against a simulated crowd of personas. You give it one sentence (audience + offer + channel); it writes a brief, casts 9 persona profiles, walks every persona through your pitch step by step, and scores the whole run - then renders an interactive HTML report you can open in your browser and interrogate with an AI chat.
It is provider-agnostic: it talks to any OpenAI-compatible API (/chat/completions), so it works out of the box with DeepSeek, OpenAI, Groq, Ollama, vLLM, LM Studio, and more - just point it at a different --base-url and --model.
01How it works
The tool models a simulated outbound campaign end to end:
1 sentence
│
▼
BRIEF one sentence is expanded into a structured brief
│ (audience, offer, channel, goals, realism budget)
▼
PERSONAS 9 (default) persona profiles are cast to sample the audience
│
▼
SIMULATE every persona walks through the pitch step by step:
│ it reacts, raises objections, reasons, and decides
│ (open? read? convert? drop? which objection?)
▼
REPORT aggregate, score, diagnose; render threads.html + report.json
│
▼
CHAT ask the model questions about the run, grounded in the snapshotEverything is deterministic in structure and stochastic in behavior: you get a fresh, seeded cast every time, and the outcome distribution is your answer.
02Requirements
| Requirement | Minimum |
|---|---|
| Python | 3.9+ (tested on 3.12.7) |
| Runtime dependency | requests>=2.28 (installed automatically) |
| Operating system | Windows / macOS / Linux (OS-independent) |
| API | Any OpenAI-compatible endpoint, or a local one (Ollama, vLLM, LM Studio) |
| API key | Required for hosted providers; not needed for local servers |
The key is never stored in the code, the repo, or the distribution zip.
03Installation
The product is a single universal source zip that works identically on Windows, macOS and Linux - any machine with Python 3.9+. There is no installer to run and nothing OS-specific to download: one file, every OS.
3.1 Install (all OSes)
pip install crowdflow-cli-<version>.zip # or: extract, then pip install .
crowdflow configure # store API key / base URL / model
crowdflow doctor # verify everything worksconfigure prompts for the API key, base URL and model, then tests the connection. A buyer-facing guide ships inside every zip as INSTALL.txt.
3.2 Developer install from source
pip install . # from this directory
pip install -e . # editable
crowdflow --versionOr run in place without installing: python -m crowdflow_cli (or the crowdflow.bat launcher).
3.3 Verify
crowdflow --version # → crowdflow 0.2.0
crowdflow doctor # full connectivity + config diagnosis3.4 Notes
- The zip contains only the product:
pyproject.toml,MANIFEST.in,README.md,LICENSE,INSTALL.txt, and thecrowdflow_clipackage. - No
.env, noconfig.json, noruns/, no build junk - the zip is exactly the product. - The API key is never inside the zip; it lives in the user config file (§5.2).
04Quick start
# 1. Store your API key (interactive prompt, connection test before saving)
crowdflow configure
# 2. Run a full backtest with one sentence
crowdflow run "Founders of seed-stage B2B SaaS with no sales team — offer a no-code
cold outreach engine that writes 500 personalized first emails an hour"
# or the shorthand — the bare first positional auto-detects a one-shot run:
crowdflow "Founders of seed-stage B2B SaaS — no-code cold email engine, 500/hr"
# 3. Open the HTML report (report already hosts automatically when you have a TTY)
crowdflow serve runs/<slug>
# 4. Ask questions about the run
crowdflow chat runs/<slug>
# 5. Everything about the effective setup in one glance
crowdflow doctorAfter a run finishes you get:
runs/<slug>/threads.html- the interactive report (opens automatically)runs/<slug>/report.json- machine-readable resultsruns/<slug>/reactions.jsonl- every persona's step-by-step decisions
05Configuration
5.1 Resolution order (highest to lowest)
1. command-line flags --key --base-url --model (per-command)
2. environment variables CROWDFLOW_API_KEY / CROWDFLOW_BASE_URL / CROWDFLOW_MODEL
3. ./.env in the current dir
4. user config file (platform user config dir, written by `crowdflow configure`)
5. built-in defaults base_url=https://api.deepseek.com/v1, model=deepseek-chatSpecial overrides:
| Variable / flag | What it overrides |
|---|---|
CROWDFLOW_CONFIG or --config PATH | the user config file location |
CROWDFLOW_SETTINGS or --settings PATH | the settings config.json location |
CROWDFLOW_CONFIG and --config are read/write: crowdflow configure writes to that path too.
5.2 Where things live per platform
| Platform | User config file | User config dir | Log file |
|---|---|---|---|
| Windows | %APPDATA%\crowdflow\config | %APPDATA%\crowdflow | %APPDATA%\crowdflow\logs\crowdflow.log |
| macOS | ~/Library/Application Support/crowdflow/config | ~/Library/Application Support/crowdflow | ~/Library/Application Support/crowdflow/logs/crowdflow.log |
| Linux | $XDG_CONFIG_HOME/crowdflow/config (or ~/.config/crowdflow/config) | same | $XDG_CONFIG_HOME/crowdflow/logs/crowdflow.log |
The user config file is a plain .env-style file (KEY=value lines) written by configure. It is git-ignored and excluded from the distribution zip.
5.3 crowdflow configure
crowdflow configure # interactive: prompt for key/base-url/model, test, save
crowdflow configure --key sk-... # non-interactive
crowdflow configure --base-url https://api.openai.com/v1 --model gpt-4o
crowdflow configure --local # write ./.env instead of the user config file
crowdflow configure --no-test # skip the connection test before saving
crowdflow configure --show # show the effective config
crowdflow configure --unset # remove the stored configsetupis an alias ofconfigure(minus--show/--unset).--localis great for project-scoped keys that shouldn't live in the user config.- A connection test (a tiny model call) runs before saving unless
--no-testis given.
5.4 crowdflow config
crowdflow config # print effective config (base URL, model, masked key, source)
crowdflow config --json # machine-readable outputconfig_source() reports where the effective key came from: environment, ./.env, the user config file path, or defaults.
5.5 crowdflow doctor
crowdflow doctor # human-readable diagnosis
crowdflow doctor --json # machine-readableChecks, in order:
- Python version and platform
- Installed version
- Config resolution (what file / env is in effect)
- Connectivity to the base URL (a minimal model call)
- Settings file load (reports any warnings from
config.json)
5.6 Pipeline settings vs API config - don't confuse them
| API config | Pipeline settings | |
|---|---|---|
| What | key, base URL, model | personas count, temperatures, realism, parallelism |
| Where | user config file / .env / env vars | config.json |
| Written by | crowdflow configure | crowdflow init (scaffold) |
| Read by | every command | every run-producing command |
06Command reference
Global options (available before/after the subcommand where relevant):
| Option | Meaning |
|---|---|
--version | print version and exit |
--runs-dir PATH | where runs live (default: settings.runs_dir, or runs/) |
--settings PATH | path to a settings config.json (env: CROWDFLOW_SETTINGS) |
--config PATH | path to the API config file (env: CROWDFLOW_CONFIG) |
--color auto|always|never | force terminal color handling (default: auto) |
-v, --verbose | debug logging to stderr and the log file |
Common per-command flags (--key, --base-url, --model) - shorthand for one-off overrides on API commands.
Full command list
| Command | Purpose |
|---|---|
init | scaffold a settings config.json |
configure / setup | set the API key, base URL, model |
config | show the effective API configuration |
doctor | diagnose config, connectivity, install |
new | expand one sentence into a run brief |
personas | cast the audience for a run |
simulate | walk each persona through the pitch |
report | score the run and render threads.html |
serve | host threads.html locally and open the browser |
chat | talk to the model about a run |
compare | diff the objection maps of two runs |
run | full pipeline: brief - personas - simulate - report |
(hidden) oneshot | full pipeline with a bare positional - see §7 |
6.1 init
crowdflow init # scaffold ./config.json
crowdflow init --force # overwrite an existing config.jsonWrites a fully-commented copy of the default settings (schema_text()) you can edit. Skip warnings on init so a scaffolded file never fails.
6.2 new
crowdflow new "SENTENCE" --slug my-run- Expands one sentence (audience + offer + channel) into a full brief via the model.
- Writes
runs/<slug>/brief.json. --slugnames the run directory; otherwise one is generated from the sentence.
6.3 personas
crowdflow personas runs/<slug> # or by slug: crowdflow personas <slug>
crowdflow personas runs/<slug> --overwrite- Casts
settings.personas.countpersonas for the run and writespersonas.json. --overwriteregenerates (and discards) existing personas.
6.4 simulate
crowdflow simulate runs/<slug>
crowdflow simulate runs/<slug> --parallel 8
crowdflow simulate runs/<slug> --overwrite- Walks each persona through the pitch step by step; writes
reactions.jsonl. --parallel Noverridessettings.simulation.parallel(personas in flight simultaneously).
6.5 report
crowdflow report runs/<slug>
crowdflow report runs/<slug> --full
crowdflow report runs/<slug> --no-host- Aggregates, scores, and diagnoses the run; writes
report.jsonand rendersthreads.html. - Default output is a compact dashboard (~13 lines).
--fullalso prints the long text report. - Hosts
threads.htmlautomatically (browser opens on a TTY) unless--no-host.
6.6 serve
crowdflow serve runs/<slug>
crowdflow serve runs/<slug> --port 8901
crowdflow serve runs/<slug> --port 8901 --no-browser- Hosts
threads.htmlon127.0.0.1and opens the browser. --port 0(default) auto-assigns a free port.--no-browserprints the URL only. Press Ctrl-C to stop hosting.
6.7 chat
crowdflow chat runs/<slug> # interactive REPL
crowdflow chat runs/<slug> "Which objection was the biggest blocker?"
crowdflow chat runs/<slug> --temperature 0.7 --max-tokens 3000- One-off mode: pass a
questionpositional; prints one answer and exits. - Interactive mode: type questions until you type
exit/quitor press Ctrl-C. Empty lines are skipped. --temperature(default 0.4),--max-tokens(default 2000).- The model is grounded in a compact snapshot of the run (§11) and must cite it verbatim rather than invent.
6.8 compare
crowdflow compare runs/run-a runs/run-bDiffs the objection maps of two runs: which objections appear in both, which are unique to each, and how impact/causes compare.
6.9 run and the hidden oneshot
crowdflow run "SENTENCE" # new run from a sentence
crowdflow run runs/<slug> # continue an existing run (skip done stages)
crowdflow run -b brief.json # start from an existing brief
crowdflow run "SENTENCE" --slug my-run --parallel 8 --full --no-host
crowdflow "SENTENCE" # shorthand for `run` via hidden oneshotThe run command runs the whole pipeline and is the one you'll use daily. See §7 for stage-by-stage behavior, including --overwrite semantics.
07The full pipeline (run / oneshot)
run chains four stages with smart "continue" behavior:
sentence
│ (or -b brief.json, or an existing run dir)
▼
STAGE 1: BRIEF expand sentence → brief.json [skip if exists]
▼
STAGE 2: PERSONAS cast N personas → personas.json [skip if exists]
▼
STAGE 3: SIMULATE walk each persona → reactions.jsonl [skip if exists]
▼
STAGE 4: REPORT score + threads.html + report.json [always re-run]
▼
auto-host threads.html (TTY only, unless --no-host)- Positional
targetdisambiguation: if it names an existing run dir (or slug) inside--runs-dir, the pipeline continues that run; otherwise it's treated as a one-sentence description. -b/--brief brief.jsonstarts from a hand-written brief instead of a sentence.--slugsets the run directory name for a new run.--overwriteredoes the current stage even if it exists (it does not wipe later stages).--parallel Noverridessettings.simulation.parallel.--fullprints the long text report;--no-hostskips hosting.- If no sentence, run dir, or
-bis given:LLMError("give a sentence, an existing run, or -b brief.json"). - A best-effort
brief.jsonis written before the brief stage raises on final failure, so you can retry with-b brief.json.
The bare shorthand (crowdflow "..." / crowdflow runs/<dir>) is implemented by _pre_scan_argv, which transparently injects the hidden oneshot subcommand. It honors global flags (-v, --color, --runs-dir, --settings, --config, including --opt=val forms) placed before the positional.
08Runs directory anatomy
Runs live in --runs-dir (default runs/). Each run is a folder named after its slug:
runs/
└── game-port-email-v1/
├── brief.json one-sentence expansion into a structured brief
├── personas.json the cast of personas (count, axes, traits)
├── reactions.jsonl one JSON line per persona — the full step-by-step
├── report.json aggregate scoring + diagnosis (§9)
└── threads.html interactive HTML report (§10)reactions.jsonl - one line per persona
Each row describes a persona's full journey:
| Field | Meaning |
|---|---|
persona | profile fields (name, title, company, context) |
mood | emotional temperature at conversion |
steps | array of step records: sent, objection text/cause/theme, reasoning, internal_thoughts, converted, and dropped-at-step markers |
This is the raw material for the report and for chat grounding.
09report.json reference
report writes a fully-structured result. Verified fields (from the reference run runs/game-port-email-v1):
| Field | Type | Meaning |
|---|---|---|
n | int | number of personas simulated |
n_steps | int | total pitch steps |
mode | string | run mode |
goal | string | campaign goal |
conversion_rate | float 0-1 | measured conversions |
expected_conversion_rate | float 0-1 | baseline you'd expect from the realism budget |
objections | array | each with label, verbatim, impact, cause, persona_count, personas |
cause_coverage | map | how many personas hit each cause (TRUST/VALUE/PRICE/FIT/INERTIA/CHANNEL/OTHER) |
worst_objection | object | the single most damaging objection |
leak | map | where the funnel leaks (opened/replied/read rates per stage) |
realism | map | opened, replied, open_rate, reply_rate vs. the realism budget |
diagnosis | array | each item is a finding with a kind (e.g. rejected-verbatim, leak, weak-close) |
top_convince | array | what actually converted people |
nothing_would_convince | array | personas nothing would convince |
headline | map | summary (one line) + do_this_first (the highest-leverage action) |
personas | array | per-persona outcomes: engagement, purchase_readiness, dropped_at_step, objections |
10threads.html - the interactive report
Rendered by the scoring/threads.py script. Pure static HTML/CSS/SVG (no server-side app - any static host serves it).
Visual language
- Dark-only theme. CSS custom properties in
:root:--bg #0e0d0c,--fg #f7f5f1,--dim #b8b3aa,--faint #847f76--line #2c2926,--rule #221f1c--accent #f0972f(amber), plus--glowand--tintderived accents
- Engagement palette: engaged
#4ade80, skimming#d8b26a, tuned_out#6f6b62(hollow), left#e0724a, blocked#e5484d. - Cause colors: TRUST amber, VALUE teal, PRICE rose, FIT periwinkle, INERTIA green, CHANNEL blue, OTHER gray.
- Typography: system font stack, 720px centered
.wrap, uppercase kickers with.16-.28emletter-spacing, h2 bottom rules with a 32px amber::aftersegment. - Stats strip across the top; amber verdict card with layered gradients and glows.
- Collapsible
<details>/<summary>sections, flex rows, orange/amber-rule callouts.
The SVG objection map
- 880×N viewBox; x = purchase readiness, y = layout row.
- Circles = people, squares = objections (hollow = the persona left anyway).
- KILLER objections get a thick stroke.
- Canvas-measured labels, deterministic collision separation, hover focus isolation (
opacity .13s). - Nodes are draggable; click-to-scroll anchors jump to the objecting persona's thread.
11Chat with a run
crowdflow chat runs/<slug> [question] builds a compact grounded snapshot (~7 KB) from brief.json + report.json containing:
- the brief
- the conversion line (measured vs expected)
headline.summaryandheadline.do_this_first- the verdict
- the leak summary
- root causes
- the full objection list with verbatims
top_convinceand per-persona outcomes
The CHAT_SYSTEM prompt (prompts.py) enforces:
- The snapshot is the only source of truth - no outside knowledge.
- Answer every claim with a verbatim citation from the snapshot.
- If the run doesn't record something: "the run doesn't record that" - never guess.
- No invented proof, no fabricated metrics.
Example session:
$ crowdflow chat runs/game-port-email-v1
> Why did nobody reply?
> What is the strongest single fix?
> exit12Comparing two runs
crowdflow compare runs/run-a runs/run-bOutputs a shared-vs-unique objection breakdown and per-run impact/cause deltas - useful for A/B testing sentences, offers, or channels against the same cast logic.
13Settings reference (config.json)
Loaded from (highest to lowest): --settings PATH / CROWDFLOW_SETTINGS - ./config.json - <user config dir>/config.json - built-in defaults. Invalid values are skipped with a warning; out-of-range values are clamped with a warning. Files may carry a BOM (utf-8-sig tolerant).
Defaults and bounds
| Section | Key | Default | Allowed | Notes |
|---|---|---|---|---|
brief | temperature | 0.6 | 0-2 | brief-writing creativity |
brief | max_tokens | 4000 | 100-200000 | - |
brief | repair_attempts | 5 | 1-10 | retries when the brief contains placeholders |
personas | count | 9 | 1-40 | personas per run |
personas | temperature | 1.0 | 0-2 | cast creativity |
personas | max_tokens | 8000 | 100-200000 | - |
personas | attention_range | [0.2, 0.6] | two values in 0-1 | - |
personas | axes | [budget, incumbent, authority, burned, unaware, channel, timing, fit] | non-empty subset of allowed axes | diversity axes along which the cast varies |
simulation | temperature | 0.9 | 0-2 | - |
simulation | max_tokens | 6000 | 100-200000 | - |
simulation | parallel | 1 | 1-64 | personas in flight simultaneously |
realism | email | open [0.20,0.50], reply [0.00,0.15] | two values in 0-1 each | the realism budget for email channel |
realism | dm | open [0.45,0.90], reply [0.02,0.25] | two values in 0-1 each | realism budget for DM channel |
runs_dir | - | "runs" | non-empty string | where runs are written |
Allowed axes: budget, incumbent, authority, burned, unaware, channel, timing, fit.
Example config.json:
{
"brief": {
"temperature": 0.6,
"max_tokens": 4000,
"repair_attempts": 5
},
"personas": {
"count": 12,
"temperature": 1.0,
"max_tokens": 8000,
"attention_range": [0.2, 0.6],
"axes": ["budget", "incumbent", "authority", "burned", "unaware",
"channel", "timing", "fit"]
},
"simulation": {
"temperature": 0.9,
"max_tokens": 6000,
"parallel": 1
},
"realism": {
"email": { "open": [0.20, 0.50], "reply": [0.00, 0.15] },
"dm": { "open": [0.45, 0.90], "reply": [0.02, 0.25] }
},
"runs_dir": "runs"
}crowdflow init scaffolds exactly this for you.
14Terminal output & colors
--color auto|always|never(defaultauto).autodisables color when output is a pipe,NO_COLORis set,CLICOLOR=0, orTERM=dumb.- Unicode glyphs degrade gracefully on limited terminals.
term.Spinnerwrites to stderr so stdout stays clean for piping/parsing.- Helpers:
term.box,term.section,term.ok/err/warn/info/step. - Default
reportoutput is the compact dashboard; use--fullfor the long form.
15Hosting the report
report and run host threads.html automatically when stdout is a TTY (use --no-host to disable). serve is the explicit form:
ThreadingHTTPServerbound to127.0.0.1only - localhost, not LAN.- Quiet request handler (no log spam).
- Browser opens via
webbrowseronly when a TTY is present;--no-browserprints just the URL. - Prints the URL plus "press Ctrl-C to stop hosting"; Ctrl-C performs a clean shutdown.
Because threads.html is a fully static file, you can also open it directly, or host it on any static server:
python -m http.server 8901 --directory runs/my-run16Logging & troubleshooting
- With
-v: debug logging goes to stderr and the log file. - Without
-v: the log file still records INFO+ (errors, unexpected failures). - Log file:
<user config dir>/logs/crowdflow.log(§5.2). - On an unexpected error, the CLI prints the traceback location of the log file for follow-up.
Common troubleshooting:
| Symptom | Fix |
|---|---|
crowdflow doctor fails connectivity | Check CROWDFLOW_BASE_URL / --base-url; confirm the endpoint is OpenAI-compatible (/chat/completions) |
| Auth errors | Re-run crowdflow configure; confirm which source the key resolves from via crowdflow config |
| Settings warnings on every run | Check the config.json keys against §13; invalid keys are named in the warning |
| Brief keeps failing to repair | Lower the ambition of the sentence; or hand-write brief.json and run -b brief.json |
| No browser opens | Expected when output is not a TTY; use crowdflow serve <run> explicitly |
17Exit codes
| Code | Meaning |
|---|---|
0 | success |
1 | LLMError or unexpected failure (details logged) |
130 | interrupted by Ctrl-C (hosting stopped cleanly) |
18FAQ
What do I need to use Crowdflow?
Three things:
- A computer running Windows, macOS, or Linux with Python 3.9+ installed.
- The zip (download it with your redemption code).
- An AI assistant API key - Crowdflow uses any OpenAI-compatible provider. It runs on DeepSeek V4 Flash by default, a budget model: at typical usage $2-4 of credit lasts up to two weeks.
Then it's two commands:
pip install crowdflow-cli-0.2.0.zip
crowdflow configure # enter your API key, base URL and modelCan I reuse my redemption code?
Yes. Your code is reusable - you can return to the redemption page and re-download the zip any time, no limits. One code is tied to one purchase. Keep it private: anyone who gets it can also redeem it.
19Glossary
| Term | Meaning |
|---|---|
| Run | one full backtest - a slug directory under runs/ with brief, cast, reactions, report, HTML |
| Brief | structured expansion of the one-sentence pitch: audience, offer, channel, goals |
| Persona | a simulated member of the audience, cast along diversity axes |
| Reaction | a persona's step-by-step journey: opening, reading, objecting, converting or dropping |
| Objection | a reason a persona resists; grouped by cause and theme |
| Cause | the root attribution: TRUST, VALUE, PRICE, FIT, INERTIA, CHANNEL, OTHER |
| KILLER | an objection that stopped everyone who hit it |
| Leak | where the funnel loses people (open - read - reply - convert) |
| Realism budget | the expected open/reply rate range for the channel, used to sanity-check results |
| Verdict | overall run judgment: e.g. PROOF (proof-of-concept quality, not a conversion rate) |
| Conversions | personas that ended in purchase/intent |