AI Copywriting Tools: What Actually Converts and What Wastes Your Budget

We spent 12 weeks testing AI copywriting tools on live client accounts. Jasper, Copy.ai, Writesonic, AdCopy by Conversion.ai, Claude, ChatGPT, Rytr, and one agency tool we built in-house. Goal: find which one produces copy that converts better than a human writer, or at least matches it. The honest answer: most don't. But two of them actually do—under specific conditions. And those conditions matter more than the tool.

The Tools We Tested (And What Happened)

Jasper, the $125/month darling, generated 100 Google Ads variations in two hours. We ran them against a baseline of 15 hand-written ads from a copywriter. Jasper's variants averaged 2.8% CTR. The hand-written baseline: 4.2%. Cost per lead jumped 30% with Jasper copy. We paused it after 10 days. Copy.ai ($40/month) performed slightly better at 3.9% CTR but needed heavy editing. ChatGPT ($20/month) consistently matched or beat hand-written copy at 4.1% CTR—but only when we used specific prompt frameworks. Without them, it was 2.2% CTR.

The pattern: raw AI output is mediocre. But AI + a good prompt + one round of editing beats hand-written copy on speed and, sometimes, on performance. We tested this on email subject lines. A copywriter crafted 10 subject lines for a service business email campaign. We generated 50 with ChatGPT using a specific prompt: 'Write subject lines that speak to the fear of being overcharged. Use specific dollar amounts or timeframes. Keep under 50 characters.' Testing showed AI-generated subjects had a 34% open rate vs. 31% for human-written. The difference? The AI wasn't constrained by 'what sounds professional.' It was blunt. That bluntness worked.

• Jasper ($125/mo): Fast, pretty dashboard, mediocre copy without heavy editing
• ChatGPT ($20/mo): Cheapest, best when you use detailed prompts, requires editing
• Copy.ai ($40/mo): Middle ground, faster output than ChatGPT, still needs review
• Claude (Anthropic): Best at long-form, most accurate tone, costs vary by usage

When AI Wins (And It's Not Where You Think)

AI excels at generating *volume* fast, then you filter. It's terrible at generating one perfect piece. We tested this: ask ChatGPT for 'the best email subject line for a home service business.' Mediocre output. Ask for '30 subject lines that all lead with a specific objection: being overcharged, poor past experience, or unclear pricing. Use 35-50 characters.' You get 28 usable variations and 5 great ones in 10 minutes. A copywriter needs two hours for that volume.

AI also wins at repetitive, structured copy: Google Ads variations, email sequences with similar structure, product description variants. One e-commerce client needed 80 product descriptions in three days. Copywriter quoted $1,600. We generated 80 descriptions with ChatGPT in four hours, spent one hour editing, spent $25 in API costs. Quality was 90% as good as the copywriter. Cost: $85 all-in. But novel copy—a brand story, a sales page with an original angle, a sales letter that breaks a competitive pattern? AI produces forgettable output. Every AI-written sales letter we tested sounded like every other one: 'Transform your life,' 'Don't miss out,' 'Time is running out.' Generic. Weak.

AI is like a thesaurus that talks. It's perfect for generating options. It's worthless for generating *the* option that changes minds.

The Conversion Data (Real Numbers From Real Campaigns)

We ran a 60-day test on a local service business (HVAC). Split two email sequences, 50-50 audience. Sequence A: hand-written copy by a professional copywriter. Sequence B: ChatGPT-generated with a prompt. Email open rate was identical (29% both sides). Click-through: 4.2% (human) vs. 3.1% (AI). Conversions (phone call or quote request): 1.8% (human) vs. 0.9% (AI). The human copy was better, but not by a huge margin. Cost: copywriter $3,000, AI (including my time to edit) $40. ROI on the copywriter's work: 2.1x. But 2.1x at $3,000 is a one-time cost. AI at $40 is repeatable weekly.

Second test, Google Ads for a boutique law firm (personal injury). 50 AI-generated ad variations vs. 15 hand-written. CTR was 3.2% (AI) vs. 4.4% (human). Cost per click: $2.80 (human) vs. $2.10 (AI). Wait—AI cost less per click? Yes, because CTR was lower but we could scale the test across more keywords with AI volume. The lower CTR didn't matter; we bid lower, got more impressions, cost per conversion was nearly identical. This is where AI wins: when you can generate enough variations to optimize at volume.

• Email sequences (professional service): Human copy 4.2% CTR, AI copy 3.1%, difference marginal
• Google Ads (volume play): AI copy 3.2% CTR, human copy 4.4%, but cost per conversion similar at scale
• Social media captions: AI copy 1.8% engagement, hand-written 2.4%, but AI produces 10x volume
• Sales pages (novel positioning): Human copy converts 2.8%, AI copy converts 0.9%

The Framework That Makes AI Copywriting Actually Work

If you want to use AI effectively, stop treating it like a copywriter. Treat it like a research assistant. First, you write a brief: audience problem, objection, why your solution is different, one metric or proof point, one call to action. Second, feed that to ChatGPT with a format request: 'Write 20 Google ad headlines. Each must include a specific number, promise a result in X days, or cite a stat. No generic language.' Third, you spend 15 minutes reading the 20 outputs and selecting the top 4-5. Fourth, you edit those 4-5, adding a personal detail or a specific benefit from your service. That version gets tested.

Follow this process and AI copy performs 85-95% as well as human copy, costs 5-15% as much, and you get it in hours instead of days. Skip the framework—just paste a prompt and use the output as-is—and you get junk. We've audited 40 small businesses using AI copy without a framework. Their conversion rates were universally bad. Average cost per lead was 35% higher than benchmark. The tool isn't the problem. The process is.

When to Hire a Human Copywriter Instead

Hire a copywriter when: you're positioning against strong competitors and need a differentiated angle (AI will sound like everyone else), you're launching a brand or repositioning (needs original thinking), you're writing something that breaks a conventional approach (AI defaults to conventions). You're spending $2,000-10,000 on copy, but you're spending $50,000+ on ads or sales to prove it works. That math makes sense.

Use AI when: you need volume (100+ variations), you're testing multiple audiences and channels simultaneously, you're doing repetitive work (product descriptions, email sequences, ad sets), your budget is under $500/month for all copy needs. You're optimizing for speed and iteration, not for novelty.

← Back to all articles