
To A/B test a cold call opener properly, change one thing, count conversations instead of dials, and decide the sample size before you start. With realistic meeting rates, you need roughly 225 conversations per version to confirm a large difference, and several thousand to confirm a small one. Most opener "tests" stop at 30 calls and crown a winner that was just a good Tuesday.
A rep tries a new first line on Monday, books two meetings, and the team adopts it on Tuesday. That is not a test. It is an anecdote with a spreadsheet.
Three things usually went wrong:
The fix is not complicated. It just requires deciding the rules before the first call, not after the first good day.
An opener has four parts you can change: who you say you are, the reason for the call, the question you ask, and the length. Change one.
And change it enough to matter. "Hi Marc, this is Julie" against "Hello Marc, Julie here" is not worth a month of calls. The differences that show up in data are structural.
The clearest example in our benchmark: 65% of reps open by explaining what they sell. Trigger-based openers, built on a reason specific to the prospect, reach a 7-10% meeting rate against 2-3% for pitch-first openers. That is a gap you can detect. A change of adjective is not.
Good candidates for a first test:
This is the part teams skip, and it is simple arithmetic.
Take a pitch-first opener at about 2.5% meetings per conversation and a trigger-first opener at about 8.5%, the middle of each range in our benchmark. A standard two-proportion sample-size calculation, at 95% confidence and 80% power, gives roughly 225 conversations per version. Call it 450 in total.
Now take a subtle change, where you hope to move from 5% to 7%. The same calculation gives about 2,200 conversations per version. More than 4,000 in total, for a two-point gain you may never be sure of.
Two lessons follow.
Test big differences. If you cannot imagine the new opener doubling the result, it is not worth a formal test. Just listen to ten calls and decide.
Translate conversations into dials. At an 11.3% connect rate, 450 conversations is about 4,000 dials. A rep dialing manually who gets four or five conversations a day needs around 100 rep-days. A team of four SDRs at 20 conversations a day each gets there in about six working days.
Volume is what makes testing possible at all. A team with few conversations cannot test openers. It can only have opinions about them.
The opener is rarely the biggest variable in a call block. The time slot and the persona are.
In our benchmark, connect rates run at 15.8% at 4pm, 14% at 10am and 9.8% at 8am. Founders pick up more around 10am, marketing around 4pm, HR around 1pm. People who answer at their natural slot are also in a better mood to talk.
If version A runs in the morning and version B in the afternoon, you tested the clock.
Rules that keep it clean:
Three numbers, all divided by conversations where the opener was actually delivered.
Count separately the pickups where the prospect hung up before you finished your name. They did not hear either version, and they should not sit in either denominator.
Leave connect rate out of the verdict. The opener is spoken after the pickup, so it cannot change who picks up. Connect rate is an input, not an outcome. If it differs between the two versions, your split was not random, and that is worth knowing.
Write the stopping rule before the first dial: "We stop at 225 conversations per version, or on Friday the 20th, whichever comes first." Then follow it.
Stopping early because one version is ahead is the most common way to fool yourself. Early leads reverse all the time on small numbers.
When the test ends, do three checks before announcing anything:
Three outcomes are acceptable: adopt the new version, keep the old one, or declare the test inconclusive and try a bigger difference. "Inconclusive" is a real result. It stops the team from arguing about a change that does not matter.
Keep a one-page note for each test: the question, the list, the two versions word for word, the numbers, the decision. In six months it is the only thing that prevents you from running the same test again.
The bottleneck in opener testing is conversations, and that is the number Sonik is built to move. Parallel dialing gives reps about five times more conversations, which turns a test that would take a quarter into one that takes a week or two.

Lists come from the signals already in your CRM, so a trigger-first version has a real trigger for every contact and both halves of the split come from the same source. Sessions run by persona at the slot where that persona picks up, which keeps the clock out of the comparison.
After the session, AI coaching works on the recorded conversations, so the "listen to ten calls per version" step is a review, not a search. If you want the baseline numbers, our article on the real cold call connect rate has them.
A fair opener test is one variable, one randomly split list, every rep on both versions, and a sample size written down before the first call. Test structural differences, such as trigger-first against pitch-first, and plan for about 225 conversations per version. This week, count how many conversations your team had last month. That number tells you how many opener tests you can honestly run per quarter.