Classifying cold email replies

How to sort cold email replies into interested, meeting, pricing, question, not now, not interested, wrong person and unsubscribe, and what to do when unsure.

Reply classification sorts each human reply by what the person wants, so warm replies get answered first and opt-outs are honoured. AY Inbox's inbox tagging labels a person's reply as Interested, Meeting, Pricing, Question, Update, Not now, Not interested, Wrong person, Unsubscribe or Legal threat. It leaves a reply unlabelled rather than guess.

A reply intent is what a reply asks of you: a call, a price, an answer, more time, or to be left alone. Reading it correctly decides who you answer first, which sequences stop, and which addresses are never mailed again. Getting it wrong in either direction costs something: a missed yes is lost revenue, and a missed "remove me" is a complaint.

First filter out what is not a person

Most of what lands after a campaign send is not a reply at all. Before any intent is read, each message gets one kind:

Kind	Label	Examples
Hard or soft bounce	Bounced	Address does not exist, mailbox full
Out-of-office	Out of office	Vacation and away messages
Automated acknowledgement	Auto-reply	Ticket receipts, "we got your message"
Notification	Notification	Sign-in codes, receipts, newsletters, platform notices
Cold inbound	Sales pitch	Someone pitching you, not answering your campaign
Internal	(none)	Your own team, or mail forwarded internally
Human reply	(none)	A real person responding to your outreach

Common senders and headers (no-reply addresses, bulk mail, out-of-office subjects, delivery failures) are decided offline, with no model call. Everything else is judged by the model. Mail no person wrote moves to the Automated view, except a notification that needs your action, which stays in the inbox labelled Action required.

Intent is read only when the kind is a human reply: an intent on a bounce would answer a question nobody asked. Out-of-office replies have their own handling, covered in out-of-office replies.

The reply intents

Label	Intent	Means
Interested	`agreed`	Agrees to the call, the partnership or the next step
Meeting	`scheduling`	Proposes, confirms or changes a specific time to meet, or asks for a call
Pricing	`wants_pricing`	Asks about price, terms or commercials
Question	`wants_info`	Open, but asking questions before deciding
Update	`in_progress`, `question_answered`	News on something already agreed, or the answer to something you asked
Not now	`not_now`	Open in principle, but the timing is wrong
Not interested	`not_interested`	Declines, without demanding removal
Wrong person	`wrong_person`	Not the right contact, or names someone else
Unsubscribe	`opt_out`	Asks to stop, complains or threatens
Legal threat	(a signal)	Threatens legal action or names a regulator

There is also unclear, for a reply whose intent genuinely cannot be read. It gets no intent label, because a wrong label is worse than none: the model is never forced to pick a bucket just to answer. The in_progress, scheduling and question_answered intents exist because a real inbox is mostly ongoing conversations, not first answers to cold mail.

Opt-outs do not wait for tagging. A reply that says "remove me" in so many words is suppressed by a phrase list read on every reply, whether or not tagging is on, unless the workspace turned off Honour replies that ask to stop; see handling opt-out replies.

What counts as positive

For deciding who to answer first, the actionable set is Interested, Meeting, Pricing and Question: each is a person asking you for something. The unibox's Hot leads view lists Interested, Meeting and Pricing; Declined lists Not interested, Wrong person and Unsubscribe.

Campaign reporting uses a coarser reading with three outcomes for a human reply: positive, negative or neutral. With tagging on, Interested, Meeting, Pricing, Question and Update read as positive, Not interested and Wrong person as negative, and Not now and unclear replies as neutral. Without tagging, offline phrase lists decide the clear cases and an optional model call the rest.

So, in AY Inbox, the positive reply rate is positive replies divided by emails sent, the same denominator as the reply rate. From the same number of sends, a campaign with 100 replies and 5 positive does worse than one with 60 replies and 20 positive. Negative replies, requests to stop, unclassified replies, out-of-office notices and autoresponders never count as positive. See analytics.

Confidence and Needs review

Every verdict carries a confidence between 0 and 1, and nothing acts on a low one:

If the kind is below 0.70, the thread is labelled Needs review, gets no other label and stays in the inbox.
If the kind is confident but the intent is below 0.70, the kind and relevance stay, the intent label is left off, and Needs review is added.
A verdict marked Needs review never holds, stops, suppresses or answers anything.

The floor is not caution for its own sake. A low-confidence answer is not reproducible: the same ambiguous message, classified three times, returned a different winning label each time while confidence stayed near 0.2. A person reading it once is cheaper than an action taken on a coin toss.

Where the verdict goes

One stored verdict feeds several features, so a reply is judged once:

The inbox. Labels, a 0 to 100 relevance score and the premade views in the unibox.
Campaign actions. Switches under Settings > Sending > Classified replies hold a contact who says Not now (30 days by default), stop one who declines, and open a task when someone asks for a call; those three are on by default. A fourth, off by default, suppresses a sender asking for removal at 0.80 or above. Only a confident human reply acts.
Auto-reply. Auto-reply answers only a confident human reply whose intent the campaign allows, at confidence of at least 0.80. See safe automatic replies.
Booking. A Meeting intent at 0.70 or above, or a clear request for a call, lets calendar booking offer open times.
Webhooks and automations. The campaign.reply_received event carries the verdict as ai_labels, ai_intent, ai_intent_confidence, ai_kind, ai_relevance and ai_needs_review. ai_intent and ai_kind are empty strings when tagging was not confident, and the keys are absent when tagging never judged the message. To be told about every interested lead, see the positive reply webhook recipe.

Languages

Tagging reads English, German, French, Spanish, Italian and Dutch out of the box. Add other languages under Settings > Sending > Tagging languages and questions, from a list of 44. Adding one names it to the classifier as a hint and adds that language's quoted-history markers and out-of-office subjects to the offline reading, so the classifier reads what the person wrote rather than your own message quoted under it. Leave off languages nobody writes to you in: each one adds patterns that could misread someone's mail.

Frequently asked questions

What counts as a positive reply in cold email?

A reply from a person who wants to move forward or learn more, such as agreeing to a call, proposing a time, asking about price or asking questions before deciding. Out-of-office notices, auto-replies, declines and requests to stop never count as positive.

How is positive reply rate calculated?

In AY Inbox, the positive rate divides the replies the classifier judged positive by the emails sent, the same denominator as the reply rate. That lets two campaigns compare on opportunities created rather than on raw replies.

What is the difference between "not now" and "not interested"?

"Not now" is open in principle but says the timing is wrong, so it is worth a later follow-up. "Not interested" declines without asking for removal, so the sequence should stop but the contact is not suppressed.

Can replies be classified automatically in languages other than English?

Yes. AY Inbox's inbox tagging reads English, German, French, Spanish, Italian and Dutch out of the box, and a workspace can add others from a list of 44 languages so quoted history and out-of-office replies are recognised in them too.