packethushRequest access

Benchmarks

Measured on Indian phone calls.

Every number on this site comes from test calls with known answers, transcribed by a named speech engine, with a 95% confidence interval. Here is how we test and what we found.

Speech recognition.

1,152 Hindi test calls · 14.3 hours · Deepgram nova-3

originalleading modelpackethushchange vs original95% interval
Word error rate, 8 kHz0.7090.5350.485−32%29.8–33.1%
Word error rate, 16 kHz0.7230.485−33%31.3–34.6%
Words the caller never said, per call17.98.56.5−64%61.4–66.6%
Word error rate, 8 kHz
0.709 → 0.485 · leading model 0.535−32% (29.8–33.1%)
Word error rate, 16 kHz
0.723 → 0.485−33% (31.3–34.6%)
Words the caller never said, per call
17.9 → 6.5 · leading model 8.5−64% (61.4–66.6%)

“Leading model”: a widely used commercial voice-isolation model for calls, run on the same audio at its strongest setting and transcribed the same way.

Where packethush leads.

On the 673 of 1,152 calls where packethush beat the original and was no worse than the leading model (within 0.02 word error rate).

−42% vs original (39.8–43.7%) · −23% vs the leading model (21.2–24.8%)

−45% vs original (43.0–47.5%) · −25% vs the leading model (22.6–26.9%)

Hindi test calls at 8 kHz · Deepgram nova-3 · 95% intervals from resampling calls · on all 1,152 calls: original 0.709, leading model 0.535, packethush 0.485

Calls with no second voice.

161 recordings of 144 Hindi callers · 2.0 hours · Deepgram nova-3

word error rateoriginalleading modelpackethushchange vs original95% interval
Caller alone0.2720.2960.271−0.4%−2.3% to +1.6%
Street noise, 10 dB below the caller0.3860.3870.388+0.5%−2.7% to +3.7%
Street noise, 5 dB below the caller0.5130.4850.469−8.4%−11.7% to −5.1%
Caller alone
0.272 → 0.271 · leading model 0.296−0.4% (−2.3% to +1.6%)
Street noise, 10 dB below the caller
0.386 → 0.388 · leading model 0.387+0.5% (−2.7% to +3.7%)
Street noise, 5 dB below the caller
0.513 → 0.469 · leading model 0.485−8.4% (−11.7% to −5.1%)

The callers of the 1,152 test calls on their own: each caller's recording through a phone line (G.711), then with Indian street noise mixed in at 10 and 5 dB below the caller's speech. With no other voice, packethush keeps the caller's level within 1 dB on 94% of calls and lowers the noise between words by 38 dB.

Signals for your agent.

Caller speaking
99.8%
right when packethush is sure (58% of 20 ms frames); area under the curve 0.972
Other voice
99.6%
right when sure (54% of frames); area under the curve 0.943
Noise level
96%
of calls within 3 dB of the true background level

Same 1,152 calls, checked against the known caller and second-voice tracks. “Sure”: a probability at or above 0.95 or at or below 0.05.

Delay
+12 ms
on top of each 20 ms packet: 96 samples at 8 kHz
Capacity
280 calls
on one 8-vCPU server (AWS c7i), every frame on time in a 60 s paced test

How we test.

  1. Real Indian voices. Each test call has one caller and a second voice, both real Hindi recordings, laid over telephone-line noise.
  2. Hard on purpose. The second voice sits 0, 5, 10 or 15 dB below the caller (576 calls with it close by, 576 with it across the room). Calls average 45 seconds.
  3. As a phone line carries it. 8 kHz audio, processed 20 ms at a time by the same library you receive.
  4. Same audio, before and after. Both versions go to the same speech engine with the language fixed; transcripts are compared word by word against what the caller said, after the same text normalisation.
  5. Honest intervals. Error rates are pooled over all words; 95% intervals come from resampling whole calls, 2,000 times, keeping each call's before and after together.

Results depend on the speech engine. The evaluation kit measures the gain with yours, on your calls. More Indian languages are added here as they are measured.

Run the same test on your calls.

The evaluation kit measures word errors before and after with your speech engine, and how many calls your servers carry.

Request access