About the company
All questions should be directed to support@voicegremlin.com
VoiceGremlin is a SaaS tool that simplifies automated testing for intelligent voice agents (IVAs) that answer phones. Modern software teams rely on test automation implemented via API calls to know their systems are working correctly after they deploy changes, but testing a virtual assistant over the phone is an inherently manual process that's tricky to automate. That's the problem we solve.
Your test automation makes two API calls to VoiceGremlin - the first to tell us what phone number to call and what you want to validate, and the second one a minute or so later to find out if the test passed or failed. The test is performed by two customized agents - one to call your IVA and role-play as one of your customers, and the second to read the transcript and judge whether your agent's behavior passed or failed the test. The response will include an unequivocal Boolean "passed" value for each test as well as the full transcript of the call.
Most test automation frameworks require a lot of configuration - first define a target, then define a test case, then define a test run that specifies which test cases to execute against which targets, etc. We can skip all of that because for an inference model, the description of what the test case should do is the test case.
This makes VoiceGremlin incredibly easy to implement - no CRUD endpoint, no SDK, and no special nomenclature to learn. You can configure your tooling to call our API directly, or use the scripts from our Quick Start.
Based on my experience building VoiceGremlin, here's what I would suggest you budget for if you decide to build this in-house:
- 1-3 days of dev time to integrate the dependencies (a telephony provider, an LLM provider, and optionally TTS/STT) and to orchestrate the multi-turn state machine
- 4-8 hours of prompt tweaking for the caller persona and judge criteria
- $0.05-$0.20 per test minute in variable costs, depending on your model and dependency choices.
- ??? hours for vendor KYC (someone from your company will need to scan their government ID and undergo identity confirmation before most providers will let you make outbound phone calls)
Adversarial (or "red team") tests are attempts to get an IVA to circumvent rules (e.g. process a refund for a product that hasn't been ordered) or do something you wouldn't want it to (e.g. use swear words). VoiceGremlin is an open-ended tool and will attempt to execute such tests if asked, but it isn't optimized for this and I would not recommend you rely on it. Humans are much better at out-smarting LLMs than other LLMs are.
Most users we talked to don't want to expose their test framework to inbound traffic, so there's currently no way for VoiceGremlin to call out to your system to tell you when a test is complete. After queuing a test your framework should query the Results endpoint every 10 seconds until the test has passed or failed. If you'd find a webhook valuable, drop us a line: support@voicegremlin.com
This is a tool for calling voice agents - it should never be used to call humans. Attempting to get the service to perform cold calls or marketing will get your account suspended. Additionally, if we ever detect a human on the other end of the line we will block calls to that number. If you believe a number or an account has been blocked incorrectly, please contact support@voicegremlin.com.
VoiceGremlin was founded in 2026 by Nick Hall, a software engineering middle-manager with a background in test automation. My goal is to follow the Unix philosophy (Write software that does one thing and does it well). There are no other products to upsell you to, no investors to satisfy, and no "pivots" planned.