AI voice agents are now ubiquitous, from walking us through a hotel reservation to booking our next doctor’s appointment. Voice agents can make things quicker, cheaper, and easier to access—but they also bring up some real ethical concerns that are hard to ignore.
Will they displace humans in various industries? And how can we best secure them ethically to keep them from harming us? Read on to find out how startups are tackling the big questions of AI voice agents and how they should be deployed with caution, thoughtfulness and strong safeguards to prevent harm.
Effects on Labor: The Call Center Landscape
One of the clearest concerns is what AI voice agents mean for labor, especially in call centers. Arkadiy Telegin, co-founder of Leaping AI, believes the change could be sweeping. “It’s going to replace all centers. Because most of it is a very repetitive job, so you just need a good conversational interface with the human and then the rest of the logic is going to fall in place.”
Despite this threat to the workers in call centers, Telegin argues that society can and should adapt. “I think we should pursue this kind of automation in parallel with some kind of social change that will make sure that call center workers can be reintegrated into society,” he says. The point is not only whether AI can replace human labor, but whether companies and governments are prepared to support the people affected by that shift.
Telegin also points to another advantage of voice agents: they can absorb difficult or abusive interactions that might otherwise fall on human workers. Describing one insurance call where a man started sex chat with the AI agent, Telegin said, “These kinds of characters are now kept away from real human beings and they pester our AI and not real humans. So I see it as a win-win.”
Thoughtfulness as a Cornerstone for AI Agents
Beyond labor concerns, there is a broader ethical issue: whether voice agents are being built thoughtfully enough to serve real human needs. Huzaifa Sial, CEO of CareForce AI, frames the work in terms of access and care. “Only a third of the people are getting the care that they need. And most of it is because just the basic stuff like scheduling and appointments and getting out to clinics is getting harder and harder. So we’re trying to solve that,” he explains.
CareForce AI uses two voice agents, David and Angelica, to close care gaps and help patients navigate routine but essential healthcare tasks. Sial describes how the system works: “What David does is actually goes through all your systems, finds the people that are due for some care and then it gives the list to Angelica, whereby she calls you right on your phone. It’s actually fascinating to hear. It’s real life engaging.”
For Sial, the value of voice AI is not simply automation, but better communication that highlights the importance of accessibility and explanation. “I’ve had folks say this is the first time I’ve really understood why I needed to do this procedure. No one’s explained this to me in Spanish or Mandarin before! Accessibility is what Angelica provides and this shows how to build around the idea of thoughtfulness.”
He adds, “I think with AI, if you are being thoughtful, I think for the first time there’s an opportunity to take work away from people, not actually add more to them.” That idea captures one of the more constructive visions for AI voice agents which doesn’t replace human care, but reduces the burdens that prevent care from being provided.
Testing AI Agents as a Guardrail
If voice agents are going to be used in sensitive industries, testing can’t be an afterthought. Sidhant Kabra, cofounder of Cekura, says the company was built to help ensure AI agents behave reliably and responsibly. He explains that customers wanted to make sure the agents “don’t hallucinate, follow the proper compliances while making sure that the workflows were being adhered to.” Cekura started in India and is now based in San Francisco having been selected by Y Combinator for 2025. It now works with more than 100 customers across healthcare, telecommunications, financial services, sales, and customer support.
Kabra stresses that high-stakes sectors require especially careful evaluation. “Healthcare and financial services are highly compliant sectors, hence making sure that the agents are reliable is a very big use case there,” Kabra told Startup Beat. “That’s why those sectors got a lot of traction for us as well, because the company shipping conversational AI in those sectors needed a very thorough testing of industry-specific tests, company-specific tests, as well as conversational tests.”
His broader warning is that AI systems can fail in unpredictable ways if they are not tested properly. “Because Gen AI is indeterministic, like LLMs, you need to make sure that you are well tested. You have made sure that all your edge cases are catered to, because if you go live without testing out the edge cases, the AI might create blunders, which can have repercussions. And hence, having a proper eval setup was pretty important.”
He also argues that the ethical boundaries of voice AI are often set by the customer. “The ethical framework is being built by the customer. So it’s very dependent on what the customer believes and whether they’re an ethical company itself,” he says. In practice, that means companies can’t outsource responsibility to the technology itself. They must define and enforce the standards they want the system to follow.
Kabra summarizes the testing framework this way: “Typically, when you think about reliability in voice AI, there are industry-specific checks that you have to do. For example, if you are building in healthcare, you have to make sure that your agent is HIPAA compliant. The second is company workflow-level checks. Each of the customers will have specific workflows, so you have to make sure that the AI is following that workflow. And the third is the conversational level checks, which is how it is behaving on interruptions, what is the latency, etc. But typically, these are the three things that need to be tested.”
The Ethics of Voice Agents
AI voice agents may be efficient, scalable, and even helpful in ways human systems have struggled to be. But their rise also forces a deeper conversation about their ethical use. They may reshape labor, especially in call centers. They may improve access to care when built thoughtfully. And they may only be safe when rigorously tested for compliance, reliability, and real-world behavior.
The central question is not whether voice agents will become part of daily life. They already are. The real question is whether they will be deployed in ways that are fair, accountable, and humane.
This article is a part of our series on the confluence of startups and ethics. If you have a take on a particularly spicy moral conundrum in the world of startups, drop us a line at info@startupbeat.com.
