What do I do when I'm scared of what my team is building?

You have the authority to shape what gets built, and you have started to fear it. That fear is not weakness or paranoia. It is conscience doing its job. The people who should worry most about powerful tools are exactly the ones with a hand on the wheel, like you. You cannot un-build the technology, and quitting would only hand the wheel to someone who feels nothing. Your fear is useful only if it stays close to the decisions. The question is not whether these systems have power — they do — but whether someone who takes that seriously is in the room when the hard calls get made. Right now that someone is you. So turn the fear into a practice, not a knot in your stomach. Name one specific way your team's work could cause harm, and put a real guardrail on it this quarter. A review step, a limit, a person who can say stop. Worry that changes nothing eats you. Worry that builds a fence protects people.