How do I catch the flaws that keep slipping into my studies?
You can design a study that mostly holds together. What still slips past you are the subtle things, the hidden confound, the sampling flaw that only reveals itself later. Those getting past you is not carelessness. They are genuinely hard to see, which is exactly why they trip up experienced people too. The reason they hide is that you are looking at your design from the inside, already convinced of how it should work. Flaws live in the assumptions you are not questioning because they feel obvious. The skill you are building is learning to attack your own design as an outsider would, hunting for the crack on purpose. Before your next study runs, take an hour to play the critic. Ask of every part: what else could explain this result, and who is missing from my sample? Better yet, hand it to someone sharp and ask them to try to break it. A flaw found before the study is a lesson. A flaw found after is a loss. Build the habit of hunting them early.