Well, you just don't know for sure what kind of contams those bugs will have brought into your kit. And, being a first timer, you really don't want to take the chance that instead of what you think is a Psilocybe strain ends up being something like Galerina spp etc. The effects of toxic mushroom poisoning can include death by kidney or liver failure, usually only after several excruciating days. Simply, it ain't worth the risk.
Your grow has to be 100% uncontaminated. Bugs = contamination.
Grow kits are in any event a ripoff. I would strongly recommend you follow the Tek on this site and do it the right way, you'll learn a LOT more and it is fun.