Thanks to everyone for the advice. I feel like I have always over built mine.
I have built 3 pygmy boats and one other. All have used 4 mm ply with 6 oz glass in and out. I am horribly hard on equipment and these kayaks seem like they could be built with 3 mm ply and 3 oz glass and be pretty strong. I drag mine everywhere and really don't fear big water (not the ocean but semi-regular trips to Superior). The only concession I give my boats is to not ram into rocks with abandon. I sit on them, bounce them around, etc.
My Pygmy boats are already very light but I want to build a kayak for a female friend and she wants to be able to load it easily herself.