You're probably getting good advice. Doubles are tough to solo in rough conditions. That is what you are doing with a 7 year old in front when the wind comes up. For whole family paddles, we've had better luck with everybody in a canoe. The kids have room to shift around, help paddle (backwards, more often than not), throw rocks in the water (you had them collect a bucket full beforehand, right?), and so forth. Also, we stop fairly often for exploring and picnics.

For kayaking, we've had our best luck with providing them with child size boats and having pond-side outings. They take turns skippering their own boat, but stop when they want to do more important things, like catching tadpoles. I'm hoping one day their priorities will change and they will want to paddle across the Strait of Belle Isle with me, but I've got a while to wait. (ages 9, 5, and 3)

Paddling with your children is not the same as paddling with your friends, but it has it's rewards.