There are two different conditions that seem similar when first encountered, both involve balance of lateral areas.

If the lateral areas above and below the waterline do not balance then the boat will react to a crosswind, either weathercocking upwind or falling off the wind. The best way to cure this IMHO is to add some side area above the gunnel, it only needs a few square inches at one end of the boat. My first boat was like that and my second one was built with a slightly lower stern to prevent it.

If the center of lateral area below the waterline is forward of the center of gravity the boat will have a tendency to round up even without wind; usually it is only noticed when you stop paddling and the boat starts to drift off to either side. However, once I paddled a kayak that was so bad it could not be kept in a straight line above a certain speed; it was rented for a river tour and I could not keep up with my companions as the boat would head for the river bank as soon as I got up to speed unless I paddled on one side ti bring it back on track, when it would round up on the other side. Later the renter told me he left off the rudder so it would not be damaged, which explained all. A skeg or a rudder fixes this problem; in a new boat reduce bow stem foot area.

Both conditions can be cured with a skeg but a rudder is more use I think. A skeg is not the best solution for a boat used in narrow waters where agility counts. I added one to my first boat and it fixed the weathercocking but it's about as maneuverable as a train; fine for lakes but a nuisance when exploring streams.

Moving the paddling position doesn't help much as it also affects underwater areas. Weight below the gunnel (out of the wind) is effective for curing weathercocking, but weight aft may induce the rounding-up problem.