It's probably not an unequal distribution of weight that is making your kayak turn, and if it was you could experiment by shifting your body weight left or right to balance the boat.

More likely, the patching has altered either the shape of the kayak or the surface drag so that one of these is not equal on the right and left sides.

Adding a skeg might help the problem, although if the turning is severe the skeg might have to be adjusted to a slight angle to the centerline which would slow the boat. Adding a rudder would also compensate for the turning problem at the expense of drag.