It won't take much to help the tracking. If you make a keel that's roughly 16" long and tapers from 0 to an inch in height - would be a good start trial. I would make it out of a scrap of pine thats about 1/2" thick. Attach it to the back end of the kayak with an electric glue gun. Try the boat and gradually use a block plane and cut it down until it feels perfect then you fashion a good one and epoxy it on with fillets etc.

If the boat is curved up in the back (rockered) then you would just carve one to fit. The important thing is to do a trial one and shape it first.