The boat appears to be a basic "downriver" design, which means that it's intended to be maneuverable and won't track strongly. Adding a keel or skeg would help that, but I suggest finishing the rest of the boat first and paddling it for a while before you make any permanent changes.

I would strongly suggest that you NOT move the cockpit; trust the original design. Shifting it aft will affect the balance of the boat (the bow will ride higher, the stern lower). This may improve the tracking slightly, but at the risk of creating a boat that lee-cocks (turns downwind constantly), which is difficult to control and potentially dangerous. It will not improve the stability.

When you create the coaming, make sure you add thigh braces as well, for better control. You can make a simple seat from Minicel foam. You may want to add cheek plates, as they provide lateral support and a place to attach a backband, unless you intend to use a foam backrest.

If it doesn't have foot braces, you'll need some.

Ultimately, you may want to add bulkheads and hatches. If not, you'll need float bags and attachment points for them.

There isn't anything you can do to improve the stability, short of adding a lot of ballast to the hull, but most likely you will get used to it after paddling it for a while and find that there are many benefits to a more responsive boat.