sanding to 100 grit before starting the varnishing should be fine. After vearnish you'll want finer grits between coats.

The reason for sanding between coats of varnish is really two-fold..

1- it levels the surface to get rid of any sags, runs, drips, etc and to remove any dust specks, bugs, whatever on the finish.

2- and here's the real reason for sanding. The adhesion between coats of varnish is mechanical not chemical. Sanding gives the next coat a "tooth" to lock into.

Lacquers and finishes of that type slightly melt the previous coats, ( in the furniture refinishing trade we refer to this as "burning in") and actually become one with them, Varnish nor Polyurethane does this. So you need to sand to the point where the gloss from the previos coat is pretty much gone. As you progress through the coats of varnish of course you'll progress to finer and finer grits also. I finish furniture up with 320 and 4-0 steel wool unless it is getting rubbed to a finer gloss- then I use rottenstone. That's another tale.

good luck.