I think you are using too small of a sled. You have no support for the cut-off wedge, and your 'fence', which backs-up the main part of your board, stops too far from the blade. And the blade isn't covered.

As Orville said to Wilbur: "Brother, something is gonna fly."

I know you know the steps, but I'll mention them for the benefit of other forum readers.

Start with a plywood base which is at least a foot longer and 6 inches wider. You'll fasten a cleat or runner to the bottom so it slides along one of the slots on the saw's table. You'll add two stop blocks so you can't push this new sled completely off the table.

Lower the saw blade, set the sled base on the saw table, slide it toward you until the front 'stp' keeps it from coming closer. Turn on the motor and slowly raise the blade so it cuts a slot through the sled base. Push the base forward to continue the cut until the rear stop block prevents you from going further. Turn off the saw and remove the sled. It is still one piece, with a zero-clearance it in it.

Remount your hold-downs and handle to the left of the slit. Make another hold down and put it to the right side of the slit. This will clamp the cut-off piece to the sled. The exact location for the three hold downs will vary depending on the scarf angle you want.

If you can't get the hold downs within an inch of the slit, run a piece of scrap wood between them, clamp or screw it to the base of the hold downs or the sled base, and run it through the saw to create a fence