Stopping my detailed review because I don't think this approach to optimizing the path is quite right.
Suppose I fill a polygon that very closely approximates a circle, that this circle contains the entire bounding rectangle of the Graphics object, and that it does not intersect any edges of the bounding rectangle. That would mean that none of the individual points intersect the bounding rectangle. For a sufficiently close approximation, no set of 3 adjacent points in the polygon would intersect the bounding rectangle.
If I understand correctly how `clip_path` is meant to work, it would remove every point except for the starting point, which would mean that nothing would be drawn, when instead the full bounding box of the Graphics object should be filled.
I think that in order to fix this case, you would need to ignore any points not added to the output rectangle. So if I have a polygon that goes A B C D (with A being a start point and B C D being line points), and point B is removed, then you need to check points A, C, and D in order to determine whether point C can be removed.