One issue is that for a square you will need > 45 degree relief angle.
The maths isn't that terrible for a polygon, you just need to consider the distance to the surface from the inscribed (or circumscribed, both ways work) circle at each angle as the work rotates.
But your HAL component does need to know the current reference (circle) diameter.
This is the component that I use, in conjunction with external_offsets:
Code:
component hexbore "A component to bore polygonal holes";
include "rtapi_math.h";
pin in float spindle-position "scaled 1.0 per rev";
pin in float radius "halui.axis.x.position-relative";
pin in unsigned facets=6 ;
pin in bit enable;
pin out signed offset "scaled x10000";
pin out float angle;
function _;
license "GPL";
author "andy pugh";
;;
double fmod(double x, double y){
return x - y * (floor(x / y));
}
FUNCTION (_){
if (! enable || facets < 3) {
offset = 0;
return;
}
angle = (fmod(spindle_position, 1.0 / facets) - 1.0 / (2.0 * facets)) * 6.283185307179586;
offset = 10000 * ((radius / cos(angle)) - radius);
}