I managed to resolve it. Turns out those boards are maybe not for the same type of encoder? The sin and cos were inverted and so were their + and -. The connector was also mirrored the wrong way. I managed to cross the DB15 connector pins in a way to get it to work, though, and I can happily report it's basically plug and play after that. All that's left is to play around with the resolution with the jumpers a bit...