Wow that seems like quite a bad limitation of modern processors
It's fallout from all the optimizations that modern cores employ. For example, the time it takes for your interrupt handler to be executed depends heavily on whether the handler is already in the instruction cache or not. These kinds of things introduce a large uncertainty into the predictable time performance of code. It's generally pretty fast thanks to these mechanisms, but not predictable. Additionally, the interrupts from the programmable logic are somewhere in the middle of the priority spectrum, so it may well be that other interrupts are serviced before the GPIOs - or your handler gets interrupted itself.
Event-driven callbacks are not supported by the C runtime in this area. The POSIX compliant way of doing these things is to call blocking system functions like select() with a list of events you are interested in, and they will return when an observed event arrives - or a timer expires, optionally. You can do that in a dedicated thread which examines the result of select(), calls your callback and then calls select() again. Or you use polling like you suggested.