Request for resources about sources of latency in embedded system design

I've built a couple of systems using PCs running LinuxCNC, but I'd like to start learning how to create my own embedded computers.

I've found quite a few resources for getting started with the design and manufacture of embedded systems. There are also open source examples like the Blueberry Pi. However, I haven't had much luck finding technical guides discussing the design choices that effect system latency. I also haven't found a really solid guide to understanding the sources of latency in the Linux preempt kernel.

Can you all recommend resources to help me learn about designing embedded systems for low latency applications? I'm most interested in guides to hardware design at the moment, but I would also like to have a deeper understanding of what impacts latency in the Linux real time kernel.

Ultimately I intend to run LinuxCNC on a lot of my projects, so resources using application controllers would be more useful than micro controllers, but anything is welcome.

That means recompiling for a different architecture...... Thats alot of work, that requires extensive knowledge. Way beyond my expertise....