Linux might have gotten its start in the PC world, but it is quickly becoming a major force in embedded applications. In the last quarter, for example, Motorola shipped 1 million Linux-based cell phones in China alone.
With this kind of success story, it's no wonder that DSP engineers are enthusiastic about using Linux. However, I think the DSP community has gotten a little too excited about Linux. I say it's time to ignore the hype and take a tough look at Linux.
One of the big appeals of Linux is that it is free: You can download a copy for free (here's one example), and there are no royalty charges. That seems to be a huge plus for Linux, until you consider that Linux has a very large memory footprint. A typical installation can eat up a few megabytes. In contrast, a lightweight RTOS like Nucleus can fit in a few dozen kilobytes. The extra memory required by Linux can significantly increase the cost of the system. (For good overviews of the costs associated with an OS, see this Green Hills page and this Wind River page.)
Another reason for moving to Linux is that it is a widely-used OS with a large developer base. While that is true, only some Linux developers have the skills needed for DSP programming. A developer of Linux-based web servers is unlikely to understand the needs of DSP systems, for example.
Perhaps the biggest problem with Linux is that it is a non-deterministic OS. For some applications, this isn't a problem. There are also products such as Jaluna OSware that attempt to give Linux better real-time behavior. However, Linux is almost certainly the wrong choice if your system needs fast interrupt responses or has other tough real-time demands.
Don't get me wrong: I'm sure that Linux will become a viable option for more and more DSP applications over time. However, Linux isn't yet the right choice for many applications—and it will never be the right choice for some applications.