I will have to take a look at it. I appreciate you bringing this method up. I wonder what the computational expense is compared to a typical FFT. I may have also found another solution, but I need to check into it more.
What's the attenuation of the 75MHz signal in water? I would think the signal would fall off really quickly, even for such a relatively low frequency. Is there a long antenna that stays near the surface? I guess an easy experiment would be to put an FM radio in a ziploc bag and dive to the bottom of a pool and see if you can still hear the radio station.
I hope you don't honestly intend to operate on 75 MHz exactly. That's the same frequency the air transport industry uses for marker beacons, and having both the FAA and FCC to answer to doesn't seem like a very good idea.
Are you perhaps proposing to operate in the 72 MHz R/C band?
75MHz actually does pretty well as long as it is not in brackish water. This project is not so much to do something completely new, just to provide a new supply of 75MHz receivers because the current supply is dwindling. I have a paper that I am trying to find for another individual. Once I find it, I will see if I can post a link to it. It gives the attenuation factors over a range of frequencies.
There is a 75MHz band that the FCC has designated for RC surface use. 72MHz is designated for air use. In that 75MHz band, they have also authorized other communication items (I think that it is pagers if I remember correctly).
I don't think you need to do an FFT. There are a couple of much simpler ways to do things. You can do an arctan (or approximation) of the IF or baseband to find phase and then take the derivative of that (i.e., the difference between the last two samples) to get the frequency. Another, possibly easier, method is to center two bandpass (or a highpass/lowpass) FIRs around the two FSK frequencies and then measure the ratio of the average power output of each (finding the power would require a squarer or square approximation and another lowpass to average).
There are a lot of other problems to deal with, though, if you need to do an AGC loop, carrier recovery, etc. I'm not sure how much your chip is already doing for you. This might be a huge task.
Edit: I just looked up the datasheet of that part. It's pretty nice, and no, it looks like you don't have to worry about AGC or carrier recovery. The output is not quadrature, so forget what I said about finding the arctan (which only works for complex signals). You'll just have two audio tones to discriminate, so I'd just use the two FIR filters. It's audio so the processing requirements are really low. I see now you already suggested the FIR filters (actually, for audio, IIR would be fine) but worried about the latency. I don't think you need to worry about that. We are talking about steering a boat, where 100ms of latency is probably nothing, and the IIR latency would be more like a couple of ms at the most.