EasyDRF-CSK
NOTE: The design has now been changed from MFSK to CSK modes.
The original EasyDRF uses OFDM modulation. OFDM is fast and bandwidth efficient, but it has a high crest factor which reduces the average modulation power level through a transmitter.
In contrast, the CSK version uses PN modulated FM, with a single tone per baud, which has a lower crest factor. This raises the average power level of the modulated data signal by at least 6dB.
Orthogonality
Modems are normally designed to use orthogonal tones. This means the tone separation needs to be wide enough to keep the sidebands from the tone switching from crossing over to the next tone frequency. This is very important if the tones are simulataneous, like in OFDM.
But using tones that are spaced too closely in MFSK mode causes tone detector overlap, and when affected by multipath this causes incorrect tone detection.
For this reason, the modem design was changed from MFSK to CSK.
Shortwave propagation problems
Signals propagated by shortwave are degraded by various problems:
- Delay spread - Multipath propagation causes the frequencies in the signal to arrive at variable times, spread over some mS.
- Doppler spread - Multipath propagation and geomagnetic storms cause the frequencies in the signal to vary. This can be as much as 50Hz in severe cases.
- Noise - Signal detection is limited by atmospheric and manmade noise levels.
Possible ways to improve data decoding:
- Use single-tone modulation to put all the power on one frequency for best SNR. This should improve received audio SNR by 6 to 10 dB over OFDM.
- Use spread-spectrum techniques to spread frequencies in the data signal to avoid multipath fading notches and reduce the effect of Doppler shift. The decoder despreads the signal before detection. This puts all the tone power back on one frequency again for best SNR.
- Use wider tone spacing to reduce the effect of Doppler shift.
- Rotate frequency usage to avoid interference.
- Detect energy increases in the signal each baud period.
- Equalize propagation delays in the received data signal.
Standard methods:
- Use interleaving to spread out decoding errors.
- Use CRC checking to validate data.
- Use error correction to correct decoding errors.
Modes
Some modes being evaluated are:
Spread-Spectrum modes
These modes use different spreading pattern codes (not FSK frequencies) to convey data symbols:- PN Code Shift Keying - This uses PN (Psudo-Noise) codes to chirp modulate (FM) a sinewave. At least 1024 codes are possible.
- Sine Code Shift Keying - Uses sinewaves to chirp modulate (FM) the sinewave tones. The modulation sinewaves are also chirped to make them more different to each other.
- Sawtooth or triangle Code Shift Keying - Uses sawtooth or triangle waveforms to chirp modulate (FM) the sinewave tones. The sawtooth or triangle waveforms are also stepped among different frequencies and phases to make them different to each other.
All of these Spread Spectrum modes may be phase sensitive, so they may require an adaptive equalizer to perform well on shortwave.
The decoder multiplies each incoming baud with each of the possible codes of an identically chirped FM sinewave in an IQ mixer. The output of the IQ mixer is integrated and the vectorsum is measured. The matching code causes a zero-beat, causing the integrator to deliver a DC signal, while the non-matching signals cause a mostly AC waveform that causes the integrator to stay closer to zero output.
Since there is no FM detector used, this method should perform very well, even with noisy signals and propagation phase errors.
Spread Spectrum
Spread spectrum methods can improve the modem performance by evening out both the channel response and the modem response.
Spread Spectrum is a way of spreading a signal out across a particular bandwidth, and un-spreading it at the receiver/decoder. This "dilutes" the signal across the spreading bandwidth, which helps to even out the effects of narrowband interference and propagation disturbances in the channel. In this application, the spreading bandwidth is only within the communications channel.
The decoder despreads the signal, putting all the power back on one frequency. If the original frequency was a single-frequency audio tone, then the result will also be a single-frequency. If the despreading is done by zero-beat, then the output will be DC. The despreading of the signal also spreads the channel noise, causing a drop in the noise floor in the decoder.
With a constant-amplitude data modulation, the most suitable Spread Spectrum mode to use would probably be chirp. Chirp typically uses a sawtooth ramp to sweep the signal over a range of frequencies, but other waveforms can also be used.
PN code chirp
Tests conducted with a PN (psudo-noise.. random numbers) based chirp spreading code showed that it was readily possible to encode and decode up to 1024 different symbol patterns on the same channel. However, it also showed that the signal became phase-sensitive, which is a troublesome side effect - as shortwave signals are often distorted in phase by multipath propagation.
- When a different PN coded despreading chirp is applied in the decoder, the decoded result resembles noise.
- When a matching PN coded despreading chirp is applied in the decoder, the decoded result has a high DC value, and very low noise. Phase errors from filtering or propagation increase this noise value, as they change the PN phase characteristics, causing degradation.
- It may be possible to adaptively tune a channel equalizer by observing the decoded noise level to compensate for this.
- The noise generated by using a different PN code in the decoder can be high amplitude. This AC noise needs to be reduced by an integrator to avoid false symbol detection.
Linear ramp chirp
A linear ramp chirp may be more suitable for this application. This is because a linear ramp has relatively low frequency content, so it will have less phase distortion problems from propagation. A linear ramp can still spread an audio tone across a range of frequencies to help avoid amplitude notches from multipath. After despreading, the tone will be back on the original frequency, where it can be sent to an FFT for tone (symbol) detection, or for CSK it can be zero-beat.
Linear ramp chirp can also improve the isolation between narrowly spaced tones, by using different ramp slopes for adjacent tones. This could improve the apparent tone spacing of the MFSK, which would be useful for reducing or eliminating the effects of Doppler spread.
Sinewave chirp
A sinewave chirp might be the easiest waveform to easily recover, as the chirp modulation waveform is only one frequency. This allows it to be filtered without changing the waveform shape. A combination of filtering and Hilbert clipping can effectively suppress noise on the recovered sinewave. The recovered sinewave could then be used for chirp and baud synchronization. Each baud rate has a different period, so each has a different chirp modulation frequency - therefore each will need it's own filter.
Sawtooth or Triangle chirp
A sawtooth or triangle chirp can also be used to encode the signal. A good pattern needs to be contructed by using different frequencies and phasing of each code. Ideally, each code should have very low correlation with all other codes for good results. This requires minimal overlap of the waveforms, although they can cross over each other. The advantage of these waveforms is that they will contain a lower frequency content than a PN code, and could possibly be mapped out to have higher isolation and thus better signal separation compared to PN codes that rely on statistical patterns to avoid each other.
Protocol
A protocol is required to transmit data in an orderly way. Transmitted symbols are grouped into frames for transmission. File objects are transmitted in segments. Each segment has a header, containing important decoding information such as segment number, segment size, transport ID, and CRC code.
The HamDRM/EasyDRF protocol uses two layers - an inner layer of frames with Convolutional coded ECC, and an outer layer of data segments with Reed-Solomon ECC. The data segments have CRC which is used to detect failed segments as erasures, to improve Reed-Solomon decoding performance by a factor of 2.
In the HamDRM/EasyDRF protocol, the filename data is sent in the header of segment 0. This is repeated at the end of the file data, in case segment 0 is lost. Since this does not guarantee the filename data will arrive, in EasyDRF the filename data is also sent in a header attached directly to the file. This guarantees delivery, but can't be decoded until there is adequate data present, due to the interleaved RS coding.
The current CSK modem design uses fixed frame sizes to avoid frame size errors due to a lack of error correction. When ECC is added later, this may change. Also there is currently no outer layer of ECC or data segments. This will be added later.
Fast Access
Ideally, a data modem should automatically sense the mode being used, and switch to it. This can be done by sending the mode at intervals, or by testing for each mode to find a match. In HamDRM and in EasyDRF this is done via the "Fast Access Channel".
A fast access channel would be easy to add by indicating the mode using the first symbol in the frame. The symbol would use the lowest of the CSK modes (256CSK) and could have it's detection integrated over time to reduce the effects of noise. It is then a matter of writing a mode table to hold the mode data. With 8-bits, there are a maximum of 256 modes possible.
Automatic baud sensing could also be added to the FFT autotuning stage. This would sense the pilot frequency spacing which is specific for each baudrate.
CSK Modem
Since the closely-spaced MFSK tones were difficult to decode correctly with multipath, the modem has been changed to use CSK mode.
Fast, closely spaced MFSK tones have wide sidebands. These sidebands become skewed by multipath fading, causing decoding errors. CSK evens out the modem spectrum which solves this problem. It also allows many more codes to be used, thus increasing the throughput of the modem.
These are some likely mode combinations:
| Code Shift Keying CSK | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Mode | Codes | Bits | Baud | Total BW | Raw speed | Sync | Frame | Chirp Hz | Code | CPU |
| (64kCSK) 65536CSK | 65536 | 16 | 5mS | ~4kHz | ~3200bps | 15mS | 256sym | ±2000 | PN | V.High |
| (32kCSK) 32768CSK | 32768 | 15 | 5mS | ~4kHz | ~3000bps | 15mS | 256sym | ±2000 | PN | V.High |
| (16kCSK) 16384CSK | 16384 | 14 | 5mS | ~4kHz | ~2800bps | 15mS | 256sym | ±2000 | PN | High |
| (8kCSK) 8192CSK | 8192 | 13 | 5mS | ~4kHz | ~2600bps | 15mS | 256sym | ±2000 | PN | Med |
| (4kCSK) 4096CSK | 4096 | 12 | 5mS | ~4kHz | ~2400bps | 15mS | 256sym | ±2000 | PN | Low |
| (2kCSK) 2048CSK | 2048 | 11 | 5mS | ~4kHz | ~2200bps | 15mS | 256sym | ±2000 | PN | Low |
| (1kCSK) 1024CSK | 1024 | 10 | 5mS | ~4kHz | ~2000bps | 15mS | 256sym | ±2000 | PN | Low |
| 512CSK | 512 | 9 | 4mS | ~4kHz | ~2250bps | 16mS | 256sym | ±2000 | PN | Low |
| 256CSK | 256 | 8 | 4mS | ~4kHz | ~2000bps | 16mS | 256sym | ±2000 | PN | Low |
All these CSK modes have now had basic tests:
| Code Shift Keying CSK | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Mode | Codes | Bits | Baud | Total BW | Raw speed | Sync | Frame | Chirp Hz | Code | Tested | CPU |
| (64kCSK) 65536CSK | 65536 | 16 | 4mS | ~4kHz | ~4000bps | 16mS | 256sym | ±2100 | PN | OK | V.High |
| (32kCSK) 32768CSK | 32768 | 15 | 4mS | ~4kHz | ~3750bps | 16mS | 256sym | ±2100 | PN | OK | V.High |
| (16kCSK) 16384CSK | 16384 | 14 | 4mS | ~4kHz | ~3500bps | 16mS | 256sym | ±2100 | PN | OK | High |
| (8kCSK) 8192CSK | 8192 | 13 | 4mS | ~4kHz | ~3250bps | 16mS | 256sym | ±2100 | PN | OK | Med |
| (4kCSK) 4096CSK | 4096 | 12 | 4mS | ~4kHz | ~3000bps | 16mS | 256sym | ±2100 | PN | OK | Low |
| (2kCSK) 2048CSK | 2048 | 11 | 4mS | ~4kHz | ~2750bps | 16mS | 256sym | ±2100 | PN | OK | Low |
| (1kCSK) 1024CSK | 1024 | 10 | 4mS | ~4kHz | ~2500bps | 16mS | 256sym | ±2100 | PN | OK | Low |
| 512CSK | 512 | 9 | 4mS | ~4kHz | ~2250bps | 16mS | 256sym | ±2100 | PN | OK | Low |
| 256CSK | 256 | 8 | 4mS | ~4kHz | ~2000bps | 16mS | 256sym | ±2100 | PN | OK | Low |
The CSK modem appears to work well. A 3kHz version will be developed later, by changing the parameters to fit a narrower channel.
More codes push more bits over the channel without much penalty, but the decoder CPU goes up fast because the decoder needs to search through every code to find a match. 65536CSK has had basic tests, and appears to work about the same as 256CSK, except for the higher speed and the very high decoder CPU usage.
For higher CSK modes like 64kCSK, the decoder needs to be multithreaded for best performance. Even so, the demands may be too high for many systems. 4096CSK has far lower CPU usage, and is still fairly fast at 12 bits per baud.
Ideas for CSK
A fast access channel could be created by adding a frame header to inform the decoder which mode was being sent. Since this data would not change during a transmission, it could be integrated for better SNR. This would be done by writing the incoming symbol data into an array and averaging the symbols before decoding. Slower information could also be sent in pages or in serial form, to avoid wasting bandwidth.
It may also be possible to detect the transmission mode by measuring the frame sync interval. If it was unique for each mode, that alone would be a good indicator of which mode was in use. Factors affecting the frame sync interval are frame size and baud rate. This would require the sync detection to try different frame sizes, which may not be ideal.
Currently, work is underway to try to optimize the modem waveform for lowest sidebands and to optimize the symbol phase. Since the waveforms are all carefully generated, it seems obvious that sending a specific phase would allow accurate channel phase measurement, which would allow an adaptive equalizer to greatly improve the channel performance.
Ironically, the spectrum spikes caused by modifying the PN code FM waveforms to cross zero at the baud transistions have proved to be very useful. Those frequency spikes appear to make very good frequency pilots! No signal power had to be wasted to provide them. An averaged FFT can easily identify both the frequency of the signal, plus the baud rate by some simple analysis.
FFT coarse frequency correction is now working. PLL fine frequency and baud sync are also working, but not optimized. More to follow...
Progress Update:
November 2024The graphics pane has been updated. The original WinDRM source code for the graphics class was missing, so it has been written from scratch. The waterfall code was very simple to draw (BitBlt the pane down 1 pixel, then draw 1 line of FFT data), but getting it to display cleanly during buffer updates required careful sync with the buffer reads and writes. This was made automatic by an adaptive timing function to ensure best results. The adaptive function adjusts the average buffer read timing to keep the read and write pointers about half the buffer size apart.
A simple "Dashboard" display has been implemented to help diagnose the modem performance. It shows some stage signal levels, plus timing and phase data:
The waterfall FFT data has had some temporal smoothing applied to improve the SNR on weaker signals.
The CSK signal, using the new waterfall code:
Currently only the waterfall, dashboard and phase displays are working.
The decoder is now working at an 8kHz sample rate to reduce CPU use. This halves the CPU usage in the decoding function, but for higher CSK modes it still requires multithreading to be added.
More to come...
October 2024
Frequency correction and timebase correction have now been added to the decoder.
All CSK modes from 256CSK up to 65536CSK have now had basic tests. All of these CSK modes appear to start being useful at about 3dB SNR. The 65536CSK mode has extremely high CPU usage - 1,048,576,000 complex multiplies per second! (at 16kHz rate - now changed to 8kHz)
A CMA automatic equalizer has now been added to the decoder. It helps reduce the phase and amplitude errors caused by multipath fading. It may be improved further yet by increasing the iterations.
An additional decision based equalizer has not performed as well as it should have. This is probably due to poor implementation. This may be improved later...
The high CPU use of the higher numbered modes could be reduced by making the decoder multithreaded. This is probably best done by breaking the frame up into 8 segments and decoding it in parallel on different CPU cores. The code to do this will need some synchronization to avoid timing problems.
A PLL has now been added to the baud decoder. The PLL takes the integrated Q phase and rotates the incoming IQ signal to phase lock it.
Earlier...
More of the LED style indicators have been enabled, to show decoding status.
Another multipath + noise simulation:
Resetting the baud to zero phase at the start, and modifying the phase to end on zero produces a controlled phase reference while keeping artifacts low. There are some spectral peaks in the passband that appear to be from the regular pattern of zeros at the baud transitions. To randomize the spectrum even more, a simple bit scrambler has now been added by taking the XOR of the data and the symbol index number.
A decoder IQ display will help improve the sync performance, by clearly indicating how well the decoder is locking.
This example has added white noise.
Earlier...
Now that the audio buffering glitches are fixed, the CSK modem has started to work quite well. It still needs an adaptive equalizer, sync improvements, and error correction, but the results appear very encouraging! Here is a multipath simulation video.
Earlier...
The CSK modem is starting to work. A linear ramp frame sync appears to work very well. At this stage there is no baud sync, but it may not be needed. Currently 256CSK is being tested, but 1024CSK or 2048CSK might be tried next...
September 2024
The Chirped MFSK is proving difficult to decode correctly when affected by multipath. So far, a channel equalizer has not helped significantly, but the code may not yet be working correctly...
Instead, work is now focusing on Code Shift Keying.
Code shift keying uses a PN code, a sawtooth, triangle or sine waveform to FM a tone oscillator. This produces a flat envelope. The FM spectrum is carefully controlled by limiting and filtering.
If sawtooth, triangle or sine waves are used, they need to be randomized in different patterns to make a series of non-overlapping code waveforms. This requires some analysis to produce ideal spreading waveforms...
It appears very easy to encode and decode 1024 different symbols in a 4kHz channel using PN codes.
1024CSK PN code test - IQ mixer output:---- Earlier updates ----
Since the MFSK data modem requires a delay equalizer to counteract the effects of multipath propagation, code to do this has been tested.
A "Constant Modulus Equalizer" appears to be the most suitable for a constant-amplitude signal like CSK.