HFU HF Underground
Technical Topics => Software => Topic started by: ChrisSmolinski on September 01, 2016, 1912 UTC
-
DGPS stations transmit the difference between positions indicated by GPS satellite systems and the known fixed position of the station. This allows higher accuracy. DGPS transmissions are 100 or 200 baud and are transmitted on frequencies from 285 kHz to 325 kHz in the longwave band. Hundreds of these stations are operated by the Coast Guard and other agencies around the world, and they can be interesting DX targets. Each station transmits a continuous stream of messages containing correction data for GPS. These messages also contain the station ID code, so they can be used to directly ID the station.
Since SDRs are capable of recording a chunk of the RF spectrum directly to a disk file, I realized that a decoder could be written to demodulate all of the DGPS channels at the same time, at both baud rates. They write this data as a I/Q file, storing the complex representation of a portion of the RF spectrum. A 50 kHz bandwidth is slightly more than enough to cover the entire DGPS band. I set my SDR software up to record overnight, then in the morning I can run the recordings through the software, and see what stations are present.
The software sets up 82 SSB demodulators, two for each of the DGPS channels, one is for decoding 100 baud and the other for 200 baud, that allows me to use a more narrow filter for the 100 baud case. The output of each demodulator goes to a DGPS decoder that looks for valid messages. A message is considered valid if it starts with the correct preamble byte, is of message type 6 or 9 (the most common sent), has a z-count (which is a time code offset from the hour) that is within a few seconds of what it should be, and passes the 6 bit parity word test. This eliminates the vast majority of bad message decodes, although every so often one will sneak through. This is because you can get multiple bit errors on a message that corrupt both the data and parity word in such a way that the parity check still passes. It is still necessary to visually inspect the decodes, and decide if a seemingly amazing DX catch is realistic, or more likely just a bad decode.
Below is a screenshot showing the output of approximately 24 hours of recordings of the DGPS band.
(http://i.imgur.com/JOzqanF.png)
The columns containing the following information:
• Count: the number of decodes of this station.
• ID: ID number of the station, stations transmit either the ID or one of the reference IDs.
• RefID1: The first reference ID of the station.
• RefID2: The second reference ID of the station.
• kHz: Frequency.
• Baud: The baud rate, 100 or 200.
• City: Station Location.
• Country: Station Location.
• Lat: Station latitude.
• Lon: Station longitude.
• km: The distance to the station from your location.
• deg: The bearing to the station from your location.
Most likely the Wiarton and Angleton decodes are corrupted messages, as the frequencies they use are both dominated by strong semi local signals.
Another way to look at the decoded data is with this graph, that shows the times that messages were received from each station
(http://i.imgur.com/dOsnozO.png)
You can see the various times stations were decoded. There are cases where a single decode was received (just a thin line), which was possibly a garbled message. But there are also cases for DX stations where several messages in a row were received (a thicker line). It is quite improbable that many messages were garbled in a row, with exactly the necessary bit errors to change the ID of the station, but also preserve the parity word check.
It is interesting to observe how two stations on a given frequency will alternate reception, as one fades out and the other fades in.
A very preliminary beta version of this program, Amalgamated DGPS, is available for download for those who wish to try it. It is only for Mac OS X, and requires I/Q recording files made in either the RF Space or Perseus format (and note that I have only tested with the former, the latter should work, but you never know). While there is no Windows version available at present, I may have one available shortly, so stay tuned!
http://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
Also another article about this software here: http://www.radiohobbyist.org/blog/?p=1907
-
I've updated this app with a new version, 0.2.0, which adds a bunch of features, including the ability to decode from regular WAVE audio files, if you do not have an SDR. The app can decode from multiple DGPS channels in the same WAVE file, as many as fit in the bandwidth. So if, for example, your radio is tuned to 300 kHz USB with a bandwidth of 6 kHz, then 301 to 305 kHz fit inside and will be decoded. You could of course tune to say 299.5 kHz and squeeze in another channel. Or make the bandwidth wider. Or both!
The graph window now shows a red graph at the top, which indicates the total number of messages per minute being decoded. It can be handy as a rough guide as to how well band conditions are.
I have also added support for a few other formats of SDR recordings, Studio1, ELAD, and Sdr-Radio, in addition to SdrDx / RF Space and Perseus formats. Note that I do not have all of these programs, so testing was done with files provided by others. I think it is all working correctly, but you never know.
It is still Mac only, but the changes to this version (which is close to a complete re-write) move me closer to being able to release a Windows version. It can be downloaded here http://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
-
Version 0.3.0 for Mac OS X has been released. This version adds live decoding from an audio input. It also adds a window of UNID stations (those that were received but the ID does not match any known ID/frequency/baud rate combinations). There's also the usual bug fixes.
Live decoding of audio was the last major feature that needs to be added. So now the task of making a Windows version can begin.
Version 0.3.0 can be downloaded here http://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
-
So is the software smart enough to triangulate your receive location based on receive timing, et al, or do you tell it where you are?
+-RH
-
So is the software smart enough to triangulate your receive location based on receive timing, et al, or do you tell it where you are?
+-RH
You have to tell it where you are located. I'm not sure that the transmissions are synchronized to UTC time, allowing for triangulation based on the time delay.
-
Wake the kids and phone the neighbors, Amalgamated DGPS is now out for Windows!
It seems to work OK, I've done all my testing on Windows 10. It should probably work on 7 and 8, maybe Vista and XP? Try it and let me know. This is the first release of a Windows version, so there are likely still some rough edges.
It decodes from both WAVE files and I/Q recording files from several popular SDR programs. I also decodes from live audio. It will decode as many DGPS channels as can fit in your audio passband (assuming your computer can keep up), I've been able to go up to 19 kHz wide.
Details and downloads here: http://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
-
Hi Chris,
Kool program! This is really nice to be able to monitor the full DGPS band......
I've found a typo a few days ago: #667 Ulling Do, South Korea, 319 kHz is showing station #001 Ull every time it is logged. The RFId is correct/the one assigned.
Also #821 Portsmouth C2CEN, VA, 313 kHz has been off air for over two years. #821 shows up occasionally and is a false decode. It is/was a testing facility for the USCG.
Next, we know there is a problem with the ADGPS RFId display. Is it your intent to show the assigned RFId's or only the received RFId? I would prefer to see what ADGPS received to further verify the log and for submission of logs to Rxx.
Interestingly, I have been seeing a lot of logs from South African stations primarily #203. Building up over a week or so from one log at a time to 4, 5, 6 daily logs. Accompanied by usually another AFS station but they only log one hit. The interesting part is that the logs are happening during AFS local daylight morning/early afternoon time (+9 hours from here - nighttime here on the left coast)! A few days none are logged but seem to follow the daily ebb and flow of what I log on HF DSC from Cape Town 8 & 12 mHz.
I listened to the Perseus SDR audio I/Q stream of several of these AFS #203 logs and it sounds like the USA station on this channel, #828 Angleton, TX fades/flutters. I ran DSCDecoder at the same time (I knew it would not log #203 as it takes DSCDecoder about 12 seconds to sync) but it skips a few seconds of solid decodes of #828 and then ADGPS logs #203 during these "skips". I checked down to the exact seconds.
Interestingly some of the ADGPS daily logs seem to show a pattern of LF propagation to some regions.
Finally, I read somewhere in the description of ADGPS that the user would have access to the station list. I did not find that function in the windoz distribution of ADGPS. Would like to see that to eliminate the "clutter" of truly impossible logs like India, Egypt and so on.
The Graphing function is also kool Chris. Once in a while it shows a gradual slope of decline in logs but mostly peaks and valleys then a decline near local daylight.
Don
-
Hi Don,
Thanks for the info about #667 and #821, I've corrected them in the station listing. I have noticed #821 popping up every now and then, with very few decodes, considering how close I am to Portsmouth, I assumed I would be getting more decodes if it was actually on the air, but did not know what the actual status was.
I'll have to look and see how difficult it would be to report the actual received RFid, right now the software does display both of the assigned values. It might be possible to display the received RFid in bold?
The app will look for your own copy of the station list, named dgps_station_list.txt and if it finds it, will use it in place of the built in list.
I do find pattern(s) of stations fading in and out, and propagation in general, very interesting. Sometimes you can see correlations between stations in the same general geographic area, and other times you can't probably because what you receive/decode is a function of not only whether that station's signal is making it to your QTH, but also whether there is another, interfering, station being received on the same frequency. It seems the more I learn about propagation, the less I know. ;D
-
Impressive software and some good DX. For those that may be interested, Card Sound Fl is at the SE tip of Florida.
What antenna were you using?
-
I've been using a 500 ft beverage (which really isn't a beverage down here). I have more recent logs in the Longwave forum, including some more distant catches, such as Portugal: https://www.hfunderground.com/board/index.php/board,16.0.html
-
Thanks for the info. Quite a list in the LW forum. I listened a bit just now with my TS-480SAT and the HyGain vertical and was able to hear more stations than I thought I would be able to. Watching the signals on Spectran and they are pretty distinctive. I will see if my neighbor will let me run a long wire across the back of his property. :)
-
Installed the software on one of my win 7 computers yesterday. Still using the Kenwood and HyGain but pretty good results so far. Cool stuff, I needed something to keep me from working DX on 40 at night. :) The bandwidth only allows receiving about 3 or 4 stations at a time though.
-
Hi Chris,
I found another station that popped up the other day and should be deleted:
#877 290 kHz Pahoa, HI (Decom 26Apr14)
Also decoding a WinRadio I/Q file is a problem: it is 32 bits, 40 kHz bw, stereo.
Cheers,
Don
-
Thanks, I deleted #877 Pahoa, it will be reflected in the next update.
Can you save 16 or 24 bit I/Q data? I am not aware of other SDRs that record the data as 32 bits, and there's likely no meaningful data in those extra bits, so I would prefer to not add a special case for processing 32 bit files. Also, I am not sure what is meant by stereo, as these are I/Q data not audio, or is that their way of signifying that the files contain two channels of data, I and Q?
-
Hi Chris,
I found another one that just popped up and has been off air since Oct 2014:
#820 Alexandria, VA It was an engineering test site.
So I guess there still will be the odd false decode. On some of the true Dx I don't think one could ever get 10 or more logs. That is why I have been keeping a log on paper to track them. Maybe as we go in to winter there will be a change.
It's like that with NDB's too. Sometimes they are there for a few cycles then gone.
Cheers,
Don
-
Hi Chris,
I found another one that just popped up and has been off air since Oct 2014:
#820 Alexandria, VA It was an engineering test site.
So I guess there still will be the odd false decode. On some of the true Dx I don't think one could ever get 10 or more logs. That is why I have been keeping a log on paper to track them. Maybe as we go in to winter there will be a change.
It's like that with NDB's too. Sometimes they are there for a few cycles then gone.
Cheers,
Don
Thanks, I will remove that one as well.
-
Hi Chris,
I just started using the new windoz version of ADGPS a couple of nights ago. Last night/this AM I logged 90 straight decodes of #971 Gatun Panama! One less in the bucket list. :D
Also the previous unid (684 on 325 popped up again this AM. It has been identified as coming from Southern Avionics Company in Beaumont, TX. They manufacture mostly NDB's but have an on the air test rig for DGPS transmitters. Apparently it just sends random data to test the transmitter. For my personal log use I named it #SAC as the call sign when they test NDBs on air on various frequencies is also "SAC".
When they were contacted they shut (684 down 31 Oct 16 awaiting word from the USCG if they could use 325 kHz. Don't know as of today if the USCG said it was OK or not.
SACo L/L: 30 03 56 N, 094 08 51 W
-
Gatun started to appear here in the logs a short time ago, I assume it was recently re-activated?
No decodes yet on 325 but I will watch for it, thanks for the tip!
-
Interesting. I've never paid much attention to the UNID window... until now.
I have a whole pile from the Jan 11 decodes (Jan 10, 2017 UTC 2320 - Jan 11, 2017 UTC 1625... so far (still decoding)):
Count ID kHz Baud
16 234 325 100
14 925 325 100
10 138 325 100
8 562 325 100
6 339 325 100
6 71 325 100
6 711 325 100
6 1023 325 200
5 299 325 100
5 879 325 100
5 340 325 100
5 208 325 200
5 913 325 100
4 701 325 100
4 411 325 100
4 255 325 200
4 407 325 100
4 39 325 100
4 961 325 100
4 1009 325 200
3 281 325 100
3 268 325 200
3 415 325 100
3 326 325 200
3 780 325 100
3 57 325 200
2 887 325 100
2 540 325 100
2 923 325 100
2 214 325 200
2 928 325 200
2 221 325 200
2 606 325 200
2 409 325 100
2 300 325 100
2 328 325 100
2 54 325 100
2 405 325 200
2 341 325 100
1 586 325 100
1 401 325 200
1 573 325 200
1 739 325 200
1 22 325 200
1 592 325 200
1 222 325 200
1 233 325 200
1 601 325 200
1 2 325 200
1 392 325 200
1 951 325 100
1 178 325 200
1 391 325 200
1 187 325 200
1 1019 325 200
1 729 325 200
1 948 325 200
1 942 325 200
1 908 325 200
1 0 325 200
1 80 325 200
1 83 325 200
1 128 325 200
1 378 325 200
1 325 305 100
1 878 325 100
1 365 325 200
1 154 325 200
1 591 325 100
1 515 325 200
1 311 325 100
Top UNIDs (> 50)
Count ID kHz Baud
568 1023 317 200
369 1023 296 100
354 1023 298 100
92 1023 296 200
74 156 311 200
70 256 298 100
69 14 317 200
58 1023 298 200
53 1017 298 200
The list is very long. Currently 4219 lines.
-
Hello Chris,
I've been testing the Windows version of the Amalgamated DGPS application on a W10 system. Very nice program ! There are however a few thing that al least makes me puzzled.
- First of all I'm wondering what the difference is between the blue and the black decodes.
- The User defined stations list that comes with the program has a wrong file name : DGPSSTATIONLISTING.TXT while it should be dgps_station_list.txt as per the instructions in the Readme file
- The row sequence in the stations list that comes with the program is not correct and doesn't fit with the instructions in the readme file.
- It would be better to make the "Max zCount error" 6 as advised in the Readme file. Too many newcomers that don't like RTFM'ing drop interest while they think it doesn't work as it should.
- The readme file says "Amalgamated DGPS can decode the entire 285 to 325 kHz DGPS beacon band ". But according to https://www.ndblist.info/datamodes/worldDGPSdatabase.pdf the band starts at 283.5. Are those stations skipped ?
For the rest, I'm really astonished what this little piece of software is doing. I'm using it together with Perseus IQ files. It saves such a lot of time of decoding frequency by frequency. Really great work !
Greetings from Belgium.
73,
Guido.
-
Hello Chris,
I've been testing the Windows version of the Amalgamated DGPS application on a W10 system. Very nice program ! There are however a few thing that al least makes me puzzled.
- First of all I'm wondering what the difference is between the blue and the black decodes.
Black means the station is being presently received, blue means it has not been received in a few seconds. It's a handy way to see which stations are still being heard right now.
- The User defined stations list that comes with the program has a wrong file name : DGPSSTATIONLISTING.TXT while it should be dgps_station_list.txt as per the instructions in the Readme file
I think I did that so the software would not default to using the external file, but instead the built in list.
- The row sequence in the stations list that comes with the program is not correct and doesn't fit with the instructions in the readme file.
I am not sure I understand what you mean?
- It would be better to make the "Max zCount error" 6 as advised in the Readme file. Too many newcomers that don't like RTFM'ing drop interest while they think it doesn't work as it should.
Yes, that is a good idea.
- The readme file says "Amalgamated DGPS can decode the entire 285 to 325 kHz DGPS beacon band ". But according to https://www.ndblist.info/datamodes/worldDGPSdatabase.pdf the band starts at 283.5. Are those stations skipped ?
Yes, I would have to see if the software can be extended to starting at say 283 kHz. Thanks for pointing this out.
For the rest, I'm really astonished what this little piece of software is doing. I'm using it together with Perseus IQ files. It saves such a lot of time of decoding frequency by frequency. Really great work !
Greetings from Belgium.
73,
Guido.
-
Thanks for the clarifications Chris.
I am not sure I understand what you mean?
In DGPSSTATIONSLISTING.TXT the row sequnece is
Freq;Baud;Station ID;Ref1;Ref2;Location;Country;Lat;Long
But according to the Readme file it should be
Station ID;Ref1;Ref2;Freq;Baud;Lat;Long;Location;Country
The latter one is working as it should.
73,
Guido.
-
In fact this piece of software is really outstanding! I'm wondering if something similar can be build to extract RDS signals from Broadcast FM IQ/WAV Files ?
-
In fact this piece of software is really outstanding! I'm wondering if something similar can be build to extract RDS signals from Broadcast FM IQ/WAV Files ?
Yes, it certainly could be done from an IQ file. Not an audio WAVE file as the RDS signal is not present.
-
Yes, it certainly could be done from an IQ file. Not an audio WAVE file as the RDS signal is not present.
[/quote]
Thanks for the prompt reply. The future looks bright.
-
I've released a beta of version 1.1.0, it can be downloaded from the bottom of this page: https://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
Changes:
Added timestamp of last decode to the main window.
Bugfix - Windows version would produce duplicate decodes using the step frequency option.
As always, keep a copy of your existing version, in case this breaks something :D
-
I've uploaded a new beta to the ADGPS page, the decode range has been extended to 283-325 kHz to cover the 283-285 Hz range used by some European beacons. Maybe I will eventually hear one of them on this side of the pond :D
Be sure to download the beta at the bottom of the page, the release version is still 1.0.9, I'll push out a new "official" release in a few days.
http://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
-
Great news Chris ! Tnx
-
Just released yet another beta, 1.1.0b4, which adds a new window, where you can create a list of stations to ignore. No more fake decodes from India ;D
-
Beta 1.1.0b5 out now, this version keeps track of the last used directories for loading I/Q files, and saving decodes.
Download from https://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
-
On winderz, it has remembered the last folder you did something in for as long as I can remember. I'm now running 1.1.0b5, but see no difference in behavior.
-
A new beta of Amalgamated DGPS is available. Version 1.2.0 b2 has two major changes:
No longer uses PortAudio for sound input, which should improve stability (previously it would often crash after a while when decoding from live audio vs a recorded file)
Now a 64 bit application for macOS.
It can be downloaded from the website: http://www.blackcatsystems.com/software/dgps_decoding_software_sdr.html
-
Hi Chris,
Question: with the decreasing number of US stations now on the air, is there any way to tell what stations are in the program's built-in list? I am decoding SDR files that are up to 10 years old, and wondered whether all the stations on the air at the time of the recording are in the list. I looked through the included DGPSSTATIONLISTING.TXT.BAK.TXT file and wondered if that represents the list that will be used when running the program? I also noticed that some of the stations in that file had a "-" in from the their frequency, and wondered if this signified a station that was not longer being checked by the program?
Thank you, and thanks for making the program available!
Brett Saylor W3SWL
-
Hi Brett,
There's a built in database of DGPS stations, which can be overrided with your own text file if named dgps_station_list.txt
DGPSSTATIONLISTING.TXT.BAK.TXT can be used as a basis for creating that file. Now, I will add... I tested doing that when I first implemented using an external station listing file, but have not tried it since, and I think I had a report from someone that it didn't work, but that was a while go, and I am not sure if they ever resolved it or not. In general, everyone is just using the built in station listing.
The stations in the file with a - are indeed off the air, and there are more since (that was from the big shutdown a few years ago).
The USCG site has a list of stations off the air, I would use it as the official reference. Also check the logs here on the HFU.
-
The stations in the file with a - are indeed off the air, and there are more since (that was from the big shutdown a few years ago).
Very good; I just discovered the UNID window, and noticed that the stations showing up there correspond to stations that are marked as "off the air" so that tells me I need to check UNID's in order to get a complete listing for these older recordings.
Thank you!
Brett
-
The UNID window is very inetersting and throwing up a few active stations here in Europe such as 304.5 #535 Klaipeda. Is it possible to export the UNID data?
Noel.
-
I don't believe it is, although I can look into adding something to do that.
-
Thank you!