We seek to understand and document all radio transmissions, legal and otherwise, as part of the radio listening hobby. We do not encourage any radio operations contrary to regulations. Always consult with the appropriate authorities if you have questions concerning what is permissable in your locale.

Author Topic: directKiwi project  (Read 623 times)

Offline linkz

  • Jr. Member
  • **
  • Posts: 31
    • View Profile
    • utilities & digimodes by linkz
    • Email
directKiwi project
« on: May 01, 2018, 0755 UTC »
Hi all

Recently I wrote a piece a software that allows you to quickly connect to KiwiSDR remotes around the world
It is based on python, so you have to install some stuff to run it (it should work on Windows/Linux/MacOS X)

What it is doing:
* fast connect from a list of <available> servers (+/- 200)
* fast switch between remotes (good for signal check)
* audio stream only
* bandwidth + AGC/MGC setting available
* update the listing by parsing kiwisdr.com/public webpage (servers listed are checked via GeoIP + port open check)
* returns the s-meter
* light CPU usage for low-power computers
* default parameters saved in config file

What it is not doing:
* real-time parameters changes
* waterfall not displayed
* and.. almost all the OpenwebRX/web interface can do

I think it could be good for signal check on different remotes, faster than opening multiple openwebRX in your web browser


All information on the project page:

http://81.93.247.141/~linkz/directKiwi/

(feedback, requests appreciated.)


&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Online TheRelayStation

  • Sr. Member
  • ****
  • Posts: 268
  • 6880 Khz AM 50W/500W - shortwavepirate@aol.com
    • View Profile
    • Email
Re: directKiwi project
« Reply #1 on: May 01, 2018, 1815 UTC »
very good job.
will the idle timeout (configured independently by each server) still apply ?
ive found an easy method to prevent the idle timeout by stopping the page script, this stops all functions except for the incoming audio.
modifying the page script also works but its not as easy and quick as simply stopping it.
this manner allows me to pass the I/Q (6Khz max BW) to software of choice such as "sdr-sharp" for the use of adjustments and plugins, other than the ability of frequency control.
6880 Khz AM 50W/500W - shortwavepirate@aol.com

Offline linkz

  • Jr. Member
  • **
  • Posts: 31
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #2 on: May 03, 2018, 2211 UTC »
Hi

I'm not sure the timeout is bypassed by the script but i think so as there is no .js scripts involved
I'm trying to fix an issue that appeared recently, i don't know why yet, but software not running and returning a
" _tkinter.TclError: already exists "
on some setups (64bits) in the python console.

It still works ok here on 32bits Win7, no updates made
python 2.7.15 has been released recently, perhaps it's the problem, I will check later

I also need some help to better code the datastream to audio output on MacOS X, it's pretty laggy the way the arrays are managed at the moment

thanks for the try

&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Online TheRelayStation

  • Sr. Member
  • ****
  • Posts: 268
  • 6880 Khz AM 50W/500W - shortwavepirate@aol.com
    • View Profile
    • Email
Re: directKiwi project
« Reply #3 on: May 03, 2018, 2308 UTC »
Hi

I'm not sure the timeout is bypassed by the script but i think so as there is no .js scripts involved
I'm trying to fix an issue that appeared recently, i don't know why yet, but software not running and returning a
" _tkinter.TclError: already exists "
on some setups (64bits) in the python console.

It still works ok here on 32bits Win7, no updates made
python 2.7.15 has been released recently, perhaps it's the problem, I will check later

I also need some help to better code the datastream to audio output on MacOS X, it's pretty laggy the way the arrays are managed at the moment

thanks for the try

&linkz
thanks for the interesting software to tinker with.
i will keep you updated with any issues/bugs i discover in the process as well as any additional features i can suggest.
interesting page you have too, linkz.
6880 Khz AM 50W/500W - shortwavepirate@aol.com

Offline linkz

  • Jr. Member
  • **
  • Posts: 31
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #4 on: May 04, 2018, 1751 UTC »
Some news

I was able to fix the issue that blocked the code run using python 2.7.15

fixed version 3.62 is now available from the project webpage

And, we test that the KiwiSDR "admin timeout setting" is bypassed by directKiwi, no restrictions applied, so.. no kicks


Regards

&linkz

Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Online TheRelayStation

  • Sr. Member
  • ****
  • Posts: 268
  • 6880 Khz AM 50W/500W - shortwavepirate@aol.com
    • View Profile
    • Email
Re: directKiwi project
« Reply #5 on: May 05, 2018, 1646 UTC »
And, we test that the KiwiSDR "admin timeout setting" is bypassed by directKiwi, no restrictions applied, so.. no kicks


Regards

&linkz
that is great though i will delegate the use of no time-out operation of directKiwi on Kiwi SDR's that are not popularly used too often, to avoid angering the host operator and users.
i have done the same method as above in the past when stopping the page script.
thanks !
6880 Khz AM 50W/500W - shortwavepirate@aol.com

Offline benji

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: directKiwi project
« Reply #6 on: May 24, 2018, 1908 UTC »
Hi linkz,

I really do like your python program and it is one of the very few using the kiwiclient interface. After working with it a couple of times a few comments / ideas.

a. S-meter.
Perhaps it may be  better to replace the RSSI S-meter reading by a measure of signal to noise. There are different definitions for SNR. It could be expressed as the decibel value of the ratio of the mean over the standard deviation of the measured signal.

Another way of calculating SNR is found at http://rx.linkfanel.net/snr.html .
Quote from the KiwiSDR  forum :
"Auto Scale" button sets WF max/min by taking one waterfall result, sorting the resulting FFT bins, calling the 50% bin the noise level and the 98% bin the signal level. Then WF min is set to the noise level -10 dB and WF max the signal level +30 dB. Works best when zoomed out so there is a variety of signal levels. The sort/bin idea is from Pierre, who uses a similar idea for SNR scoring at http://rx.linkfanel.net and http://rx.linkfanel.net/snr.html
End Quote

I believe this method is also used to give the reported dB value for WSJT modes. Of course the -10 and + 30 dB are not needed for this purpose. Would be interesting to see that as a time plot together with S-meter readings and look under various QRN and QRN conditions at channels with voice and data transmissions to see if this is a useful to quantify what we hear and perceive as "better audio quality". I guess the kiwiclient is the place to add a SNR computation.


b. Scanning over a limited number of kiwi sites and frequencies
I find myself entering a number of frequencies in the GUI of your directKiwi project and then looking at a number of different kiwis to see how they compare.

Actually if this was automated that would be really nice. Something I was thinking of is have the user enter a list of say 10 frequencies, then select from the list of 200+ kiwis his/her favourite ones (perhaps also 10) to monitor. Scanning thru this table will give a dynamically updated view of the "best" kiwi site and frequency combination. Scanning could be sequential per kiwi ip address or perhaps in parallel with multiple web sockets open at the same time.

A 10 by 10 matrix having for instance frequencies as columns and host kiwi as row could display the latest SNR value. When enhanced with some colour coding akin to a heat map this would look quite very cool. The "best" selection of the moment could further be used to provide audio.
Using kiwclient to interrogate remote kiwis via the internet involves of course some time delay so a realistic refresh rate for such a matrix is probably 30-60 seconds.

Anyway just throwing around some ideas for this already nice directKiwi program.


Best regards, Ben

Offline linkz

  • Jr. Member
  • **
  • Posts: 31
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #7 on: June 02, 2018, 0759 UTC »
Hi Ben

thanks for the ideas

a: I don't think i can do this, as the SNR calculation, if i understand correctely is based on a wide display waterfall analysis and "github.com/dev-zzo/kiwiclient" is unable to get a waterfall.
What i can do is getting from linkfanel or marco (IS0KYB) already calculated SNRs and putting the data in the node listing ?
Anyway i wonder if a SNR calculation based on a fixed bandwidth (max 6kHz) demodulated SSB frequency is easy to do and reliable, i mean it should be difficult to compare the desired AF signals strength (it can be anything, not just like WSPR sigs) with the noise background in such a narrow BW..

b: scanning different remotes for various frequencies in an automated way should be codable... but if SNR can't be calculated, we'll just get s-meter info and (recorded?!) audio, so just a s-meter graph or table as result.. not sure if it can be useful.
BTW, i already planned to add a "favorite" system in the node listing, with checkboxes and basic filtering.
Also if i get the already calculated SNR values of each node as mentionned in the "a" answer, I can also filter the worst remotes from the listing, by adding the setting of "do not display remotes that has SNR below than: " but remember the node listing is not dynamic.


ISSUE: Recently i have noticed the listing had mixed countries and hosts. It is due to a specific remote located in India, for some reasons this node is not displayed correctly in kiwiSDR.com/public summary page (due to missing informations)
That make my update process completely failed, by mixing up the .db with false datas.....

I have fixed that issue in the 3.63 version, by computing that specific node informations manually. That new version is available on the project webpage : http://81.93.247.141/~linkz/directKiwi/


Note: I'm working on adding a real RECORDING button in the GUI. In the early versions I have already had a plugin that can record, but it was just based on pygame and can cause different issues depending on the user client PC settings (mainly the capture device setting) so i have removed it.
Remember this project begun by using "github.com/dev-zzo/kiwiclient" stuff in console, script was originally capable writing streams into a .wav file. I have just modified one or two lines of the code to play the audio instead of writing the .wav
So it will not be difficult to come back to the original behaviour, but simultaneously with audio output in speakers


&linkz



Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline benji

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: directKiwi project
« Reply #8 on: June 04, 2018, 0555 UTC »
Hi Linkz,

I noted the addition of microkiwi_waterfall.py to Christoph's kiwiclient.git which indeed takes a snapshot of a specified host kiwi. With default arguments it shows the calculation result over the whole 30 Mhz range but can be narrowed to a high zoomlevel. Locally I use it for manually selecting a suitable antenna for a chunk of the spectrum. This "global" SNR gives a pretty good indication for SNR at a certain kiwi like Marco shows at http://sibamanna.duckdns.org/sdr_map.html.

Another indication of SNR could be the dB value of signal mean over standard deviation just for the passband. The idea of suggesting a measure of SNR is an attempt to avoid kiwis with high RSSI readings due high local QRN or QRM end up on top of the list.

As you mentioned for voice signals within the passband of the selected mode it remains to be seen if there is any benefit of having such an SNR measure. Probably best way to find out is to plot this together with RSSI in a real time graph.

To go a step further I guess for digital signals measured BER would be a better qualifier. For speech SNR is not well defined since the power spectral density of a voice signal is highly variable with time and almost as random as the noise itself. There are routines trying to separate speech from noise first such as using VAD (voice activity detection) or syllabic rate detection and then estimate SNR. However this is probably overkill for the present project.

Anyway thank you for coming back to my suggestions and a filter for to exclude the worst remotes will already be a good addition.

Best regards, Ben.