We seek to understand and document all radio transmissions, legal and otherwise, as part of the radio listening hobby. We do not encourage any radio operations contrary to regulations. Always consult with the appropriate authorities if you have questions concerning what is permissible in your locale.

Author Topic: directKiwi project  (Read 10343 times)

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
directKiwi project
« on: May 01, 2018, 0755 UTC »
Hi all

Recently I wrote a piece a software that allows you to quickly connect to KiwiSDR remotes around the world
It is based on python, so you have to install some stuff to run it (it should work on Windows/Linux/MacOS X)

What it is doing:
* fast connect from a list of <available> servers (+/- 200)
* fast switch between remotes (good for signal check)
* audio stream only
* bandwidth + AGC/MGC setting available
* update the listing by parsing kiwisdr.com/public webpage (servers listed are checked via GeoIP + port open check)
* returns the s-meter
* light CPU usage for low-power computers
* default parameters saved in config file

What it is not doing:
* real-time parameters changes
* waterfall not displayed
* and.. almost all the OpenwebRX/web interface can do

I think it could be good for signal check on different remotes, faster than opening multiple openwebRX in your web browser


All information on the project page:

http://81.93.247.141/~linkz/directKiwi/

(feedback, requests appreciated.)


&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline TheRelayStation

  • Sr. Member
  • ****
  • Posts: 445
  • 5150 Khz AM 60W Hack RF
    • View Profile
    • Live Streaming
    • Email
Re: directKiwi project
« Reply #1 on: May 01, 2018, 1815 UTC »
very good job.
will the idle timeout (configured independently by each server) still apply ?
ive found an easy method to prevent the idle timeout by stopping the page script, this stops all functions except for the incoming audio.
modifying the page script also works but its not as easy and quick as simply stopping it.
this manner allows me to pass the I/Q (6Khz max BW) to software of choice such as "sdr-sharp" for the use of adjustments and plugins, other than the ability of frequency control.
5150 Khz AM 60W Hack RF
shortwavepirate@aol.com
https://vaughn.live/cabletv

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #2 on: May 03, 2018, 2211 UTC »
Hi

I'm not sure the timeout is bypassed by the script but i think so as there is no .js scripts involved
I'm trying to fix an issue that appeared recently, i don't know why yet, but software not running and returning a
" _tkinter.TclError: already exists "
on some setups (64bits) in the python console.

It still works ok here on 32bits Win7, no updates made
python 2.7.15 has been released recently, perhaps it's the problem, I will check later

I also need some help to better code the datastream to audio output on MacOS X, it's pretty laggy the way the arrays are managed at the moment

thanks for the try

&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline TheRelayStation

  • Sr. Member
  • ****
  • Posts: 445
  • 5150 Khz AM 60W Hack RF
    • View Profile
    • Live Streaming
    • Email
Re: directKiwi project
« Reply #3 on: May 03, 2018, 2308 UTC »
Hi

I'm not sure the timeout is bypassed by the script but i think so as there is no .js scripts involved
I'm trying to fix an issue that appeared recently, i don't know why yet, but software not running and returning a
" _tkinter.TclError: already exists "
on some setups (64bits) in the python console.

It still works ok here on 32bits Win7, no updates made
python 2.7.15 has been released recently, perhaps it's the problem, I will check later

I also need some help to better code the datastream to audio output on MacOS X, it's pretty laggy the way the arrays are managed at the moment

thanks for the try

&linkz
thanks for the interesting software to tinker with.
i will keep you updated with any issues/bugs i discover in the process as well as any additional features i can suggest.
interesting page you have too, linkz.
5150 Khz AM 60W Hack RF
shortwavepirate@aol.com
https://vaughn.live/cabletv

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #4 on: May 04, 2018, 1751 UTC »
Some news

I was able to fix the issue that blocked the code run using python 2.7.15

fixed version 3.62 is now available from the project webpage

And, we test that the KiwiSDR "admin timeout setting" is bypassed by directKiwi, no restrictions applied, so.. no kicks


Regards

&linkz

Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline TheRelayStation

  • Sr. Member
  • ****
  • Posts: 445
  • 5150 Khz AM 60W Hack RF
    • View Profile
    • Live Streaming
    • Email
Re: directKiwi project
« Reply #5 on: May 05, 2018, 1646 UTC »
And, we test that the KiwiSDR "admin timeout setting" is bypassed by directKiwi, no restrictions applied, so.. no kicks


Regards

&linkz
that is great though i will delegate the use of no time-out operation of directKiwi on Kiwi SDR's that are not popularly used too often, to avoid angering the host operator and users.
i have done the same method as above in the past when stopping the page script.
thanks !
5150 Khz AM 60W Hack RF
shortwavepirate@aol.com
https://vaughn.live/cabletv

Offline benji

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: directKiwi project
« Reply #6 on: May 24, 2018, 1908 UTC »
Hi linkz,

I really do like your python program and it is one of the very few using the kiwiclient interface. After working with it a couple of times a few comments / ideas.

a. S-meter.
Perhaps it may be  better to replace the RSSI S-meter reading by a measure of signal to noise. There are different definitions for SNR. It could be expressed as the decibel value of the ratio of the mean over the standard deviation of the measured signal.

Another way of calculating SNR is found at http://rx.linkfanel.net/snr.html .
Quote from the KiwiSDR  forum :
"Auto Scale" button sets WF max/min by taking one waterfall result, sorting the resulting FFT bins, calling the 50% bin the noise level and the 98% bin the signal level. Then WF min is set to the noise level -10 dB and WF max the signal level +30 dB. Works best when zoomed out so there is a variety of signal levels. The sort/bin idea is from Pierre, who uses a similar idea for SNR scoring at http://rx.linkfanel.net and http://rx.linkfanel.net/snr.html
End Quote

I believe this method is also used to give the reported dB value for WSJT modes. Of course the -10 and + 30 dB are not needed for this purpose. Would be interesting to see that as a time plot together with S-meter readings and look under various QRN and QRN conditions at channels with voice and data transmissions to see if this is a useful to quantify what we hear and perceive as "better audio quality". I guess the kiwiclient is the place to add a SNR computation.


b. Scanning over a limited number of kiwi sites and frequencies
I find myself entering a number of frequencies in the GUI of your directKiwi project and then looking at a number of different kiwis to see how they compare.

Actually if this was automated that would be really nice. Something I was thinking of is have the user enter a list of say 10 frequencies, then select from the list of 200+ kiwis his/her favourite ones (perhaps also 10) to monitor. Scanning thru this table will give a dynamically updated view of the "best" kiwi site and frequency combination. Scanning could be sequential per kiwi ip address or perhaps in parallel with multiple web sockets open at the same time.

A 10 by 10 matrix having for instance frequencies as columns and host kiwi as row could display the latest SNR value. When enhanced with some colour coding akin to a heat map this would look quite very cool. The "best" selection of the moment could further be used to provide audio.
Using kiwclient to interrogate remote kiwis via the internet involves of course some time delay so a realistic refresh rate for such a matrix is probably 30-60 seconds.

Anyway just throwing around some ideas for this already nice directKiwi program.


Best regards, Ben

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #7 on: June 02, 2018, 0759 UTC »
Hi Ben

thanks for the ideas

a: I don't think i can do this, as the SNR calculation, if i understand correctely is based on a wide display waterfall analysis and "github.com/dev-zzo/kiwiclient" is unable to get a waterfall.
What i can do is getting from linkfanel or marco (IS0KYB) already calculated SNRs and putting the data in the node listing ?
Anyway i wonder if a SNR calculation based on a fixed bandwidth (max 6kHz) demodulated SSB frequency is easy to do and reliable, i mean it should be difficult to compare the desired AF signals strength (it can be anything, not just like WSPR sigs) with the noise background in such a narrow BW..

b: scanning different remotes for various frequencies in an automated way should be codable... but if SNR can't be calculated, we'll just get s-meter info and (recorded?!) audio, so just a s-meter graph or table as result.. not sure if it can be useful.
BTW, i already planned to add a "favorite" system in the node listing, with checkboxes and basic filtering.
Also if i get the already calculated SNR values of each node as mentionned in the "a" answer, I can also filter the worst remotes from the listing, by adding the setting of "do not display remotes that has SNR below than: " but remember the node listing is not dynamic.


ISSUE: Recently i have noticed the listing had mixed countries and hosts. It is due to a specific remote located in India, for some reasons this node is not displayed correctly in kiwiSDR.com/public summary page (due to missing informations)
That make my update process completely failed, by mixing up the .db with false datas.....

I have fixed that issue in the 3.63 version, by computing that specific node informations manually. That new version is available on the project webpage : http://81.93.247.141/~linkz/directKiwi/


Note: I'm working on adding a real RECORDING button in the GUI. In the early versions I have already had a plugin that can record, but it was just based on pygame and can cause different issues depending on the user client PC settings (mainly the capture device setting) so i have removed it.
Remember this project begun by using "github.com/dev-zzo/kiwiclient" stuff in console, script was originally capable writing streams into a .wav file. I have just modified one or two lines of the code to play the audio instead of writing the .wav
So it will not be difficult to come back to the original behaviour, but simultaneously with audio output in speakers


&linkz



Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline benji

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: directKiwi project
« Reply #8 on: June 04, 2018, 0555 UTC »
Hi Linkz,

I noted the addition of microkiwi_waterfall.py to Christoph's kiwiclient.git which indeed takes a snapshot of a specified host kiwi. With default arguments it shows the calculation result over the whole 30 Mhz range but can be narrowed to a high zoomlevel. Locally I use it for manually selecting a suitable antenna for a chunk of the spectrum. This "global" SNR gives a pretty good indication for SNR at a certain kiwi like Marco shows at http://sibamanna.duckdns.org/sdr_map.html.

Another indication of SNR could be the dB value of signal mean over standard deviation just for the passband. The idea of suggesting a measure of SNR is an attempt to avoid kiwis with high RSSI readings due high local QRN or QRM end up on top of the list.

As you mentioned for voice signals within the passband of the selected mode it remains to be seen if there is any benefit of having such an SNR measure. Probably best way to find out is to plot this together with RSSI in a real time graph.

To go a step further I guess for digital signals measured BER would be a better qualifier. For speech SNR is not well defined since the power spectral density of a voice signal is highly variable with time and almost as random as the noise itself. There are routines trying to separate speech from noise first such as using VAD (voice activity detection) or syllabic rate detection and then estimate SNR. However this is probably overkill for the present project.

Anyway thank you for coming back to my suggestions and a filter for to exclude the worst remotes will already be a good addition.

Best regards, Ben.

Offline benson

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: directKiwi project
« Reply #9 on: February 26, 2019, 1349 UTC »
Hello Linkz,

Just to let you know that I enjoy using the new version V4.0 of your DirectKiwi program.  The graphical display makes it even easier to compare and browse around the available Kiwis.

I installed it directly in Python running on a windows machine and one thing I noted is that the square icon for the the nodes is a bit small and sometimes points on the coastline of the base map look quite similar. Perhaps a larger or different symbol might help. I am aware that making it too large will cause trouble in areas with high density of Kiwis. I tried different background maps and symbol colours but not much improvement.

For the "nice to have list" a request to cycle manually through the list of favourites nodes.
When this would be done automatically at an n second interval a mean RSSI for each node could be shown....Anyway thank you for making this intuitive and useful GUI available.

Best regards,Ben.

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #10 on: March 16, 2019, 2148 UTC »
Hi Ben

Thanks for the feedback again, the only ones i got so far, in fact I have no idea how many people use the software  :P

For Windows you can enable map zooming by uncommenting line 443 of the python code
Hope it will be better for you but it's a bit bugged as the mouse wheel will also act for the embedded console output.. ah Windows..

I'm not really decided to work on directKiwi anymore, my focus is/was on directTDoA project nowadays (version 4.20 is almost at its best)
BTW, the GUI map display is originally from this project so I've decided to merge some of the codes in directKiwi to make it better.

Regards

&linkz



Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #11 on: June 10, 2019, 1953 UTC »
Hi all

I have released the v5.00 version of directKiwi python GUI code today

Some months ago, I've told you I will not work on the code anymore but finally I've done some modifications to fit my needs :P

Main changes:
* No more s-meter, yes.. no more. No real interest to me and consuming CPU time for nothing.
* The switch between nodes is now faster as you just have to left-click on the square icons to move from node to node.
* It's still possible to get KiwiSDR informations, connect via browser, check SNR, add/rem favorite/blacklist. To do so, just right-click on node icons.
* The code has been cleaned, commented and some parts were totally re-written.
* The directKiwi.cfg file is now in json format. Note that if you already have a filled config file from old GUI versions (<= v4.00), I've add some code to convert it for v5.00 compatibility, just backup & copy your .cfg in the new v5.00 directory. You'll keep your previous settings then.
* Audio parameters (LP filter/HP filter/AGC/MGC/gain/threshold/slope/decay/hang) are now shown in an unique Tk window.
* You can now change the map icon size
* Map zoom is still desactivated but you can still uncomment lines 379 (for Windows) or 380,381 ( for Linux/MacOS) manually to allow that functionnality if needed.

http://81.93.247.141/~linkz/directKiwi/
https://github.com/llinkz/directKiwi


&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #12 on: November 09, 2019, 2305 UTC »
FYI

directKiwi 6.0 is out.

no more pygame & scipy (for MacOS), sounddevice python module + libsamplerate instead -> works with 20kHz KiwiSDRs + no audio compression set by default
IS0KYB microkiwi_waterfall script added (SNR measurement + waterfall display), IS0KYB SNR website source is not used anymore + code clean up



http://81.93.247.141/~linkz/directKiwi/
https://github.com/llinkz/directKiwi


&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)

Offline Josh

  • DXing Phenomena
  • *******
  • Posts: 4322
    • View Profile
Re: directKiwi project
« Reply #13 on: November 09, 2019, 2324 UTC »
Perhaps a mode or stand alone one click app to tdoa in keeping with the minimum needed to do the job?
We do not encourage any radio operations contrary to regulations.

Offline linkz

  • Jr. Member
  • **
  • Posts: 54
  • French Alps
    • View Profile
    • utilities & digimodes by linkz
    • Email
Re: directKiwi project
« Reply #14 on: November 10, 2019, 0108 UTC »
Done... v6.10

using ?ext=tdoa,lat:yyy,lon:xxx,z:5,<submitted node> URL parameter

Note: The submitted node function will probably not working all the time as I'm not using the same algorithm to define empty "tdoa_id" fields from the nodes


&linkz
Grenoble area, French Alps (JN25)
FDM-S2 / Perseus / JRC NRD-535 / R&S EK 070 / CSF RS560 / KiwiSDR / Codan 9360 / Codan NGT SR
Skywire horizontal loop / T6 LPDA / Wellbrook ALA1530LNP / Cushcraft R8
also listening HF from the mobile w/ Barrett 950 & whips
active on IRC (#wunclub)