Check out the SDR forum on radioreference.com...I was going through the same teething pains. Try tuning to whatever freq your rtlsdr sees as the control channel (and without a -d or -q offset) and see if you start pulling in talkgroups (audio or not). What I did from there was a lot of trial and error with the control freq in trunk.tsv....I’d try fine tuning in the op25 program and then
punch that corrected freq into trunk.tsv as the control freq.For the constellation plot (‘2’) apparently you’re trying to minimize the spread of the data points within each of the 4 sections. I THINK what you want in the datascope (‘4’) is 6 very distinct and clear openings in the data stream....this isn’t perfect but gives you some idea of what I’m talking about...
https://goo.gl/images/q453W6In short...tuning is a lot of trial and error!!!!
UPDATE: apparently you should put the actual control freq. from the RR database into the trunk.tsv file, and then correct any RTLSDR tuning errors within OP25 or with the -q and -d options. Not exactly sure why....
Also, here's a screencap from gnuplot on a successful tuning effort...