Sunday, September 25, 2016

Hojo and the case of the garbled WSPR packet

Hojo and the case of the garbled WSPR packet

Problem

After implementing my third telemetry packet, I found that data was getting garbled in my WSPR packets.  Gathering some data, It appeared that the garbling was occurring as follows:

Expected Data Received Data 
QE6IAB
QE6IAB (correct)
Q86IAB
Q8AIA (garbled)
Q26IAB
Q2AIA (garbled)


Diagnostics

False start

I spent a lot of time assuming that this problem was in my data, and that somehow my unsigned integer data got converted to signed integer, and the arithmetic to convert the integers to characters was garbling the string.  After a bunch of debugging, I did make one minor code change that would have dealt with that situation, but I wasn't confident I had nailed it.

Conclusive data


While testing the fix above, I was confident that my code should now generate clean telemetry strings, even if I didn't exactly understand the failure scenario that caused the problem in the first place..  I had run through every scenario I could think of with both good and bad data, and the strings were being created correctly.  I had moved on to other testing in preparation for flight, when I happened to notice that the data, as received by WSPR on my radio, was NOT what my program intended to send.  In fact, the data in the tracker was correct, but it was incorrect when received and decoded by the radio.  Ah hah!!!  The problem was not in the encoding of my ASCII string.  It was lower down, in the WSPR encoding routines!

I ran some diagnostics, and confirmed the problem.  I generate a "tone array" based on the characters to send.  Here's a sample of the tone array, as generated by my code, and what the tone array should have been, when running the "wsprcode.exe" program, shipped for Windows.

From my code:
Building Tone array for Q36FAA  FN12  60
3 1 2 0 2 2 2 2 1 0 0 0 3 1 3 2 2 0 3 0 2 1 0 3 3 1 3 0 2 0 
0 2 0 0 3 0 2 3 2 1 0 0 2 0 2 2 1 0 3 3 2 0 1 1 2 1 2 2 0 3 
3 2 3 0 2 0 2 3 1 2 3 2 1 2 1 2 3 0 0 1 0 0 1 2 1 3 0 2 2 3 
1 2 1 0 3 0 2 2 3 0 2 0 2 2 3 2 0 1 2 0 3 3 3 2 3 3 2 0 1 3 
0 1 0 0 0 1 1 1 2 0 0 2 2 1 2 1 0 0 3 1 2 0 0 2 2 0 0 3 1 2 
1 0 3 3 0 0 2 3 1 0 2 0
From wsprcode.exe:
Channel symbols:
      3 1 0 0 2 0 2 2 3 0 2 0 3 1 1 2 2 0 3 0 2 3 0 3 1 3 3 0 0 2
      2 2 2 2 1 0 0 1 0 1 2 2 2 0 0 0 3 0 1 1 0 0 1 1 0 1 0 0 2 3
      3 2 1 0 0 0 0 3 3 2 3 2 3 0 3 2 1 2 0 1 2 0 3 2 3 1 2 2 2 1
      3 2 1 0 3 0 0 0 1 0 2 2 2 2 1 0 2 1 2 2 1 3 1 2 1 3 0 2 1 3
      0 3 2 0 2 1 1 1 2 2 0 2 0 1 0 1 0 0 1 1 2 0 2 2 0 0 0 3 3 0
      1 0 3 3 0 0 0 3 1 0 2 0
They don't match!

 I had gotten a wspr encoding subroutine from a blog post by Mark VandeWettering (K6HX).  I love the code because it's really lean.  There are other routines out there, but they're using malloc() and free() calls and doing all kinds of whacky stuff.  Mark's routine was short, elegant, and brutally efficient.

The comments at the code made it very clear what the problem might be, once I realized it was an encoding issue:

/* Callsigns must be 2x3, 1x3, 2x1, or 1x2 for the purposes of this code */

Therein lies the rub.   "2x3" means "Two characters before the digit, and three after."  So, a telemetry callsign like "QE6IAB" fits that beautifully.  However, a telemetry callsign like "Q86IAB" is ambiguous.  I needed to read his code.

The Fix


Looking at the code, he was basically right-adjusting the string into an array, forcing the digit to be in column 3 (array index 2).


if (isdigit(callsign[1])) {
 /* 1x callsigns... */
 for (i=0; i<strlen(callsign); i++)
    call[1+i] = callsign[i] ;
    } else if (isdigit(callsign[2])) {
 /* 2x callsigns... */
 for (i=0; i<strlen(callsign); i++)
    call[i] = callsign[i] ;
    } else {
 return 0 ;
    }

This code was misinterpreting the callsign, in the event of two consecutive digits.   A quick and dirty fix was this:

if ( (isdigit(callsign[1])) && (!isdigit(callsign[2])) ) {
 /* 1x callsigns... */
 for (i=0; i<strlen(callsign); i++)
    call[1+i] = callsign[i] ;
    } else if (isdigit(callsign[2])) {
 /* 2x callsigns... */
 for (i=0; i<strlen(callsign); i++)
    call[i] = callsign[i] ;
    } else {
 return 0 ;
    }

Conclusion

That one-liner fixed my problem, and my telemetry callsigns are now coming in beautifully in my current flight.

| 2016-09-25 14:04 | KD2EAT   | 14.097061 | IN29 | +33 | DC5AL-R   | 20m  |
| 2016-09-25 14:06 | QQ6LFT   | 14.097055 | IN29 | +53 | DG7BBP    | 20m  |
| 2016-09-25 14:08 | Q46LAB   | 14.097060 | IN29 | +60 | DC5AL-R   | 20m  |

I'm grateful to Mark for sharing his code so generously on github for the benefit of the ham community!

Saturday, September 24, 2016

Flight Wisp1c_11

Flight Wisp1c_11


Planned Changes

  • Same basic configuration as Flight Wisp1c_10.
  • Implemented a code fix (wisp1g code base) correcting a telemetry packet problem which occurred when the second and third character of the telemetry packet were both numeric.

Preparation


Item Weight (g)
Tracker
13.4
Tape and String attaching balloons
2.3
Free Lift
7.4
Total
21.7


My Hydrogen fill rig, out behind my shed.  Note the long adapter between the tank and the regulator.  That converts from the reverse-threaded connector on the tank, to the standard threads on the regulator that I also use for Helium.
Late night build.  I was rushing to prepare this tracker for the Hamfest.  I was in the office until nearly 1:30am fixing code bugs, and prepping this tracker for flight.  All set, here.  The tracker came out weighing exactly the same as the previous flight - 13.4g.

Day 1 - 9/24/2016


Launched at 10:15am local time at the Chemung County Fairgrounds, Elmira New York, on the occasion of the Elmira Hamfest.


About to launch the balloons.  Note, I'm holding the balloons by the tether, which has the counterpoise attached to it.  This can be risky, as the antenna is just 36ga magnet wire.  I'm careful to assure that the tether is shorter than the antenna wire, so that if it's tugged, it won't stress the antenna.  I had to hold it for a few moments until the breeze let up.  You can see the balloons being pulled downwind.  When I released the tracker, it swung perilously close to the ground before gently rising.  I was concerned we might run into parked cars, but fortunately, the winds cooperated and it took an "up elevator" pretty quickly, clearing all the obstacles.
There they go!  There was one fairly hefty cloud overhead, but the skies were overall pretty clear, thankfully!






The tracker did well throughout the day.  Final position is there on the map.  It averaged 93 mph and about 10,777 meters..  The hysplit prediction shows some weather ahead.  Let's hope we make it by.  Some of that weather is VERY high.
Reception throughout the day was AWESOME. Here's all the sites that received the tracker today.

Tracking:  http://aprs.fi/#!mt=roadmap&z=11&kml=http%3A%2F%2Fthehojos.com%2F~mqh1%2Fwisp1c_11_1.kml&call=a%2FKD2EAT-11&timerange=604800&tail=604800


Day 2 - 9/25/2016

It's alive and hauled ass overnight, averaging 164 mph as it covered 1841 miles.   It slowed down a little during the day, but still averaged 139 mph as it covered another 1182 miles.


At the end of the day, the hysplit predictions looked really good to continue east.  The downside is that the predictions indicate slower wind speeds.



WSPR coverage was excellent.  The lost packets we had during the day were restricted to the morning and afternoon, when the supercap was not fully charged.

Additional telemetry - observations on bootups

The additional telemetry received today was very interesting.  The last few packets from yesterday and the first from this morning.

| 2016-09-24 21:42 | Q86DAB   | 14.097046 | FM79 | +60 | K9AN    | 20m  |
| 2016-09-24 21:48 | KD2EAT   | 14.097046 | FM79 | +33 | K9AN    | 20m  |
| 2016-09-25 08:54 | KD2EAT   | 14.097059 | HO41 | +33 | PI4THT  | 20m  |
| 2016-09-25 08:56 | QC6CIF   | 14.097059 | HO41 | +43 | DC5AL-R | 20m  |
| 2016-09-25 08:58 | QA6LAB   | 14.097043 | HO41 | +60 | G4CPD   | 20m  |


Yesterdays Q86DAB:

  • "8" - 8 transmit cycles
  • "D" - 3  boots
  • "A" - 0 watchdog reests
  • "B" - 1 GPS reset

This mornings "QA6LAB":

  • "A" - 10 transmit cycles
  • "L" - 11 boots
  • "A" - 0 watchdog resets
  • "B" - 1 GPS reset
The two instances of KD2EAT received account for the transmit cycles incrementing from 8 to 10. That lines up.

The number of boots incremented from 3 to 11.  That's quite a few times that the code started up, but the tracker didn't stay up long enough to transmit anything.  I presume these happened both at shutdown time, and boot in the morning.  

I think this data speaks to the fact that I need to do more work to stabilize the tracker in low solar situations.  I have code that holds the tracker in a lower current loop until the supercap voltage is over 3500 millivolts.  For the next flight, I'll try increasing that a bit.   Another option might be to enforce a hard-coded delay after first boot of the day, to allow the sun to come up a bit more.


Day 3 - 9/26/2016



Day 3 went pretty smoothly in terms of flight. The altitude and speed was pretty consistent. The only real problem was with the GPS.  It was having trouble with locks, and my software had to reset it 14 times throughout the day.  I suspect it'll be OK in the morning, but this speaks to a need to put a more complete power off / reset into the next iteration of my board.


The prediction still looks awesome for the coming days. All models show it going straight across Asia.  This does raise some concerns, as Turkey has some very heavy air traffic near Istanbul.
a
This model depicts the air traffic over Turkey, and the potential routes for the balloon.  Let's hope we keep to a more southerly trajectory and stay away from all the planes!



WSPR reception was extraordinary today, with one station in Australia hearing several of the packets.  Statistics are below in the summary table.






Day 4 - 9/27/2016

We awoke and safely traversed the Turkish airspace.  The tracker picked up quite a lot of speed overnight, getting back to over 100mph during the day.  It averaged around 10,555 meters in altitude -  a bit lower than previous, but still quite serviceable.

We ended the day in Georgia.  All systems looked nominal.  Interestingly, after days of flight, we ended the day only 50km off the original prediction line!  Wow!

The GPS was still a bit wonky during the day. I'll need to work on better GPS resetting code.  The good news is that we always recovered!

The prediction for the coming days still looks awesome.  Straight across!


WSPR reception wasn't as good today, but still perfectly adequate.  There's no indication that any packets were dropped other than at boot up and power down time.

Day 5 - 9/28/2016



Day 5 was a scary one.  Packets didn't come in until very late in the daylight cycle.  Only a total of 11 packets were received all day, as compared to 900+ on previous days.  Decoding the packets, however, showed the tracker to be in excellent health and floating at 10,777 meters (up from the previous few days).



This station is western Russia was one of the two stations that received packets today.





This Japanese station was the other one receiving packets.    With luck, propagation will be better tomorrow.

Mark Conner, N9XTN, was kind enough to provide this ensemble prediction for the coming days.  It looks a bit grim.  The tracker may turn and end up looping around northern Russia.  Here's hoping we take one of the few tracks continuing east back to the US, to complete my circumnavigation!

Day 6 - 9/29/2016

Day 6 was another poor one for received packets. Still, we got enough to document forward progress.  The tracker seems healthy, and is maintaining about 10,666m altitude.


The predictions are going downhill.  Habhub shows the hysplit as hooking back toward Russia.  Bummer!


Mark / N9XTN ran up this model.  At least this one shows a few paths coming back toward North America!

We had a few Russian stations receiving us today.  That was helpful!  Still, pretty poor reception.



Day 7 - 9/29/2016 - 9/30/2016

We awoke in Inner Mongolia.  Reception remained poor throughout the day.

Another issue was with GPS resets.  Somewhere between Day 6 and Day 7, the global GPS reset counter went from 5 to 26.  Since the counter wraps at 26, it's conceivable some bug reset the counter to 0, but it's also possible that we had 21 resets.  I'm not sure what happened there.  The data is so sparse it's difficult to tell.  Altitudes were pretty good, at 10,666 all day.




There are now more predictions showing us heading into the US, and fewer back to Russia. That's good news!

Very few WSPR receiving stations, and very few packets received, as seen in the statistics at the bottom of this post.

Day 8 - 9/30/2016 - 10/1/2016

We awoke on Day 8 off the coast of Japan, moving along quite nicely.  We received few spots during the flight day, but all indicated that things were well.  The tracker was a little low, averaging about 10,333 meters in the spots we received, but it averaged 129 mph during the day.  It covered an amazing 1927 miles in the night before waking up, and another 846 during daylight.

The predictions have improved greatly, showing us coming into the states, with a predicted landfall in California on Monday.  Yay!

We had very few receiving stations throughout the day, but we had one heroic receive from Australia, and Alaska starting hearing us with regularity.

Day 9 - 10/1/2016

I awoke to discover that the predictions took a HORRIBLE turn.  All indications were now pointing toward a left turn back to Russia.

Sure enough, when the tracker awoke, it was dead on the prediction, and seemingly headed for a left turn.

I reran the predictions on predict.habhub,org, and it shows a loop-de-loop in the Kara sea, EXACTLY where my previous flight disappeared.  Wunnerful.

The weather overlay showed an explanation for the problem,  There's a big front right there.  Apparently, those clouds weren't very high, as they didn't seem to interfere with the tracker at all.  Let's hope it stays that way!

WSPR reception was pretty good today.  The Alaska stations were receiving with great consistency.  Japan and Australia were still helping out.

Let's hope we, at least, fly swiftly on our reprise of Russia.


Day 10 - 10/2/2016


No packets were received from the tracker today.  It was expected to make a late start (more than 24 hours), due to its projected rapid westward progress, but simply never checked in.   It's entirely possible it's not being heard due to its extreme distance from receiving stations.  We'll have to wait and see for a few days.



Statistics


Day Rcvd / Lost
Pkts
Pct
Rcvd
Late GPS locks GPS Resets Watch
dog Resets
Farthest Rcv Station (miles) Average stations / beacon
Max / Beacon
Start Grid / Stop Grid Dist covered Night / DayMPH Night  / DayAvg Float Altitude (m)
1
92 / 4
95.8
9
1
0
3,857
10.0 / 24
FN12od / FM79ad
0 / 604
0 / 92
10,888
2
114 / 8
93.4
6
0
0
3,720
11.9 / 23
HO41ci  / IN66rq
1841 / 1182
164 / 139
10,777
3
107 / 3
97.2
17
14
0
17,010
19.5 / 50
JN01ne /  JM45ji 
538 / 579
42 / 56
10,777
4
106 / 4
96.3
10
3
0
3,587
9.0 / 17
KM36tf / LN12mc
1056/ 918
82/ 110
10,555
5
10/ 28
26.3
1
0
0
3,144
0.4 / 2
MN91jr / NN01jt 
1825/ 103
98 / 69
10,777
6
25/ 27
48.1
2
0
0
4,156
0.6 / 3
NN81mv / ON02ta 
1096 / 236
46 / 75
10,666
7
18 / 57
24.0
6
0
0
2,270
0.4 / 3
ON72cd   /  PN23nl
645 / 561
52 / 99
10,666
8
41 / 68
37.6
5
0
0
6,106
0.5 / 3
RN13wi  / AN01dl
1927 / 846
123 / 129
10,333
9
62 / 9
87.3
1
1
0
1,561
1.7 / 3
BO37kk / BP01ft
1606 / 376
122 / 79
10,444
10
xx / xx
xx
x
x
x
xx
x / x
xxxxxx / xxxxxx
x / x
x / x
x

Wednesday, September 14, 2016

Flight Wisp1c_10

Flight Wisp1c_10

Planned changes

  • Two 36" Qualatex envelopes.  Unstretched.  Using Hydrogen this time.
  • Transmit frequencies: 20 meter band only.  10 minute intervals.
  • New code version (wisp1k).  Additional telemetry is now stored in EEprom.  A secondary telemetry packet will be sent hourly.  The data in the secondary packet includes total number of transmit cycles, number of tracker boot-ups, number of watchdog resets, and number of GPS resets.
  • Gatewayed to APRS at kd2eat-10.  The secondary telemetry packet data will not be sent to APRS, but can be manually decoded for diagnostic purposes.
  • Antenna built the same as Wisp1c_9, but additional tether added between the end of the counterpoise and balloon, to reduce potential interference.
  • VCC on the board is reconfigured to 3.3 volts (from 2.5 and 2.7 on previous flights) to maximize transmit power.

Preparation

As per the previous flights.  Nothing particularly warranting new photos.


Item Weight (g)
Tracker
13.4
Tape and String attaching balloons
2.3
Free Lift
7.4
Total
21.7








Preflight prediction

Looks pretty good:

Day 1 - 9/15/2016

It was a glorious day to launch from my building!








Here are pictures of a pair of balloons in the same configuration as I launched today.  I just didn't happen take a picture of today's balloons.  Today's were filled with hydrogen, and looked a little less full than these.





Here's the tracker, getting it's first lock and doing a transmit test prior to launch.  Note the blue LED illuminated on the left side.  That's the transmit light.








And it's off!!!!  The winds were calm.  It went basically straight up!   You can see the tracker dangling below, and if you look carefully, the antenna hanging below the tracker.

It flew 370 miles, starting in grid square FN12sk and ending in FN51hl.  More statistics are in the table at the bottom of the blog.




The altitude with the hydrogen was quite nice.  It floated comfortably around 10,333 meters, peaking at 10,666 a few times.

The GPS still had a few late locks, but none missed and no resets to speak of.  There IS a problem with the new telemetry packet.  They were sent on schedule, but there seems to be a problem which is causing the telemetry callsign to be garbled at times.  I'll have to investigate it more.

The antenna performed VERY well.  Below is a map of the stations that received it today.




Predictions for the future still look pretty consistent.  It's going to loop around Europe, if it makes it there.   The weather ahead looks clear, so barring any problems, I'm optimistic that it'll be there in the morning.



Day 2 - 9/16/2016


The tracker awoke on schedule in grid GNOO0b, having traveled 562 miles overnight.  We ended the day in grid GN61ek having traveled 592 miles by light of day.  Very nice!

The altitude held steady at about 10,777 meters throughout the day.









APRS coverage was great throughout the day.  Here's a map of the stations that received the tracker today.



The prediction for the coming days has improved.  It looks like we may not loop-de-loop after all.  It's hard to be sure, but at least this hysplit model shows it curving out to the southeast, instead of back northwest!
I tagged the bubble that should be about where we wake up in the morning.


GPS reset

We did have one GPS reset today, and the new telemetry packet confirmed it.  The transmit sequences should be 10 minutes apart.  When we're late, it'll be some multiple of 2 minutes later.  The GPS is reset if we can't get a lock for 5 consecutive minutes.  Well, one of the transmissions was six minutes late.  That indicated that the GPS should have been reset.  Sure 'nuff, when the next secondary telemetry packet came out, on the hour, it showed that a GPS reset had occurred.  Hurray for the new functionality.  There's still a glitch causing some garbled data, but I'll track that down before the next flight.

Day 3 - 9/17/2016

The tracker woke up on schedule at around 4:14am Eastern in grid square HO61li, having traveled 1190 miles overnight.  It was really moving fast today, averaging over 100mph for the day.  We ended in grid square IP26qb after traveling 1105 miles in daylight.  It was a little lower all day, around starting at around 10,444 meters, and gradually sinking to 10,222 meters near the end of the day.  I'm hoping this is due to the high latitude, and not a leak.


The tracker was received very well throughout the day.  Here's a map of the receiving stations during the 12 hour period of activity.


The hysplit predictions are improving.  Many of them now show the payload traveling east, rather than doing a loop-de-loop.  Yay!



Day 4 - 9/18/2016



Day 4 went OK.  The tracker awoke on schedule around 2:16am Eastern in grid square KQ18oq, having traveled 1136 miles overnight.  It had slowed down significantly, moving at about 59 mph through the short day.  It only beaconed for about 5 1/2 hours before it the sun was too low on the horizon.  It's WAY up north, and the solar cells don't work so well.  It ended the day in grid square LQ39xc, some 328 miles traveled by day. The good news was that the altitude was up again.  Most altitude reports were at 10,666 meters.



The ensemble predictions for the trip are looking better.  It's most likely to continue east, now, rather than looping around Scandinavia.

Given the short day, I'm guessing it'll be about 2AM UTC (10pm EST) before we see the tracker again.




Propagation was poor today.  The receiving stations are shown at the left. We averaged 3.3 receiving stations per beacon, and had the worst packet reception percentage to date, at 85%.

Day 5 - 9/19/2016


It was a slow day today.  The tracker woke up in grid square MQ36gs, having traveled only 311 miles overnight.  The tracker operated through a very short day cycle - less than 5 hours - due to the angle of the sun.  It ended in grid square MQ67lj, just 108 miles later, averaging 18 mph. Altitude held strong at 10,666 meters for the bulk of the day,.




The predictions look favorable for getting pretty well across Russia in the coming days.  Predictions later in the day showed the curves turning east, which is awesome!



We had very few receiving stations today, though it was gratifying to see that we were received in Japan.  Let's hope that trend continues!   Overall, receive statistics for the day were pretty good, given the low number of average stations receiving packets!


Day 6 - 9/20/2016

It's "hide and seek" day for the tracker.  No packets received during its short transmission window. Remaining optimisitc that it's just due to the short daylight cycle and extreme distance from receiving stations.

Final Status

The tracker was not heard from after day 5.  It's unclear what caused the failure.

Statistics


Day Rcvd
Pkts
Lost Pkts Percent
Rcvd
Late GPS locks GPS Resets Watch
dog Resets
Farthest Rcv Station (miles) Average stations / beacon Max stations / beaconDaylight Dist
covered
(miles)
Avg MPH
1
118
4
96.7
4
0
0
3,738
7.5
15
370
37
2
110
10
91.7
5
1
0
7,986
7.4
18
592
57
3
120
8
93.8
2
0
0
9,483
15.2
25
1105
112
4
64
11
85.3
1
0
0
2,781
3.3
12
328
59
5
58
12
82.9
1
0
0
3,554
1.5
5
108
18
6
0










7











8











9