This post introduces a new feature to wifibroadcast: Forward error correction. This helps lowering the data rate and/or increasing the reliability of the transmission.
Early after I published wifibroadcast there were suggestions on improving the forward error correction (FEC) mechanism. While I agreed on these suggestions, my primary goal was to first create something that is usable to see if the idea behind wifibroadcast is worth following. After some months of testing and dozens of other people having also great experiences with wifibroadcast I was confident enough to start to “improve” things.
Initially I implemented a simple retransmission. Each packet got sent several times, increasing the chance that at least one packet got through. This worked sufficiently well but had some issues:
- Bandwidth: The only parameter to increase the reliability was the retransmission rate. And this rate basically multiplied the net bandwidth. This is quite obvious: If you transmit each packet three times, you need three times the bandwidth.
- Relieability: Although it seems unlikely that all N retransmissions of the same packet are lost that phenomenon existed. The explanation is quite simple: If you roll a dice often enough, the likelihood of some strange combinations happening needs to be considered. For example, if you roll a million times, you could expect to see the sequence “1”-“2″-“3″-“4″ nearly a thousandth times. The same is true for wifibroadcast. We are sending a lot of packets so unlikely things can easily happen.
Some commentators suggested to take a look at udpcast, a tool that is able to transfer data over a lossy unidirectional link. And well, that is exactly what wifibroadcast does! The FEC code there was well written and easy to adopt for my use case. In addition, the code is fast enough to run on a Raspberry PI A+. In the code Vandermonde matrices are used to perform the FEC.
The idea behind the code is very simple. Your data has to be divided into packets and a fixed number of them are grouped together to form a block. Lets assume we have a block size of 8 packets. The FEC code now allows you to calculate an arbitrary number of FEC packets (that have the same size as the DATA packets). Lets assume we use 4 FEC packets. If now a DATA packet gets lost, it can be replaced by any of the 4 FEC packets. So we might be able to repair up to 4 DATA packets, regardless which ones of the 8 are affected. This leads to two important advantages:
- Bandwidth: Since we used in the example above 4 FEC packets per 8 DATA packets, the bandwidth requirements are +50%. This could not have been archived with retransmission. There the “smallest” bandwidth with redundancy would be +100%
- Reliability: Since any DATA packet can be repair with any FEC packet, the chances of corruptions drop significantly. It is no more important which packets are corrupted, only how many.
The new FEC feature has been merged into the default branch of wifibroadcast (together with the new fifo interface introduced here)
To update (both on tx and rx!):
hg pull hg update make clean && make
The basic usage of wifibroadcast has not changed. Only some parameters have now a different meaning:
- -b This parameter describes the number of DATA packets per block. So this parameter has more or less the same meaning as with the retransmission version
- -r This parameter has changed from number of retransmissions to number of FEC packets. Each block will be appended with -r FEC packets.
- -f Packet size. This parameter is identical to the retransmission version in that it defines the size of a packet. The only difference is that packets now cannot be smaller than -f.
- Changing the coding rate while keeping block size constant: Increasing the number of FEC packets increases the reliability of the link at the expense of higher bandwidth. A FEC ratio of 50% is roughly(!) comparable (in terms of reliability) to a retransmission rate of 2. If your block size is 8 DATA packets then this would result in 4 FEC packets.
- Changing the block lengths while keeping the coding rate constant: An increased block length can increase the reliability of a link without increasing the bandwidth. However, this increases the latency since it needs to be waited until a full block of data has been received before the transmission can begin. Why does this increase reliability? Very often, blocks are only mildly corrupted so that not all FEC packets will be needed. However, these unused FEC packets are useless for the following block. By increasing the block size you are also increasing the “range” where the FECs can be applied.
It is also important to note that both tx and rx need to agree on -b, -r and -f. Otherwise the transmission does not work anymore.
How to set the parameters
The values of the parameters depend on your application. There are two things to consider: The coding rate and the block length. The coding rate is defined by the ratio between DATA and FEC packets. The block length is simply given by the -b parameter.
Since with this new FEC feature the bandwidth requirements of wifibroadcast can be lowered quite significantly, new WIFI modulations are now usable. I’ve added a new patched firmware under patches/AR9271/firmware/htc_9271.fw.mcs1 that if being copied to /lib/firmware/htc_9271.fw (on tx) enables the MCS1 modulation. This gives you 13mbit/s air data rate of which roughly net 10mbit/s are usable. So this modulation would be perfect to use if you have a h264 data rate of 6mbit/s with a 8/4 coding rate (resulting in a 9mbit/s bandwidth requirement).
The lower modulation rate should give you a better range if you are flying at high distance or with massive occlusions.
This post shows how to add an overlay with telemetry information onto a video stream
Since the last flyaway of my quad having a telemetry link that transmits GPS coordinates was high on my wish-list. Wifibroadcast was prepared long time ago to transfer a second data stream in parallel to the video data by defining ports. But until now I did not make use of it. You can see a video of the OSD in action here:
This post describes how to set up the OSD display. Additionally, two changes required to wifibroadcast are described before the actual OSD stuff.
My quad uses a Naze32 flight controller that is able to provide FrSky telemetry data over a serial link. The serial port of the Naze32 is connected to a USB2SERIAL converter that is plugged into the tx Raspberry. The telemetry data is then transmitted using wifibroadcast over a second port in parallel to the video. On the receiving Raspberry the telemetry data is received, parsed and drawn onto the screen.
Wifibroadcast minimum packet length
Telemetry data is very different to video data in that it consists of very small packets. For example, most of the FrSky packets are only 5 bytes long. Using the default settings of tx this would lead to a single packet being sent for each small telemetry data unit. This of cause would create a high overhead since the ratio of payload to required header data is very low. To avoid this issue I added a command line parameter -m that can be used to define a minimum payload length. If given, tx waits until it has captured the minimum number of bytes and only then sends a packet.
Several tx instances in parallel
As said above wifibroadcast had the “port” feature for quite some time. The idea was to start several instances of tx, each with a different -p parameter. I used this method for my first telemetry experiments but noticed something odd: Whenever I started a second tx instance in parallel to the video tx process, the video tx was influenced. This was even the case when the second tx did not send any data at all. Quite strange. The result of this was that the video transmission was less fluid and stuttering a bit. Quite a high price for just a bit of telemetry data…
I basically had two options: Trace the cause of this issue in the kernel or adapt tx so that a single instance is able to send several data streams. I went for the latter one because I think it required less effort.
The route I went for was that tx got a new parameter -s that describes the number of parallel streams that shall be transmitted. If this parameter is omitted then tx behaves just like before, transmitting a single stream that is read from standard input. If however two or more streams are requested, tx changes the way it works. It then creates named FIFOs under /tmp. For example, if -s 2 is given, tx creates two named FIFOs called /tmp/fifo0 and /tmp/fifo1. Everything that gets written into fifo0 will be transported over the first port and everything written into fifo1 gets transported over the second port. Quite simple. The actual port number can be influenced by the -p parameter. It serves as an offset. Assuming that -p 100 is given, then fifo0 sends on port 100 and fifo1 on port 101.
The next section will show a usage example of this new mode that should make things clearer.
This feature is still not in the master branch since I had not yet the time to test it well. But after my first successful flight I will merge it into the master.
Putting things together
These commands assume that you have already installed wifibroadcast (as described here).
First, change to the “tx_fifo_input” branch in wifibroadcast:
cd cd wifibroadcast hg pull hg update tx_fifo_input make
Next, start a tx instance in the background with a retransmission block size of 4, a minimum packet size of 64 bytes and two parallel streams:
sudo ./tx -b 4 -m 64 -s 2&
All that is needed is to connect the data sources to the named FIFOs:
sudo su stty -F /dev/ttyUSB0 -imaxbel -opost -isig -icanon -echo -echoe -ixoff -ixon 9600 #set up serial port raspivid -ih -t 0 -w 1280 -h 720 -fps 30 -b 4000000 -n -g 60 -pf high -o - > /tmp/fifo0 & #video cat /dev/ttyUSB0 > /tmp/fifo1 & #telemetry
Now video and telemetry data is transmitted in parallel.
The video reception can be left unchanged (refer to here) and should work out of the box.
To get the OSD working, we first need to check out and build my OSD project:
cd hg clone https://bitbucket.org/befi/frsky_omx_osd cd frsky_omx_osd make
And finally you need to pipe the telemetry data into the OSD viewer (note the -p 1 for the telemetry port):
cd cd wifibroadcast sudo ./rx -p 1 -b 4 wlan0 | /home/pi/frsky_omx_osd/frsky_omx_osd /home/pi/frsky_omx_osd
Of cause, you can also save the telemetry data using the “tee” command.
In case you just want to test the OSD without the whole wifibroadcast stuff there is a FrSky telemetry log included in the repository:
./frsky_omx_osd . < testlog.frsky
The OSD works well for me. But be warned: Things are still a bit hacky around here. I wouldn’t guarantee that my FrSky protocol interpretations are always right. No wonder seeing how that protocol is designed. It is a perfect example of what happens if companies outsource their coding tasks to mental institutions ;)
It should be easy though to replace the frsky.c parser with a different one (also for a different protocol). Volunteers are welcome :)
This post describes a new feature of wifibroadcast: Software diversity.
Update 06/02/1025: The new diversity code from 06/01 worked fine on my desktop pc but showed some rare problems on a Raspberry A+. The TX_RESTART was triggered sometimes by out of order packets (with a block distance of 10 blocks!). Due to that I increased the threshold for the TX_RESTART trigger. The Downside is that the empirical tuning of the -d parameter does not work anymore. So I’ll have to write a tool that determines the -d value automatically…
Update 06/01/2015: I’ve added a new parameter to wifibroadcast for diversity. With -d you can define the number of retransmission blocks that will be kept as a window buffer. The parameter defaults to one (which minimizes latency). However, if you use diversity with wifi adapters of different types, they might deliver their data at different times. The -d parameter should be set so that the window buffer spans over the time difference between the two adapters.
The empirical approach of finding the right setting is simple: If -d is too low, you should see often the message “TX RESTART”. Increase the -d parameter until this message disappears and you have found the correct setting.
An active commenter on this blog, malkauns, has started development on software support for diversity. The general idea is: You use more than one WIFI receiver and combine the received data. Due to multipath and other effects the probability that you receive a single packet correctly increases with each added receiver. Together with malkauns I integrated this feature into the default branch of wifibroadcast.
The usage is extremely simple: Just call wifibroadcast with more WIFI adapters:
./rx wlan0 wlan1
Please note that currently you need two identical adapters. Different types can introduce a delay between the received data streams that currently cannot be processed by wifibroadcast. In this case the systems works but you gain little advantage from diversity.
My first experiments showed that the reception quality increases drastically. I tested the feature in a noisy environment with lots of surrounding networks. In every case the packet drop rate was at least one order of magnitude lower when using two adapters. In 90% of the cases it was even better by two orders of magnitude.
Conclusion and outlook
The results are very promising. I think in the future I’ll use diversity on all of my flights. Since the TL-WN722N is very cheap, this feature should be useful to many adopters. There are still some open issues with diversity that need to be solved.
The support of different adapter types in parallel is one thing that is planned. Also, there is possibly some potential for smarter FEC in this case. Currently the diversity only works on packet basis. It may be a good idea to dive into their content. We’ll see :)
This post presents just another FPV video transmitted using wifibroadcast
After my last crash I rebuilt my quad and tried it out today. And I am getting more and more used to the FPV feeling. I am still a total noob compared to others but there is progress :)
Since I was a bit afraid to crash again I went for maximum safety: triple retransmissions, 5MBPS, 30fps with a keyframe every 15 frames (thus 2Hz). Due to that and a dirty lens the image quality is not optimal. Again, I had the receiver in my pocket and my body occasionally blocked the line of sight. Therefore, there are some disturbed video frames. I really need something to put the antenna on!
Here is the video (recorded at the ground station):
This post presents my first experiences using the wifibroadcast video transmission.
To be honest, this post is a bit contradictory.
The title uses the word “reliable” and later in this post I’ll describe my first three FPV flights where the last one ended in a really bad crash.
Analyzing the cause showed that it was actually my fault, not the transmissions one.
But before I come to the bad part let’s start with the good news which is a range test.
Extended range test
The last range test was a bit limited in terms of range since the area I was in was limited to 500m line of sight (LOS).
This time I had a car available and drove to a small hill where I’ve placed the video sender.
You can see a picture of the valley down the hill here:
I used the ALFA AWUS36NHA with the omni antenna that came with it as a transmitter.
The H264 data rate was 7mbps with a double retransmission.
On the rx side I used the TP-LINK TL-WN722N with my home-made double biquad antenna:
The results were really astonishing.
I drove down the valley all the way to the end right before I lost line of sight.
At this point (at 3km distance to the transmitter) I had rock-solid video image.
Since the double biquad antenna is a directional antenna its sensitivity decreases when the transmitter is not in the center of the receive coil.
And again I was surprised that I could move the antenna +-30° both horizontally and vertically until I lost too many packets.
This is really good news: In case your quad stays in a radius of 3km you are safe to fly withing a 60° sector.
This also indicates that the maximum distance is way beyond 3km.
My guess would be that at least 5km should be possible.
I also tested a dipole antenna at the receiver.
This gave a maximum distance of 900m with the advantage of a 360° coverage.
The end of my last FPV flight
As I have mentioned before my last flight ended quite brutally.
And I have learned a lot from it.
Two main problems have caused that crash: Strong wind (30km/h) and blocked line of sight.
I was flying at a distance of around 200m against the wind.
When I turned around the quad was moving with the wind and suddenly lost height which blocked the LOS between the antennas.
The stream was quickly so disrupted that I could not guess its current position and height.
While I was trying to move the quad back to my position (I did not know that the LOS was blocked due to the height) the strong wind pushed the quad away horizontally.
When I (blindly) increased altitude the quad was already pushed away so far that the LOS was now blocked horizontally due to trees close to me.
I still received data but that was mostly unusable.
Maybe each 5 seconds a good frame blinked quickly and was then disrupted again.
When I lost contact completely I was forced to “land”.
The problem now was: Where is it? I knew that the battery was almost dead so the RC activated beeper would soon be quiet.
I ran around like a mad dog listening for the beeping.
After one hour I gave up and “accepted my loss”.
Back home I converted the received video stream of the crash flight to single images using gstreamer.
This way I was able to recover a more or less intact image 10s before the crash:
This image allowed me to localize the approximate position and finally I have found my quad sitting in a field (luckily the green antenna stood out of the long grass).
The impact took place at a distance of 700m to the rx station.
This means that the wind pushed the quad in 30s around 500m away from me…
Aftermath and conclusions
Although the crash was quite hard all the electronics and motors stayed intact.
Most likely this is a result of the sandwich construction that protects the electronics between two plates.
However, all of the 8mm carbon fiber rods that hold the motors broke quite brutally.
Under normal circumstances these rods are really durable.
So the crash seemed to be quite hard.
My conclusion is that a blocked line of sight using 2.4GHz transmission is deadly.
In the future I’ll take care that my rx antenna has a clear field of view of at least 180°.
Also my first reaction in case of a lost video connection will be to gain altitude.
This incident also motivates me to buy a GPS receiver that transmits the position of the quad and also provides an automatic “return to home” function.
PS: A little cheer up
For not letting this post end with bad feelings I show you the video of my first FPV-only flight.
I shot it using the ALFA AWUS36NHA in the air with a dipole antenna and a TP-LINK TL-WN722N also with a dipole on the ground. The video is decoded and displayed using a raspberry pi as shown here.
The occasional errors in the video are because I had the receiver in my pocket. Due to that my body blocked quite often the line of sight between the two antennas. The images are wobbly because it was quite windy that day (around 20km/h). I know, compared to other FPV videos this is sooo boring. But hey, it was my first flight ;)
In this post I’ll explain my ground station that receives and displays video data using wifibroadcast
My ground station
Let’s start with a picture:
Here you see my ground station. To the left you see my FPV glasses which contain a 7″ HDMI display and a fresnel lens. The enclosure is made out of foam and at the back a strap from ski-goggles is attached. Right next to the “helmet” you see a 3-cell battery that powers the system. On the right you find the Raspberry Pi A+ that has been integrated into a USB hub enclosure (there was enough room to keep the hub :). The Raspberry is powered by a cheap 5V DC/DC converter.
Sample video of a weak link
Following is a video where I have intentionally removed the antenna from the receiving wifi dongle. This shows some of the artifacts you could experience under bad signal conditions. I know, it looks really bad but remember: A lot of bad images and some good images is better than no images at all. And to get bad images like that with an antenna attached you really need a high distance or a solid block of the line of sight.
This post shows how video data transmitted using wifibroadcast can be received and displayed on a Raspberry pi
As I have described here the whole wifibroadcast pipeline has been ported to Android. However, since this is highly device specific it may be of interest for people to have a simple and cheap way to rebuild a mobile device that can receive and display a video stream. As a bonus this also has a lower latency.
The hardware is very simple: I bought a 7″ HDMI display for 42€ and connected it to a Raspberry Pi A+. The Raspberry Pi also uses a TL-WN722N wifi card to receive the wifibroadcast stream. Together with a fresnel lens this display could be easily integrated into self-made FPV glasses (and I am planning to do that).
The software part is equally simple. First I tried to use gstreamer to decode and display the video. Gstreamer is able to use OMX to deliver accelerated decoding of h264 but displaying the content needs to be done outside of OMX. And it seemed as if the CPU was too weak to display the (already decoded) video on the screen. Therefore I looked for an OMX pipeline that goes all the way down to the display. And actually there is already a nearly fitting example program in every Raspbian installation: /opt/vc/src/hello_pi/hello_video
This program reads h264 data from a file and displays it on the screen. Since the whole pipeline is using OMX it is efficient enough to display a full-HD video.
I modified that program so that it reads its video data from stdin and fixed the framerate to 60Hz. This way the incoming data stream should always be processed faster than it arrives.
You can find the modified version of it here:
Just copy the video.c file to /opt/vc/src/hello_pi/hello_video and build the software as follows:
Then you can use the following command to receive video data and display it on the screen:
sudo ./rx -b 8 wlan1 | /opt/vc/src/hello_pi/hello_video/hello_video.bin
I am quite happy with that solution. The latency is as good (if not better) as on my PC. Also, the CPU of the Raspberry is just used by 10%. So there is still enough power for GUI stuff available.
Since the latency on my Android tablet is higher I think I’ll use the Raspberry receiver for my FPV glasses and give the tablet to bystanders. That is also a point where wifibroadcast behaves just like good old analog: Other people can simply join your video stream with their compatible device.