Skip to content

Using Raspberry as a display device for FPV video received via wifibroadcast

April 24, 2015

This post shows how video data transmitted using wifibroadcast can be received and displayed on a Raspberry pi

As I have described here the whole wifibroadcast pipeline has been ported to Android. However, since this is highly device specific it may be of interest for people to have a simple and cheap way to rebuild a mobile device that can receive and display a video stream. As a bonus this also has a lower latency.

Hardware

The hardware is very simple: I bought a 7″ HDMI display for 42€ and connected it to a Raspberry Pi A+. The Raspberry Pi also uses a TL-WN722N wifi card to receive the wifibroadcast stream. Together with a fresnel lens this display could be easily integrated into self-made FPV glasses (and I am planning to do that).

Software

The software part is equally simple. First I tried to use gstreamer to decode and display the video. Gstreamer is able to use OMX to deliver accelerated decoding of h264 but displaying the content needs to be done outside of OMX. And it seemed as if the CPU was too weak to display the (already decoded) video on the screen. Therefore I looked for an OMX pipeline that goes all the way down to the display. And actually there is already a nearly fitting example program in every Raspbian installation: /opt/vc/src/hello_pi/hello_video
This program reads h264 data from a file and displays it on the screen. Since the whole pipeline is using OMX it is efficient enough to display a full-HD video.
I modified that program so that it reads its video data from stdin and fixed the framerate to 60Hz. This way the incoming data stream should always be processed faster than it arrives.
You can find the modified version of it here:

https://bitbucket.org/befi/hello_video

Just copy the video.c file to /opt/vc/src/hello_pi/hello_video and build the software as follows:

/opt/vc/src/hello_pi/rebuild.sh

Then you can use the following command to receive video data and display it on the screen:

sudo ./rx -b 8 wlan1 | /opt/vc/src/hello_pi/hello_video/hello_video.bin

Conclusion

I am quite happy with that solution. The latency is as good (if not better) as on my PC. Also, the CPU of the Raspberry is just used by 10%. So there is still enough power for GUI stuff available.
Since the latency on my Android tablet is higher I think I’ll use the Raspberry receiver for my FPV glasses and give the tablet to bystanders. That is also a point where wifibroadcast behaves just like good old analog: Other people can simply join your video stream with their compatible device.

Advertisements

From → Uncategorized

53 Comments
  1. nicegraham permalink

    Hi, I’m probably missing something basic on this but I’m not receiving anything at all when I run this on a pi http://i.imgur.com/ghw3tOf.png

    Any ideas?

    thanks,
    Graham

  2. nicegraham permalink

    It appears the wlan settings aren’t persisted so I had to redo the monitor/channel etc and I’m now seeing messages on my rx pi.

    DLT_IEEE802_11_RADIO Encap
    Lost a packet! Lossrate: 0.000000 (0 / 1)
    Lost a packet! Lossrate: 0.500000 (1 / 2)
    Lost a packet! Lossrate: 0.666667 (2 / 3)
    Lost a packet! Lossrate: 0.750000 (3 / 4)
    Lost a packet! Lossrate: 0.012270 (4 / 326)
    Lost a packet! Lossrate: 0.015291 (5 / 327)
    Lost a packet! Lossrate: 0.018293 (6 / 328)
    Lost a packet! Lossrate: 0.010294 (7 / 680)
    Lost a packet! Lossrate: 0.003822 (8 / 2093)
    Lost a packet! Lossrate: 0.002663 (9 / 3380)

    Still not getting any video though.

    • Hi

      Is this issue specific to the usage of a raspberry at the receiving end (or does the pipeline run when using a PC as a receiver)? If so, did you apply the fix to wifibroadcast that I described in the post?

      Do you see nothing at all on the screen or is there some rubbish?

      • nicegraham permalink

        I’ve only tried from a pi as the receiver. I’ve managed to to a raw gstreamer stream from the tx pi to my laptop (http://pi.gbaman.info/?p=150). I couldn’t get your code to compile on my macbook.

        Yes, I’ve commented that line and rebuilt. I don’t see anything at all, just the command line output – no window opens or anything.

      • Ok. The messages you saw about lost packets show that we have data flowing from tx to rx which is good. Could you give me your tx command-line as well? This way I could try to reproduce your setup.

        In the meantime you could check if the data transfer works by using wifibroadcast manually:

        On the rx-side: sudo ./rx wlan0 -b 1
        On the tx-side: sudo ./tx wlan0 -b 1

        Then type lines of text into the tx-program and watch if they appear on the rx-program (note that you will need two lines of input to see the first line. The output is always delayed by one line).

  3. nicegraham permalink

    my tx command is:
    raspivid -t 0 -h 720 -w 1280 -fps 25 -hf -b 2000000 -o – | gst-launch-1.0 -v fdsrc ! h264parse ! fdsink | sudo ./tx -r 2 -f 1024 wlan0

    (I have a raspberry pi camera)

    I tried your suggestion and see the messages appearing on the rx side as I type them and hit return – so I it looks like I’m close! 🙂

    • Try to use this as tx command: raspivid -ih -t 0 -w 1280 -h 720 -fps 48 -b 2000000 -n -g 48 -pf high -o – | sudo ./tx -b 8 -r 2 wlan0

      Gstreamer is not needed since we are directly sending out raw h264 data coming directly out of raspivid. The key difference though is to also have the “-b 8” parameter at the tx side. Since this value is not transmitted it needs to match on both sides (refer to https://befinitiv.wordpress.com/2015/02/26/improved-retransmission-scheme-and-multiplexing/ ). Other important parameters are “-ih” for getting SPS/PPS headers all the time (this allows you to start the receiver at any time. Otherwise it must be started before the tx program. Without this flag restarting your receiver would not be possible). The “-g 48” parameter is also important. This gives you a fresh key-frame each second. If there was an error in the received stream an image defect could otherwise survive very long, resulting in an unusable video up to the next key-frame. I don’t know the default value but I think it is 10s or more) Please give me a note if this fixed your problem 🙂

      • nicegraham permalink

        It’s working! I have the rx pi connected to my tv – http://i.imgur.com/tFTj1qs.jpg

        For posterity, my current commands are…
        tx:
        sudo raspivid -ih -t 0 -w 1280 -h 720 -fps 48 -b 2000000 -n -g 48 -pf high -o – | sudo ./tx -p 0 -r 2 -b 16 -f 1024 wlan0

        rx:
        sudo ./rx -p 0 -b 16 wlan0| /opt/vc/src/hello_pi/hello_video/hello_video.bin

        On my setup, movement on the camera introduces a lot of noise. It would be unflyable if mounted to a multicopter. Any thoughts on that?

        Each time I boot the pi I have to reapply the wlan0 settings, do you have to do the same?

        At this point the only difference I have between the setups are the wifi dongles and I have 2 of the ones you recommended on order. I’ll loop back when they arrive and I try them out.

        Thanks for the help!

      • Cool 🙂 I am currently preparing an overview page where all the infos are gathered. This should help other people getting started.

        To your questions: Yes, you have to apply the wifi card settings upon each boot. In my quad I put all of the commands into a script (including the raspivid and tx command). This way I don’t need to take care of that anymore. I also added a service which starts that script automatically.

        Could you provide a sample of the noise you are seeing? It might be that your channel is busy and you are loosing packets. “Lost a packet” messages could be a hint on that. In that case you should move to another channel and/or increase the retransmission count.
        Another effect could be that 2mbit/s is quite low for HD at 48 fps with a 1s key-frame iterval. You could try to lower resolution and framerate and to increase the key-frame interval. The video shown here https://befinitiv.wordpress.com/2015/04/07/details-about-my-flight-setup-and-first-tests-of-wifibroadcast-in-the-air/ was shot with 7mbit/s for example. Just play around a bit with the settings until you have found a good set of parameters.

  4. nicegraham permalink

    I increased to 7mbit/s and saw a good improvement. With more experimentation there was no real noise introduced if I had the pi and camera stationary and created a lot of movement in the cameras field of view, the noise was introduced when moving the whole rig around so something may be a bit loose.

  5. Quick update: I fixed the bug in wifibroadcast which required to comment the line “if (prd.m_nRadiotapFlags & IEEE80211_RADIOTAP_F_FCS)” (rx.c:228) if you run it on the Raspberry. The radiotap header parser was buggy…

  6. malkauns permalink

    Hi. First of all thanks for all your work! I have a problem though. I can’t seem to get decent bandwidth using your code. I am using a TP-Link TL-WN722N on the raspberry pi 2 and a TP-LINK TL-WN721N on my laptop both running on 2.462 GHz (channel 11). Here’s a log of a run I did just now from the pi 2:

    root@raspberrypi:/tmp# sudo ifplugd -S -i wlan0
    root@raspberrypi:/tmp# sudo ifconfig wlan0 down
    root@raspberrypi:/tmp# sudo iwconfig wlan0 mode monitor
    root@raspberrypi:/tmp# sudo ifconfig wlan0 up
    root@raspberrypi:/tmp# sudo iwconfig wlan0 channel 11
    root@raspberrypi:/tmp# cat /dev/zero | ./tx wlan0
    Raw data transmitter (c) 2015 befinitiv GPL2
    64 data packets sent (interface rate: 128.000)
    128 data packets sent (interface rate: 128.000)
    192 data packets sent (interface rate: 96.000)
    256 data packets sent (interface rate: 102.400)
    320 data packets sent (interface rate: 91.429)

    If I receive these packets (with hardly any packet loss) on my laptop and pipe the output to a file it appears to receive at a rate of around 120 kilobytes/s. Shouldn’t I be getting much more than this? The wifi cards are right next to each other.

    • Hi

      Thanks for the detailed problem report! Did you use the patched firmware on the TX side? Without that the default air data rate is at 1mbps which could explain the low throughput. Another reason might be that the choosen channel is very busy. If the TX card has to wait for a long time until the channel is free the rate can drop significantly. If the Problem still exists in an empty channel then please ask again. I’m sure that we can fix it together 🙂

  7. malkauns permalink

    Thanks for your quick reply. Any idea where I can get this patched firmware for the TL-WN722N for the PI?

  8. malkauns permalink

    It works!! Tried this with video and got it working with barely perceptible lag. I measured the lag to be around 170ms on average (using stopwatch recording) but it doesn’t seem to be noticeable at all. Seems like it would be decent for acro flying so I’ll be putting this on my mini-quad soon for some HD FPV action! Here are the commands i’m using (running as root):

    PI transmitter:
    raspivid -ih -t 0 -w 1280 -h 720 -fps 40 -b 3000000 -n -g 5 -pf off -if both -o – | ./tx -p 0 -r 2 -b 16 -f 512 wlan0

    PI receiver:
    ./rx -p 0 -b 16 wlan0 | /opt/vc/src/hello_pi/hello_video/hello_video.bin

    Laptop receiver (with gstreamer):
    ./rx -p 0 -b 16 wlan1 | gst-launch -v -e filesrc location=/dev/fd/0 ! h264parse ! decodebin2 ! xvimagesink sync=false

    Thanks for all your help!

    (note: webpage may change the dash ‘-‘ character)

    • malkauns permalink

      By the way i’m using TP-Link TL-WN722N on the raspberry pi 2 and a TP-LINK TL-WN721N on my laptop both on 2.4GHz channel 1. I was using channel 11 before but got lots of glitches due to interference in my neighborhood. As soon as I changed to the less noisy channel 1 the picture became solid and stays relatively solid as I move around the house with walls in between even though the TL-WN721N has a crappy built-in antenna. 🙂

    • Happy to hear that! The 170ms sound a bit too high. Did you measure that on the PC or PI receiver? You might try to use a lower retransmission block size (-b 4 or -b 8 on both sides). This way fewer packets get buffered. I think I had the lowest latency at 768×480@48fps so you might try that as well.

      By the way (maybe a bit too late for you) I’ve created an overview page which summarizes everything and should give a good starting point to newcomers: https://befinitiv.wordpress.com/wifibroadcast-analog-like-transmission-of-live-video-data/

  9. malkauns permalink

    Thanks, now on to my next problem. Either I change my transmitter to something other than 2.4GHz like EzUHF or I get this whole thing working on 5GHz. I tried all 5GHz channels on the Alfa AWUS051NH (tx) to TL-WN721N (rx) but the packet loss is always so high I barely see a picture. Have you or anyone had a similar experience?

    • Why do you need to change? Because oft your RC TX? I never tried 5GHz because I still fly with 35MHz. Sorry but I’m afraid I cant help you on that one.

      • malkauns permalink

        Well the other issue is if I’m at an FPV racing meetup. Pretty much everyone will be transmitting on 2.4GHz making video using this method unusable if I couldn’t get 5GHz working.

  10. malkauns permalink

    Ok, this is really strange! After much testing I have found that the only 5GHz channel that appears to work between the 2 cards is channel 48 (5240MHz). All other channels fail. Not sure why this is. I’m glad its 5240MHz though as all of the analog FPV TX’s (boscam/immesionRC etc.) use between 5665 and 5945. 🙂

    • Good to know. Thanks!

    • JR4 permalink

      Have you tried two Alfa units for rx and tx?

      • malkauns permalink

        I’ve tried the same Alfa’s for both rx and tx and the rx loses too many packets for it to be usable. However, the TP-LINK 5ghz card receives fine from the Alfa tx.

  11. JR4 permalink

    Interesting, which drivers were you using?

  12. malkauns permalink

    rt2800usb for both cards (AWUS051NH and TL-WN721N)

    • JR4 permalink

      did you disable power saving? (sudo iwconfig wlan0 power off), that seems to help with packet loss, also setting the country code or patching the channel -1 issue.
      Also, have you tried the rt2870sta driver? There’s been success with packet injection with this driver but which version of the 51NH are you using, the newer v2 (still v2 but updated chipset) models have the rt3572 chipset.

      Nothing against the TL-WN721N but external antennas on both RX/TX are definitely a plus.

  13. malkauns permalink

    I have the v2 Alfa’s. Running that power command results in:

    Error for wireless request “Set Power Management” (8B2C) :
    SET failed on device wlan7 ; Invalid argument.

    • Jr4 permalink

      Working on the command.

      Do you know what chipset you have? There are two v2 models, one revision with the rt3070 and a newer one with the rt3572.

      • malkauns permalink

        Bus 001 Device 065: ID 148f:3572 Ralink Technology, Corp. RT3572 Wireless Adapter

    • Jr4 permalink

      Run just iwconfig post the output.
      should output your different devices. I assume from what you posted above that your driver is associating your wifi card as wlan7 (so amend the command to wlan7 instead of wlan0).

      Also, if we knew which chipset you had it would be great to see how it impacts packet loss, etc. would definitely be beneficial to build a different driver (rt2780sta?) than the rt2800usb if it is not working well.

      • malkauns permalink

        hmmm, not sure what you mean. I ran:

        iwconfig wlan7 power off

        and got that error.

        chipset is RT3572 btw

      • Jr4 permalink

        Ok good so it’s the rt3572; the newer one. That should open up a few things. Start with disabling power saving then if needed dive into some alternate drivers. No reason we can’t make this work! (Although there might simply be other cheap USB adapters that do the trick, let’s get this working)

      • Jr4 permalink

        What does iwconfig show?

  14. malkauns permalink

    wlan7 IEEE 802.11abgn Mode:Monitor Frequency:2.484 GHz Tx-Power=20 dBm
    Retry long limit:7 RTS thr:off Fragment thr:off
    Power Management:on

    • Jr4 permalink

      Ok. So there is definitely a way to get power management off which should makes things much better but, that being said the overall better solution is to use the most recent driver from mediatek. Even their drivers from new other chipsets still point back to the 2800 driver so the newest driver is best (after backing up the original just in case).

      • malkauns permalink

        I managed to get power management off but still the same results. Major packet loss!

      • Jr4 permalink

        Ok. So that’s ruled out. What command did you end up using? Changing country code maybe for different channel access?

        Probably need to test a different driver then. The one from mediatek will work with it and should hopefully yield better results. Unfortunate the card is so poorly performing.

  15. Alan permalink

    Hi, can you add a zoom function and maybe the signal strength on the RX unit as OSD?

    Alan

  16. Broccoli permalink

    Hi,
    This truly looks like an awesome project !, Impressive work Mr B !! 🙂

    I am trying this but cant get it to work :(.. prehaps some of you guys can help with some debugging.

    My setup : Two TL-WN722N ( tx, rx ) – Patched the TX one with your firmware.
    Two raspberry PI B+, OS: 4.0.9+
    both boards is also connected to via cable ethernet using ssh.
    RX is also connected via HDMI to screen.

    Result so far:
    I DO manage to send text between rx and tx !,So I have some connection at least :), but I dont see any video/or display.

    ( Get the same output as “nicegraham” : http://i.imgur.com/ghw3tOf.png )

    Tried “Malkauns” debug test:
    4224 data packets sent (interface rate: 1267.200)
    4288 data packets sent (interface rate: 1286.400)
    4352 data packets sent (interface rate: 1305.600)
    4416 data packets sent (interface rate: 1324.800)
    4480 data packets sent (interface rate: 1344.000)
    4544 data packets sent (interface rate: 1363.200)
    4608 data packets sent (interface rate: 1382.400)
    4672 data packets sent (interface rate: 1401.600)
    4736 data packets sent (interface rate: 1420.800)
    4800 data packets sent (interface rate: 1440.000)
    4864 data packets sent (interface rate: 1459.200)

    Have tried a number of different settings, all the same ( these are some examples) + different channels:
    sudo raspivid -ih -t 0 -w 640 -h 480 -fps 48 -b 2000000 -n -g 48 -pf high -o – | sudo /home/pi/wifibroadcast/tx -b 8 -r 2 $NIC

    #sudo raspivid -ih -t 0 -w 1280 -h 720 -fps 40 -b 3000000 -n -g 5 -pf off -if both -o – | sudo /home/pi/wifibroadcast/tx -p 0 -r 2 -b 16 -f 512 $NIC
    #sudo raspivid -ih -t 0 -w 1280 -h 720 -fps 48 -b 2000000 -n -g 48 -pf high -o – | sudo /home/pi/wifibroadcast/tx -p 0 -r 2 -b 16 -f 1024 $NIC
    #sudo raspivid -ih -t 0 -w 1280 -h 720 -fps 30 -b 4000000 -n -g 60 -pf high -o – | sudo /home/pi/wifibroadcast/tx -b 4 -r 4 -f 1024 $NIC
    #sudo raspivid -ih -t 0 -w 1280 -h 720 -fps 48 -b 7000000 -n -g 48 -pf high -o – | sudo /home/pi/wifibroadcast/tx -p 0 -r 2 -b 16 -f 1024 $NIC

    Is there some extra traces that is possible to swich on ?

    BR Mathias

  17. Broccoli permalink

    Hi Again..

    Just wanted to say that I localized the problem :).. it was a combination of:

    1. Overclock + crappy usb-charger (less than 700mA) on B+, seems to have problem to cope with the Juice.

    2. Raspivid “stdn output” command, faulty copy, somehow “-o -” became”-o –” , meaning No output :).. ( And I must have missed that tx always prints the interface-rate, but it didnt with “-o –”.

    Now I have picture and are making fine adjustments.

    Keep up the excellent work !

    Best Regards Mathias

  18. Hello, I have a few questions and would appreciate any reply. I would want to put my stronger wifi adapter on the ground station correct? and Is their a way to use wifibroadcast on windows and if so how? Thank you.

    • Hi. The stronger adapter should be on your aircraft. Afaik there is no Windows port of wifibroadcast.

  19. So if im getting this right… the tx is on the plane?

    • Yes. Alhough upstream is technically possible wifibroadcast is intended to be used as a downstream for video and status data.

  20. Im sorry for asking to many questions I dont have much experience with Linux but I have an pretty good wifi adapter that I was hoping to power from the ground station because its kind of heavy so Would changing it into an upstream effect anything.

Trackbacks & Pingbacks

  1. Relieable 3+km HD FPV solution | befinitiv

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: