Apr 212014
 

This is heavily based on this Wiki entry. Go there to read more about the background of toolchains.

As a quick summary for you: we want compilation to be faster than on the Raspberry Pi. MUCH faster.

Set up crosscompiling

Create / use an Ubuntu machine. I use 12.04 LTS.

You can check your Ubuntu version with lsb_release –a

Run the commands:

sudo add-apt-repository ppa:linaro-maintainers/toolchain

This will add the linaro toolchain repository.

sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) main universe restricted multiverse"

This will add the universe repository and all other available repositories (universe being the one we need additionally).

aptitude update

To get the contents from the repositories. Now install the crosscompiling toolchain:

apt-get install gcc-arm-linux-gnueabi

dpkg -L gcc-arm-linux-gnueabi

This will show the location of the newly installed libraries.

Official note:

Note that this toolchain defaults to ARMv7 with Thumb2. If you want to use it for older processors you have to add “-marm” into CFLAGS.

Apr 212014
 

First of all, remove xpra and cython if you had them installed:

aptitude purge xpra cython

Update your package lists, as we are going to install a lot of packages:

aptitude update

Prepare required prerequisites

Then follow the instructions on the xpra Wiki for building Ubuntu / Debian style:

apt-get install libx11-dev libxtst-dev libxcomposite-dev libxdamage-dev \ python-all-dev python-gobject-dev python-gtk2-dev

apt-get install xvfb xauth x11-xkb-utils
apt-get install libx264-dev libvpx-dev libswscale-dev libavcodec-dev

The file mentioned in the how-to, vpx.pc should exist:

cat /usr/lib/pkgconfig/vpx.pc

You will need to install and compile Cython from sources, as the version in the Raspbian repository is too old (0.15.1 vs. 0.16 minimum needed).

wget http://www.cython.org/release/Cython-0.20.1.tar.gz
tar -xzf Cython-0.20.1.tar.gz

change into the newly extracted directory. Install cython:

python setup.py install

This will take quite a while. Test that you have the correct cython version:

cython --version

should yield Cython version 0.20.1

Download and extract source

wget https://www.xpra.org/src/xpra-0.12.3.tar.bz2
tar -xjf xpra-0.12.3.tar.bz2

Note: there may be a newer package, check, please.

Change into the extracted directory. We need to apply a patch:

patch < patches/old-libav.patch

Enter xpra/codecs/dec_avcodec/decoder.pyx as the file to patch

Next patch (several files in one go):

patch < patches/old-libav-pixfmtconsts.patch

Simply copy and paste the “Index file” the patcher asks for, for example xpra/codecs/csc_swscale/colorspace_converter.pyx

Next patch (also several files):

patch < patches/old-libav-no0RGB.patch

Act like above (copy & paste file name, without leading / ).

It also contains a useful README, which tells you the next step is:

./setup.py install --home=install

After the compilation is done, you should either (always) set the Pythonpath to include the install subdirectory, like this:

export PYTHONPATH=$PWD/install/lib/python:$PYTHONPATH

or install the “finished” files to the appropriate targets. From the install directory do:

cp bin/* /usr/bin/.
cp -R lib/* /usr/lib/.
cp -R share/* /usr/share/.

xpra will now be the newest version:

xpra –version

xpra v0.12.3

You will still have to set the PYTHONPATH to the new files in /usr/lib/python, though:

The PYTHONPATH environment variable needs to be set:

export PYTHONPATH=/var/lib/python:$PYTHONPATH

 

Test & Test results

OK, here’s how to set up a test session:

Set up a test server, which has xpra installed (you can install it through the winswitch packages, will get you the newest xpra version on Ubuntu & Debian)

Start X Windows, open LXTerminal, run the following commands.

export PYTHONPATH=/var/lib/python:$PYTHONPATH

Start an xpra session via SSH (can be killed using Ctrl-C, and reconnected to using the same command):

xpra start ssh:maxcs@192.168.1.61:122 –start-child=xterm –encoding=h264

Read the manpage (man xpra) to have a look at some other options

Test results

xpra-raspberry-h264

rgb, png encodings are too high-latency.

jpeg is barely usable, even when resizing the application (for instance Abiword) to not full-screen usage.

webm encoding delivers worse quality, but seems a bit more usable

h264 decoding is NOT done in hardware in the default code (we’ll look into this). Surprisingly it is still the “most fluid to use” one.

I suspect that no decoding in H.264 is taking place, and server side xpra falls back to a different encoder (webm?) Anyways, one can even “watch” videos (a couple of frames each second with heavy artifacts) with this.

For very light administration / checking of remote contents, etc. xpra can be used as is. We will need to enable hardware decoding of h264, though, for it to yield real benefits.

Please note: our interests solely rest in streaming TO the Raspberry Pi, not FROM the Raspberry Pi – we will not test / patch in order to speed up administration of the Pi at this point.

 

Notes & Further reading

Dependencies of xpra package:

(you can show this using “apt-cache showpkg xpra” on a machine which has the package in the newer version, e.g. Ubuntu AMD64):

Dependencies:
0.12.3-1 – python2.7 (0 (null)) python (2 2.7.1-0ubuntu2) python (3 2.8) libavcodec53 (18 4:0.8-1~) libavcodec-extra-53 (2 4:0.8-1~) libavutil51 (18 4:0.8-1~) libavutil-extra-51 (2 4:0.8-1~) libc6 (2 2.14) libgtk2.0-0 (2 2.24.0) libswscale2 (18 4:0.8-1~) libswscale-extra-2 (2 4:0.8-1~) libvpx1 (2 1.0.0) libx11-6 (0 (null)) libx264-120 (0 (null)) libxcomposite1 (2 1:0.3-1) libxdamage1 (2 1:1.1) libxext6 (0 (null)) libxfixes3 (0 (null)) libxrandr2 (2 4.3) libxtst6 (0 (null)) python-gtk2 (0 (null)) x11-xserver-utils (0 (null)) xvfb (0 (null)) python-gtkglext1 (0 (null)) python-opengl (0 (null)) python-numpy (0 (null)) python-imaging (0 (null)) python-appindicator (0 (null)) openssh-server (0 (null)) python-pyopencl (0 (null)) pulseaudio (0 (null)) pulseaudio-utils (0 (null)) python-dbus (0 (null)) gstreamer0.10-plugins-base (0 (null)) gstreamer0.10-plugins-good (0 (null)) gstreamer0.10-plugins-ugly (0 (null)) python-gst0.10 (0 (null)) openssh-client (0 (null)) ssh-askpass (0 (null)) python-numeric (0 (null)) python-lz4 (0 (null)) keyboard-configuration (0 (null)) xpra:i386 (0 (null))

CheckInstall

Optional: install checkinstall, to create a package which you can easily remove or re-deploy to other computers:

aptitude install checkinstall

 

Troubleshooting

Patches

error: implicit declaration of function ‘avcodec_free_frame’

you need to apply the patch patches/old-libav.patch

error: ‘AV_PIX_FMT_YUV420P’ undeclared

you need to apply the patch patches/old-libav-pixfmtconsts.patch

error: ‘PIX_FMT_0RGB’ undeclared

you need to apply the patch patches/old-libav-no0RGB.patch

The other patches were NOT needed in my experimental compilation.

 

ImportError: No module named xpra.platform

Once you try to execute xpra (from LXTerminal preferably), you may get this message. The PYTHONPATH environment variable needs to be set:

export PYTHONPATH=/var/lib/python:$PYTHONPATH

Apr 192014
 

libavg is a German project to ease the building of multimedia applications – this can be anything from a movie / touch interface installation in a museum to a quick demo of a future application you throw together yourself, before (if needed) delving into “hardcore” programming.

libavg supports a variety of text, graphics, audio and video output, and a variety of input possibilities (e.g. multitouch). Have a look at their showcase to see some of the possibilites.

Luckily for us, libavg has been ported and optimized for the Pi, and they provide a pre-compiled package.

Installation

Follow the instructions on https://www.libavg.de/site/projects/libavg/wiki/RPI for installing the tarball (see link at the bottom of the page, which you can “wget”.

There are also instructions for compiling from source, using QEMU, on that page.

Usage

After installation, you can test libavg with the classic “hello world” program.

Please refer to this page https://www.libavg.de/site/projects/libavg/wiki/HelloWorld for the source.

One gotcha: X Windows needs to be running in order for the program to execute. (“startx”). Else the software will complain:

RuntimeError: No available video device

Video

Video playback is NOT hardware accelerated on the Pi currently with libavg.  In my test a H.264 encoded low-res movie was played at normal speed, but seemed quite choppy. Audio was OK. Decoding and rendering happens via the CPU (as per the “node” idea – combining multiple videos / other nodes for output), which maxes it out. The same video works fine using omxplayer, though.

In this blog article the team announces intentions to work on OpenMax IL integration. In the same article, a comment states that as per January of 2014 there has been no work done in this area, lacking necessary manpower & knowledge resources for the development.

Documentation

libavg has an extensive documentation, for instance this page for area nodes (including the video node).

It also has a blog, with some interesting entries, for example:

Video decoding using libav and ffmpeg – detailing some of the problems behind video decoding in general, and libav/ffmpeg in particular.

Mar 152014
 

With this interesting tool you can redirect input over the network – control other Linux boxes as if you were physically sitting in front of them and using a USB mouse and keyboard.

The project’s GitHub repository is found here:

https://github.com/MerlijnWajer/uinput-mapper

More documentation is available on this site: http://hetgrotebos.org/wiki/uinput-mapper

Here is an introduction how to set up keyboard and mouse forwarding via SSH to a second Linux box.

keyboard-mouse

Installing uinput-mapper

as user root (sudo su):

aptitude install git-core

cd /opt

git clone https://github.com/MerlijnWajer/uinput-mapper.git

cd uinput-mapper

make

This will check out the tool into the /opt directory. This is not a requirement per se, you can also install it in a different directory of your liking.

make will build the file “uinputmapper/uinput_gen.py”

Use the same procedure on the server.

Connect to server

Test the connection by logging in via SSH and the appropriate key to the server (you need to set this private / public authentication up first, of course – see this article, for instance).

ssh root@192.168.1.61

Should log you in to the remote server. Change the username, (the path to the key if needed – can be specified with the –i option), and the IP address according to your setup.

Log out again.

Have a look at the input devices on your “local” machine, from which you will be redirecting the input:

ls -alh /dev/input/*

Sometimes, keyboards will create two devices – one for the additional (multimedia?) keys.

Connect (with error / stdin logging)

./input-read -G /dev/input/event0 -G /dev/input/event1 -D | ssh root@192.168.1.61 “/opt/uinput-mapper/input-create &>>/tmp/errorlog “

If nothing happens, have a look at /tmp/errorlog on your server:

Traceback (most recent call last):
  File “/opt/uinput-mapper/input-create”, line 73, in <module>
    fd, ev = in_f.load()
EOFError

Try connecting with the compatibility option for Python < 2.7 in this case.

./input-read -C -G /dev/input/event0 -G /dev/input/event1 -D | ssh root@192.168.1.61 “/opt/uinput-mapper/input-create -C &>>/tmp/errorlog2 “

please note, that the compatibility flag needs to be given on BOTH sides (local and remote part) of the command.

You can verify your Python version like this:

root@cloudsource2:/opt/uinput-mapper# python –version
Python 2.7.3

If everything works allright, you can leave away the last part (starting with the ampersand “& …”) which just redirects standard output and error from the server for debugging.

Feb 082014
 

Last month we posted a gstreamer pipeline for avi playback and simply presumed a working gstreamer installation. Now we have added a couple of packages to our growing repository, that make it very easy to install.

The packages include the gst-omx module, which allows h264 de- and encoding with full use of the Raspberry Pi’s graphics hardware. But even with hardware support, the current stable series (1.2) doesn’t perform very well, because it does unnecessary copying of the video data. This is why these packages are based on the current development series (1.3), which means that they are not as mature and well tested as the slightly older 1.2 versions. But so far we haven’t run into any problems of that nature.

As a side note: The gstreamer versioning can be a little confusing. The packages you can find in software repositories are usually marked with “-0.10” or “-1.0”. The 1.x series is API and ABI stable and supersedes the previous stable 0.10 series and packages marked 1.0 can contain any version from the 1.x series.

The rest of the article will be about installing the packages, how to see if they work and a couple of things you can try if they don’t.

Installation

Add our repository and install gst-omx-1.0.

The omx plugins need GST_OMX_CONFIG_DIR to be set. You can do that temporarily like this:

export GST_OMX_CONFIG_DIR=/usr/etc/xdg/

or add the same line to your .profile as a permanent solution.

Test

You can use gst-inspect-1.0 to list all installed gstreamer plugins. Combine with grep and it’s easy to find out if gstreamer knows about the omx plugins:

gst-inspect-1.0 | grep omx

should give you something like this:

omx:  omxmpeg2videodec: OpenMAX MPEG2 Video Decoder
omx:  omxmpeg4videodec: OpenMAX MPEG4 Video Decoder
omx:  omxh263dec: OpenMAX H.263 Video Decoder
omx:  omxh264dec: OpenMAX H.264 Video Decoder
omx:  omxmjpegdec: OpenMAX MJPEG Video Decoder
omx:  omxvc1dec: OpenMAX WMV Video Decoder
omx:  omxh264enc: OpenMAX H.264 Video Encoder

To play a file you can use the previously mentioned pipeline:

gst-launch-1.0 filesrc location=test.avi ! avidemux ! h264parse ! omxh264dec ! autovideoconvert ! eglglessink

(Note that gstreamer is not a media player, but a framework that can be used by one. These commands are meant to support developers.)

Troubleshooting

Errors during playback

Make sure the memory split is generous with the video hardware, otherwise you may get seemingly random and unrelated errors during playback.

The omx plugins don’t show up

If gst-inspect-1.0 doesn’t show the plugins, try deleting gstreamers registry:

rm .cache/gstreamer-1.0/registry.armv6l.bin

Pipeline debugging

A very useful feature for debugging pipelines is the possibility to create a visual graph of the pipeline. Set GST_DEBUG_DUMP_DOT_DIR to a temporary directory before running the pipeline:

mkdir tmp
GST_DEBUG_DUMP_DOT_DIR=./tmp/

Gstreamer will dump a lot of debugging information in *.dot files in that directory. You can use dot from the graphviz package to create image files from those:

dot -Tpng x.xx.xx.xxxxxxxxx-gst-launch.FOO.dot > graph.png

Links

  1. gstreamer pipeline for avi playback
  2. list of packages in our repository
  3. gstreamer project website
  4. streaming H.264 via RTP
  5. slightly outdated but useful StackExchange
  6. Raspberry Pi forums
Oct 252013
 

We have already blogged about a better approach to use FreeRDP (with the DirectFB rendering backend) on the Raspberry Pi (here). We have been promising an easy-to-install package. Now we are deliviring this promise!

(This article will be updated with more information shortly. Please refer to our original article for more hints & information about setting up and running dfreerdp).

This post will give instructions on how to use the pi3g package repository to install the dfreerdp package. This means that if you’re running Raspbian, you don’t have to compile anything and will receive updates to this and other packages automatically.

The quick version

Add our repository (repository.pi3g.com) and install dfreerdp.

You may need to update the package cache (sudo aptitude update) to see the package.

The long version

Add the pi3g repository

Adding the repository is pretty easy and, in principle, works the same as for every other software repository. Create the file /etc/apt/sources.list.d/repository.pi3g.com.list with the following line:

deb http://repository.pi3g.com/debian/ wheezy main

A quick way to do that is this command:

sudo wget -O /etc/apt/sources.list.d/repository.pi3g.com.list http://repository.pi3g.com/sources.list

Then add our public key, which will ensure the packages are not intercepted and altered during download. The command apt-key add <keyfile> will let you do just that. Or in one command:

wget -O - http://repository.pi3g.com/pubkey | sudo apt-key add -

Install the dfreerdp package

First make sure your system is up to date:

sudo aptitude update
sudo aptitude upgrade

(It’s a good idea to run these two commands every now and then so your system doesn’t fall too far behind.)

Then simply install the package dfreerdp like any other package. For example with this command:

aptitude install dfreerdp
Aug 212013
 

Aka how to view YouTube videos on your Raspberry Pi from Midori or Chromium!

youtube-on-the-raspberry-pi

We have pulled together some software magic and are proud to announce an easy installation procedure for PiVid, the YouTube / HTML5 Video / VeeHD Raspberry Pi video enabler:

Installation

Open a shell – for instance LXTerminal, if you are working on the Raspberry Pi desktop, it can be opened by double-clicking the LXTerminal icon – and enter the following commands:

Step I: Please add our Repository

sudo wget -O /etc/apt/sources.list.d/repository.pi3g.com.list http://repository.pi3g.com/sources.list
sudo wget -O - http://repository.pi3g.com/pubkey | sudo apt-key add -
sudo aptitude update

This will:

  • set up our repository as an additional package source on your Raspberry Pi (your main sources.list will not be modified in any way)
  • get our public key
  • fetch the available packages from our repository server

Step II: Install PiVid

sudo aptitude install pivid

This will:

  • install PiVid and dependencies (other packages which PiVid requires to work)

Please press “Enter” when asked whether you want to continue, and once again “Enter” during the installation process of the youtube-dl package, when asked to.

Step III: Activate PiVid

service pivid start

This will:

  • start the pivid service (it talks to your browser and to omxplayer to play your video)

If you want to stop the pivid service, simply use “service pivid stop” – this will save a small amount of computing resources.

Step IV: Install PiVid in your browser

Install PiVid in Midori:

To use the userscript in Midori (Raspbians default browser), you have to activate the “User addons” extension. Start Midori. Open the preferences dialog (Menu>Preferences), select the Extensions tab and tick “User addons”. (“User addons” is the very last entry, use the scroll bar to find it). Click “Close”.

Then simply open the script ( file:///usr/share/pivid/replace.user.js ) in your browser (copy & paste the link into your URL bar, and press enter) and click “Install user script” when asked.

pivid-userscript-midori

 

Install PiVid in Chromium:

To install pivid in Chromium, please open a shell in the X desktop environment (e.g. LXTerminal), and start Chromium like this:

# chromium --enable-easy-off-store-extension-install /usr/share/pivid/replace.user.js

pivid-userscript-chromium

Please click “Continue”, and then click “Add” on the “Confirm New Extension” dialogue:

click-add

Chrome will confirm, that the extension has been added.

Usage

Please note: after every reboot, you need to start the pivid service, as described above. Open LXTerminal, and (as user pi), enter:

service pivid start

You can close LXTerminal after starting the service, if you want. If the service pivid is already running, it will tell you so – you don’t need to worry about entering it twice by accident.

Starting PiVid will open a web server on your Raspberry Pi’s port 3000. This will be accessible on your local network – and can be used to control video playback (of external URLs) on the Raspberry. This is by design, and we plan to release more documentation about this soon.

YouTube

youtube-raspberry-chromium

Navigate to a YouTube video (for example: Xcylin, shooting with a frickin’ laser). First, the browsers will display “Flash Player missing”, but after some seconds, the web site should reload, and display the PiVid play button, as seen above for Chromium.

Simply click this button, and the video should launch in a short while. This will be indicated by a “pause” button:

youtube-raspberry-c-pause

If you want to see the video again, please reload the page – PiVid currently can’t identify when the video has stopped playing.  The pause button currently has no function, but you can use omxplayer keyboard control (see below).

VeeHD

VeeHD is an alternative video hosting platform / community, sporting an interesting selection of movies, documentaries, etc.

PiVid supports playback on VeeHD. You may need to log in as a VeeHD user for some videos to work, and some videos are hosted on very slow servers – where streaming is impossible with PiVid. Please download the video and play it with omxplayer instead.

Usage: Search with the VeeHD search, and click on the video you would like to see. PiVid will show you the play button after a short while:

veehd-raspberry-pi

Omxplayer keyboard control

Space or p = Pause / Resume
– / + = Decrease / Increase Volume
q = Quit

 

Problem handling

YouTube play button not showing

If the PiVid play button does not show up on a YouTube video, instead of the warning about an incompatible browser, please try to simply reload the page – it should show up now.

Restarting the service

If you run into any unexpected problems using the software, try restarting the pivid service:

service pivid restart

If this does not help, try restarting your Raspberry, and starting the service after that. If still stuck, comment below. (Use “pivid” instead of “service pivid start” to see some advanced debugging output).

update-rc.d

Please note, that update-rc.d can’t be used right now to install the service as autostart on boot – we are investigating the cause and will release an update soon. (pivid service will start, but without video playback.) Please use the manual starting method ( service pivid start ) in the meantime – thank you!

Background

PiVid works as a server (written in the uber-cool Node.JS), which will display an IFRAME in your browser. Through a user script, the browser can submit the URL needed to retrieve and play the video. Once you click on the “play” button, PiVid will invoke some software incantations (sprinkle the equivalent of magic fairy dust – we’re building on the shoulders of giants – you guys know who you are :-) – and voilà – the video location will be passed to omxplayer, the trusty workhorse of Raspberry Pi video playback!

You can, of course, fork us on GitHUB.

By the way – we have also packaged a newer version of Node.JS to go along with PiVid on our repository – package name is “nodejs”. Stay tuned for more about this.

The idea for PiVid was taken from a very similar client-server approach, which was coded in Python, but seems to be abandoned now. Thank you for the inspiration!

Reference

Aug 032013
 

LibreOffice ist mittlerweile frei als Paket für Raspbian verfügbar. Die Installation geht wie bei jedem anderen Paket auch, ganz einfach so:

sudo aptitude update
sudo aptitude upgrade
sudo aptitude dist-upgrade
sudo aptitude install libreoffice

unter der Konsole (bspw. unter LXTerminal).

Man sollte sich einiges an Zeit mitnehmen, da der Prozess länger dauert (> 100 MB download, > 300 MB im installierten Zustand).

Anschließend kann man unter X-Windows die LibreOffice Suite aus dem Start-Menü unter dem Punkt “Office” ganz einfach starten. LibreOffice Writer, LibreOffice Calc, und die anderen Programme sind wie gewohnt bedienbar.

Raspberry Pi LibreOffice Edition

Wir haben eine besondere LibreOffice Edition für den Raspberry Pi entwickelt: auf dem Gehäuse des Raspberries befindet sich das LibreOffice Logo. Für jedes verkaufte LibreOffice Edition Kit spenden wir für Dich 5 € an die Document Foundation, die gemeinnützige Organisation hinter LibreOffice.

In unseren LibreOffice Edition Raspberry Pis ist LibreOffice bereits vorinstalliert – so kannst Du gleich loslegen.

Klicke bitte auf das Bild um in den Shop zu gelangen:

libreoffice-essentials-kit

Aug 022013
 

This is a work still in progress with unsatisfactory results (image quality, delay, very low frame rate), but here’s for the brave-hearted and those who are researching into the same direction:

Set up Windows streaming host

This can be a multi-monitor machine. Your left-most monitor will be streamed.

I generally use FullHD resolution for testing.

  • Install a Direct Show Screen Capture Filter for Windows. We used the direct show filter provided with “Screen Capturer Recorder” by Roger D Pack. Roger also includes an audio direct show capturer. And all free of charge – a real bargain 😉
  • Maybe a reboot is necessary here
  • Install latest version of ffmpeg from Zeranoe. Opt for the static builds (probably 64 bit if you are running a modern Windows 64 bit OS on a modern computer)
  • extract the download to a safe location
  • Open PowerShell, and navigate to the location

List the available screen filter devices:

This and all following shell commands are to be issued in the PowerShell. 

.\ffmpeg -list_devices true -f dshow -i dummy

This will show you the available input devices to capture from. My list looks like this, for instance:

 DirectShow video devices
  "Integrated Webcam"
  "screen-capture-recorder"
 DirectShow audio devices
  "Microphone (2- High Definition Audio Device)"
  "virtual-audio-capturer"

Start the stream:

.\ffmpeg -f dshow -i video="screen-capture-recorder" -vcodec libx264 -vprofile baseline -preset ultrafast -tune zerolatency  -pix_fmt yuv420p -b:v 400k -r 30  -threads 4  -fflags nobuffer -f rtp rtp://192.168.1.14:1234

I used PowerShell to start this, thus the .\ is needed in front of an application in the current folder.

  • libx264 is used as video codec, rather than mpeg4 (for superior quality – the Raspi is capable of H264 hardware decoding)
  • baseline profile needs to be used together with –pix_fmt yuv420p – this basically reduces the encoding to a simple subset of the full standard. Leaving out these two options led to the streaming not working, but you may be able to figure out something – please comment!
  • -preset ultrafast and –tune zerolatency both accelerate the video output. I have a latency of about 1 – 2 sec. in our lab here
  • -b:v 400k sets the target bitrate (as variable)
  • -r 30 this sets the framerate to 30
  • -threads 4 – give more threads to ffmpeg
  • -fflags nobuffer – should decrease latency even further. Not sure if it does, though.
  • -f rtp – specifies the output format. Here we use rtp, and stream it directly to the raspberry – which has the IP 192.168.1.14 on our network. You can choose whatever you like for the port, by an odd coincidence we chose 1234. Aliens?!?

Hit “Enter” and ffmpeg will start streaming. It will show you handy statistics – current frame number, framerate, quality, total size, total time, current bitrate, duplicated capture-frames, dropped capture-frames (i.e. the capturing rate does not align with the streaming rate). Do not worry too much about those for now.

Please note that you need some horsepower for capturing, encoding and streaming in real-time.

Set up Raspberry Pi

omxplayer can’t handle RTP streams directly – thus, we resort to GStreamer.

GStreamer 1.0 includes special support for the Raspberry Pi’s Broadcom SoC’s VideoCore IV hardware video functions (also known as OpenMax). Unfortunately, the Raspbian maintainers do not want to include it (yet), in order not to diverge too far from the official Debian repositories.

Luckily for you, though, someone has precompiled the binaries and set up a repository. See this thread for more background information, or simply follow my instructions:

sudo nano /etc/apt/sources.list

This will open nano to edit your package repository list. Please add the following line into this file:

deb http://vontaene.de/raspbian-updates/ . main

After saving the file (Ctrl + O, Ctrl + X), run the following commands:

sudo aptitude update
sudo aptitude install libgstreamer1.0-0-dbg gstreamer1.0-tools libgstreamer-plugins-base1.0-0 gstreamer1.0-plugins-good gstreamer1.0-plugins-bad-dbg gstreamer1.0-omx gstreamer1.0-alsa

This will install the necessary gstreamer1.0 & components.

Start the stream receiver & decoder chain:

gst-launch-1.0 -v udpsrc port=1234 caps='application/x-rtp,payload=(int)96,encoding-name=(string)H264' ! queue ! rtph264depay ! h264parse ! omxh264dec ! autovideosink sync=True

This can be done as user pi. Please note, that this may not be the perfect command to achieve playback, but it is a good starting point – as it works!

Gstreamer sets up “pipelines”, in which data is passed on in transformed state from step to step. While it seems to be quite a bit at the first look, it is very logical in itself, once you have figured it out.

  • we specify a UDP source (udpsrc), the port, and “caps”
  • Without the RTP caps, playback is not possible. Apparently they are not provided along with the stream? Thus, we have to specify the caps manually.
  • In the caps we specify some information for the pipeline
  • queue may be omitted, I am not sure what it does
  • rtph264depay – depayload h264 data from rtp stream
  • h264parse – parse h264 data
  • omxh264dec – decode the data with BroadCom OpenMAX hardware acceleration
  • autovideosink – put the result on the display
  • sync=True – I am not sure whether this does anything, or whether it is in the right place and form. It was an attempt to fix the gst_base_sink_is_too_late problems (but it did NOT fix them).

Issues

slow screen updates

These are very likely caused by a slow screen capture refresh rate, this may be better with a different screen capturer.

On Windows 8, with a pretty powerful Core i7 machine, I get possible fps 15.41 (negotiated for 30 fps). This is using Roger’s / betterlogic’s screen-capture-recorder. Roger claims this is due to Aero.

See more about it here  and here (also provides a list of available other directshow screen capture filters).

artifacts

Gstreamer shows massive H.264 artifacts – Matthias Bock has opened an issue for this, and some further hints.

This seems to be related to the bitrate set in FFMPEG – if I lower it to ~ 400 k, the artifacts become less distorted, and image quality is quite OK. Also, use a variable bitrate instead of a constant one.

gst_base_sink_is_too_late()

This may be related to the Pi’s fake hardware clock (?). It also appears when running gstreamer with a simple test image setup:

gst-launch-1.0 videotestsrc ! autovideosink

gstbasesink.c(2683): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstEglGlesSink:autovideosink0-actual-sink-eglgles:
There may be a timestamping problem, or this computer is too slow.

 

The command above will display a test video image.

Sound

I have not tried sound yet. Sound shoud be input into ffmpeg using the following arguments:

-i audio="virtual-audio-capturer":video="screen-capture-recorder"

This directly from Roger’s GitHUB documentation.

Ideas

  • try to use gstreamer on Windows for streaming?
  • Adjust Parameters for betterlogic/Roger’s direct show capturer
    • apparently it hits the ceiling at 15 fps with Aero on
  • Use a different direct show capturer
  • Tune quality for ffmpeg stream

Background info

  • H.264 is MPEG-4 Part 10 or = MPEG-4 AVC – and is the more modern and data-efficient codec format (“advanced video coding”);
  • whereas MPEG-4 Part 2 = MPEG-4 Visual is based on the older image compression standards used in MPEG-2, and also implemented in DivX, Xvid, etc.
  • you can also use .\ffplay –i udp://:1234 to test the streaming output on the local machine. The video quality IS NOT TO BE USED AS A REFERENCE. It just shows, that it “works”. Change the target IP accordingly (“localhost” instead of the Raspi’s IP will do, I believe.)

References

Optimization WordPress Plugins & Solutions by W3 EDGE