Apr 202014
 

We’re working on streaming a multimedia remote desktop to the Raspberry Pi.

In the future we envision, you shall be able to use a webbrowser on the Pi at normal speeds – including YouTube videos, etc. – operate with CPU / GPU intensive applications – as all the processing is done on a server, and just the H.264 stream rendering on the Raspberry Pi.

First steps

A very interesting application stack to serve this purpose is already available: WinSwitch & xpra

Look at the WinSwitch homepage for installation instructions – it is really quite easy.

WinSwitch bundles several remote clients (VNC / xpra / RDP) with an easy-to-use interface, and broadcasts servers / clients (via Avahi / Bonjour).

A very first demonstration of the capabilities of this stack can be obtained installing WinSwitch on your “normal” desktop machine, and on a server.

Server

As a server we currently use the “fastest available” Intel Atom processor currently on the market –  Intel(R) Atom(TM) CPU  C2750. We are looking into using AMD’s ARM 64 bit processor as server hardware in the future (power-efficiency!), and the performance should be roughly comparable.

Client

As a client we use a Dell Inspiron notebook (with Windows 8.1), Core i7 processor, FullHD resolution.

There is a xpra package available for the Raspberry Pi, which is based on a quite old version of xpra, and will not connect to our server. This may be related to the huge version difference between the two packages, wrong setup, or special tweaking done by winswitch to xpra.

Test results

YouTube videos

We can stream a webbrowser running YouTube fullscreen in FullHD, which will use about 50 % of the server’s total resources (decoding one or several videos in FullHD, encoding one FullHD stream). This is possible in low-latency, at about 30fps and high quality. Yes, this does include an audio stream, too.

The stream uses about 40 Mbp/s of bandwidth, and is much more reliable (less choppy) if streamed over LAN, instead of WLAN. In fast-moving scenes video will still be a bit choppy, but tolerably.

The encoding is done in software, using x264.

Streaming ONE browser window is possible with our server hardware in good quality (for video content) from either the host directly, or from a virtual Ubuntu machine (KVM-virtualized) inside it.

TWO browser windows will start to degrade the quality, even if trying to force best quality and lowest latency.

image

This screenshot shows the YouTube video in the browser being streamed on the client.

Application streaming

Winswitch allows you to stream single applications. Performance / latency is very good on a local network, keyboard / mouse delay is barely noticeable.

In general, applications will be quite useable and seem responsive.

Applications requiring precise mouse / screen cordination, like graphics software will not be usable (at least with our hardware setup).

image

This screenshot shows Firefox being streamed through xpra.

image

VLC media player being streamed – sound works

image

GIMP: barely usable (too much delays)

image

word processing with AbiWord: OK performance (could be better, but it’s usable)

 

Desktop streaming

Streaming a desktop with xnest / Xephyr / xpra from inside a virtualised Ubuntu container on the base hardware is NOT possible at low-latency (30 fps) with high-quality. (With our server hardware)

In order to test it, you have to install additional packages on the server:

aptitude install xnest xserver-xephyr

and set the desktop default to xpra, possibly after restarting the server / client:

image

Apparently frame-grabbing / mirroring the desktop / going through the X layer uses up much more processor resources.

image

Gameplay is quite smooth – but full screen video playback would be a problem.

Some hints

  • authentication with private/public key pairs may be problematic without additional configuration, for first tests I recommend to re-enable password login for SSH.
  • audio for the browser may require alsa and pulseaudio
  • This does not work on the Raspberry Pi yet, this is our next step (compiling a package for it).

 

Outlook

  • encoding performance and latency may be enhanced significantly using NVidia’s NVENC hardware encoding / framegrab API – which xpra supports.
Mar 152014
 

With this interesting tool you can redirect input over the network – control other Linux boxes as if you were physically sitting in front of them and using a USB mouse and keyboard.

The project’s GitHub repository is found here:

https://github.com/MerlijnWajer/uinput-mapper

More documentation is available on this site: http://hetgrotebos.org/wiki/uinput-mapper

Here is an introduction how to set up keyboard and mouse forwarding via SSH to a second Linux box.

keyboard-mouse

Installing uinput-mapper

as user root (sudo su):

aptitude install git-core

cd /opt

git clone https://github.com/MerlijnWajer/uinput-mapper.git

cd uinput-mapper

make

This will check out the tool into the /opt directory. This is not a requirement per se, you can also install it in a different directory of your liking.

make will build the file “uinputmapper/uinput_gen.py”

Use the same procedure on the server.

Connect to server

Test the connection by logging in via SSH and the appropriate key to the server (you need to set this private / public authentication up first, of course – see this article, for instance).

ssh root@192.168.1.61

Should log you in to the remote server. Change the username, (the path to the key if needed – can be specified with the –i option), and the IP address according to your setup.

Log out again.

Have a look at the input devices on your “local” machine, from which you will be redirecting the input:

ls -alh /dev/input/*

Sometimes, keyboards will create two devices – one for the additional (multimedia?) keys.

Connect (with error / stdin logging)

./input-read -G /dev/input/event0 -G /dev/input/event1 -D | ssh root@192.168.1.61 “/opt/uinput-mapper/input-create &>>/tmp/errorlog “

If nothing happens, have a look at /tmp/errorlog on your server:

Traceback (most recent call last):
  File “/opt/uinput-mapper/input-create”, line 73, in <module>
    fd, ev = in_f.load()
EOFError

Try connecting with the compatibility option for Python < 2.7 in this case.

./input-read -C -G /dev/input/event0 -G /dev/input/event1 -D | ssh root@192.168.1.61 “/opt/uinput-mapper/input-create -C &>>/tmp/errorlog2 “

please note, that the compatibility flag needs to be given on BOTH sides (local and remote part) of the command.

You can verify your Python version like this:

root@cloudsource2:/opt/uinput-mapper# python –version
Python 2.7.3

If everything works allright, you can leave away the last part (starting with the ampersand “& …”) which just redirects standard output and error from the server for debugging.

Optimization WordPress Plugins & Solutions by W3 EDGE