Sunday, July 12, 2009

what i learned about red5, rtmp and flash

i'm living with red5 for last 5-6 months and i realised i've no entry about red5, i'll try to recover the loss now with one huge entry.

first of all, rtmp's specs are not open, so red5 is a reverse engineering product, and it's a pretty successful project if we consider thah facebook is using it. by the way, adobe had announced they'll open rtmp specs at first half of 2009, but yet they didn't.

rtmp is a protocol on tcp, i don't understand why it isn't udp. tcp is a lossless protocol, so in live video streaming it's hard to prevent delay if bandwith is insufficient. it's not possible to drop packets for incoming stream because you can't touch flash player code. it's only possible to drop packets in server side for stream subscribers, but you've to write some code for this, it's not a configuration thing.

if you need to do some scaling or re-broadcast incoming stream to other red5 servers, you've to be inside the code too. people prefer terracotta[1] for scaling. for re-broadcasting it's impossible to find sample code, but it's possible to find theorical solutions in email lists[2].

by default, red5 uses port 5080 for http, 1935 for rtmp, 8443 for rtmps and 8088 for rtmpt. so company firewalls are problem, generally only allowed port is 80. but we had to serve http for our webservice and rtmp for streaming from the same red5, this means two ports at the same time. we overcomed this obstacle with some servet settings in our web.xml[3]. rtmpt means http actually, transfers stream data with continous http requests. so it's possible to serve your webservice and stream server from the same port by adding this settings to your web.xml:


<servlet>
<servlet-name>rtmpt</servlet>
<servlet-class>org.red5.server.net.rtmpt.RTMPTServlet</servlet>
<load-on-startup>1</load>
</servlet>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/fcs/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/open/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/close/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/send/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/idle/*</url>
</servlet-mapping>


it wasn't easy to make ssl work, we recognized that rtmps does not mean rtmp over ssl, in fact there is no such thing as rtmps, it's more appropriate to say rtmps=rtmpts=https. we were using 0.7.4 early, but it just didn't work with ssl, we moved to 0.8.X versions, after some tries we succeeded to make it work at exact version 0.8-RC2, neither earlier nor later.

and in flash side, you should look flash restrictions before designing anything. for example, sizes of permission boxes in flash is constant in flash, if you create a smaller flash, user can't give permission to access camera/microphone. if user doesn't check remember option in permission box, s/he need to give permission for every flash one by one. if in your screen, a part of the flash is not in eyesight, it's not possible to click on any button on flash or you can't make flash fullscreen without a user click. these are adobe's security precautions. and you have to deal with many security related requirements if you want to serve flash from a different server and your website from another, i advice you to read flash's security policy first. this kind of small things can cause whole design changes totally.

and using shared objects for communication between flash clients and server is not a good idea, red5 implementation has some problems with shared objects. we prefered to use remote function calls (flash.net.NetConnection.call) instead.

we're developing software on linux, so writing actionscript was a question mark for us, but with flex3sdk and ant we did it without problem. however i must warn you, if you want to do some design job there is no mature tool for linux, so it wouldn't be a right choice.

we had to push the lines about quality to see if it's possible to use red5 in closed-circuit broadcasts. we saw that it's possible to catch dvd quality (480 lines) with flash player without much effort. dvd quality requires 7-8 mbit/s bandwidth, by default we couldn't exceed 1 mbit/s limit in upload at first time, no one mentions about this limit in any where. you need to add some code to your application to activate bandwidth controls in order to exceed this limit.

flash player couldn't play incoming stream effectively after 480 lines. flash player is capable to encode camera data with h.263 and vp6 codecs, it can't encode h.264 but can decode it to play. to reduce the amount of data going to server it's important to use h.264. by the way it's possible to re-encode incoming stream in server side with xuggler[4] before broadcasting it. we didn't want to add cpu load in server side so we wanted to encode it in client side. we looked for other alternatives rather than flash player to stream camera data, we couldn't find much, probably because of rtmp's closed specs. there were many rtsp clients, if there were something converting rtmp to rtsp, alternatives would be increased in considerable amounts.

vlc version 0.9 and later have support for playing rtmp but no support for broadcast. meanwhile you need to remove some controls for rtmp connection handshake in vlc code and recompile it in order to make it work with red5. with vlc, playing quality was much better than flash player but with much delay. we couldn't investigate cause of delay yet. for broadcast we found a java code, but we didn't prefer to use it because it was not mature enough. we had to use adobe flash media encoder, that means windows dependency, something that we don't want. by the way you have to remove content type control in red5 codes and recompile it to be able to broadcast over rtmpt in adobe flash media encoder.

quality is more related with video interfaces you use rather than encode-decode issues. to be able to broadcast to red5 server from flash, your computer must recognize your video source as a webcam. if you use a capture card, you can use any source that you can connect to your TV (any TV channel, live screencasts, any camera with composite output, dvr cameras). it's important to move data over digital interfaces such as dvi and hdmi. component interface is the only interface supporting hd quality (1080 lines). but if you want high resolution and digital interfaces on a capture card, prices rises exponentially.

that's what i can remember so far, i can add new information as i remember. i mentioned about some key points casually, if you want some more detail about anything mentioned above, just leave a comment.

[1] http://www.terracotta.org/confluence/display/wiki/Red5+and+Terracotta+POC
[2] http://osflash.org/pipermail/red5_osflash.org/2007-December/017650.html
[3] http://gregoire.org/2009/01/28/rtmpt-and-red5/
[4] http://www.xuggle.com/xuggler/

2 comments:

Maxime C. said...

Great post, really.
If you meet these problems, anyone that would acheive the same goal will meet its too.
Because you suggest us to post if we want more details, can you explain a little bit more what bugs did you meet with Shared Objects ?
Thank you.

Thomas said...

Would it be possible that you give some details on how to setup rtmps for a Red5 application? It's really challenging.