Sunday, July 12, 2009

what i learned about red5, rtmp and flash

i'm living with red5 for last 5-6 months and i realised i've no entry about red5, i'll try to recover the loss now with one huge entry.

first of all, rtmp's specs are not open, so red5 is a reverse engineering product, and it's a pretty successful project if we consider thah facebook is using it. by the way, adobe had announced they'll open rtmp specs at first half of 2009, but yet they didn't.

rtmp is a protocol on tcp, i don't understand why it isn't udp. tcp is a lossless protocol, so in live video streaming it's hard to prevent delay if bandwith is insufficient. it's not possible to drop packets for incoming stream because you can't touch flash player code. it's only possible to drop packets in server side for stream subscribers, but you've to write some code for this, it's not a configuration thing.

if you need to do some scaling or re-broadcast incoming stream to other red5 servers, you've to be inside the code too. people prefer terracotta[1] for scaling. for re-broadcasting it's impossible to find sample code, but it's possible to find theorical solutions in email lists[2].

by default, red5 uses port 5080 for http, 1935 for rtmp, 8443 for rtmps and 8088 for rtmpt. so company firewalls are problem, generally only allowed port is 80. but we had to serve http for our webservice and rtmp for streaming from the same red5, this means two ports at the same time. we overcomed this obstacle with some servet settings in our web.xml[3]. rtmpt means http actually, transfers stream data with continous http requests. so it's possible to serve your webservice and stream server from the same port by adding this settings to your web.xml:


<servlet>
<servlet-name>rtmpt</servlet>
<servlet-class>org.red5.server.net.rtmpt.RTMPTServlet</servlet>
<load-on-startup>1</load>
</servlet>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/fcs/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/open/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/close/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/send/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/idle/*</url>
</servlet-mapping>


it wasn't easy to make ssl work, we recognized that rtmps does not mean rtmp over ssl, in fact there is no such thing as rtmps, it's more appropriate to say rtmps=rtmpts=https. we were using 0.7.4 early, but it just didn't work with ssl, we moved to 0.8.X versions, after some tries we succeeded to make it work at exact version 0.8-RC2, neither earlier nor later.

and in flash side, you should look flash restrictions before designing anything. for example, sizes of permission boxes in flash is constant in flash, if you create a smaller flash, user can't give permission to access camera/microphone. if user doesn't check remember option in permission box, s/he need to give permission for every flash one by one. if in your screen, a part of the flash is not in eyesight, it's not possible to click on any button on flash or you can't make flash fullscreen without a user click. these are adobe's security precautions. and you have to deal with many security related requirements if you want to serve flash from a different server and your website from another, i advice you to read flash's security policy first. this kind of small things can cause whole design changes totally.

and using shared objects for communication between flash clients and server is not a good idea, red5 implementation has some problems with shared objects. we prefered to use remote function calls (flash.net.NetConnection.call) instead.

we're developing software on linux, so writing actionscript was a question mark for us, but with flex3sdk and ant we did it without problem. however i must warn you, if you want to do some design job there is no mature tool for linux, so it wouldn't be a right choice.

we had to push the lines about quality to see if it's possible to use red5 in closed-circuit broadcasts. we saw that it's possible to catch dvd quality (480 lines) with flash player without much effort. dvd quality requires 7-8 mbit/s bandwidth, by default we couldn't exceed 1 mbit/s limit in upload at first time, no one mentions about this limit in any where. you need to add some code to your application to activate bandwidth controls in order to exceed this limit.

flash player couldn't play incoming stream effectively after 480 lines. flash player is capable to encode camera data with h.263 and vp6 codecs, it can't encode h.264 but can decode it to play. to reduce the amount of data going to server it's important to use h.264. by the way it's possible to re-encode incoming stream in server side with xuggler[4] before broadcasting it. we didn't want to add cpu load in server side so we wanted to encode it in client side. we looked for other alternatives rather than flash player to stream camera data, we couldn't find much, probably because of rtmp's closed specs. there were many rtsp clients, if there were something converting rtmp to rtsp, alternatives would be increased in considerable amounts.

vlc version 0.9 and later have support for playing rtmp but no support for broadcast. meanwhile you need to remove some controls for rtmp connection handshake in vlc code and recompile it in order to make it work with red5. with vlc, playing quality was much better than flash player but with much delay. we couldn't investigate cause of delay yet. for broadcast we found a java code, but we didn't prefer to use it because it was not mature enough. we had to use adobe flash media encoder, that means windows dependency, something that we don't want. by the way you have to remove content type control in red5 codes and recompile it to be able to broadcast over rtmpt in adobe flash media encoder.

quality is more related with video interfaces you use rather than encode-decode issues. to be able to broadcast to red5 server from flash, your computer must recognize your video source as a webcam. if you use a capture card, you can use any source that you can connect to your TV (any TV channel, live screencasts, any camera with composite output, dvr cameras). it's important to move data over digital interfaces such as dvi and hdmi. component interface is the only interface supporting hd quality (1080 lines). but if you want high resolution and digital interfaces on a capture card, prices rises exponentially.

that's what i can remember so far, i can add new information as i remember. i mentioned about some key points casually, if you want some more detail about anything mentioned above, just leave a comment.

[1] http://www.terracotta.org/confluence/display/wiki/Red5+and+Terracotta+POC
[2] http://osflash.org/pipermail/red5_osflash.org/2007-December/017650.html
[3] http://gregoire.org/2009/01/28/rtmpt-and-red5/
[4] http://www.xuggle.com/xuggler/

Saturday, July 11, 2009

red5 sample for flowplayer bwchek plugin

to be able to make this rtmp sample work, i had to understand whole flowplayer structure. i want to share the settings i'd done, maybe it could save your time.

index.html:
<html>
<head>
<script type="text/javascript" src="flowplayer-min.js"></script>
</head>
<body>
<div id="info">click below to see most appropriate video to your bandwidth</div>
<a class="rtmp" id="red5"><img src="img/showme.png" /></a>
<script language="JavaScript">
$f("red5", "swf/flowplayer-3.1.0.swf", {
log: {
filter: 'org.flowplayer.bwcheck.*',
level: 'debug'
},

clip: {
url: 'skyandice.flv',
urlResolvers: 'bwcheck',
live: true,
provider: 'rtmp'
},

plugins: {

bwcheck: {
url: 'swf/flowplayer.bwcheck-3.1.0.swf',
netConnectionUrl: 'rtmp://red5ip:1935/bwcheck',
bitrates: [40, 150, 400, 700, 1000],
serverType: 'red5',
rememberBitrate: false,

onBwDone: function(url, chosenBitrate, bitrate) {
var el = document.getElementById("info");
el.innerHTML = "Your speed is: " +bitrate+ "<br />Video file served: " +url;
}
},

rtmp: {
url: 'swf/flowplayer.rtmp-3.1.0.swf',
netConnectionUrl: 'rtmp://red5ip:1935/oflaDemo'
}
}
});
</script>
</body>
</html>


files used:

http://static.flowplayer.org/img/player/btn/showme.png # img altina
http://static.flowplayer.org/video/skyandice-40.flv
http://static.flowplayer.org/video/skyandice-150.flv
http://static.flowplayer.org/video/skyandice-400.flv
http://static.flowplayer.org/video/skyandice-700.flv
http://static.flowplayer.org/video/skyandice-1000.flv
# skyandice*.flv'ler /usr/lib/red5/webapps/oflaDemo/streams altına (artık siz nereye kurduysanız)
http://flowplayer.org/releases/flowplayer.bwcheck/flowplayer.bwcheck-3.1.0.swf
http://flowplayer.org/releases/flowplayer.rtmp/flowplayer.rtmp-3.1.0.swf
http://flowplayer.org/releases/flowplayer/flowplayer-3.1.0.zip
# flowplayer-3.1.0.swf, flowplayer.controls-3.1.0.swf, flowplayer-3.1.0.min.js de bu zip dosyasından geliyor
# tum swfler swf dizini altına


of course you have to install bwcheck and oflaDemo applications at server side from 'http://red5ip:5080/installer' url.

i couldn't find this information when i googled around, i hope it will be useful for other googlers.

to be able to do a job in anarchy

in zeitin we are working in a pretty free environment, it's more close to an anarchy rather than a democracy. our priority list is the only point where anarchy ends. when we were in parkyeri, we had weekly coordination meetings that department heads and co-founders attend to put jobs in order for every department in that week.

we are a small team in zeitin now, so we began to do weekly meetings which everyone attends, and we were trying to solve prioritization problems. but our anarchic system didn't allow us to continue these meetings. so as a solution we preferred to leave this job to one volunteer, giray, probably it was the best solution for a small team.

this week, alper got crazy because of a job i'd done that was not in our priority list. this is a side effect of anarchy, it's not so easy to estimate when and who will react when you do something. afterwards i learned that alper had wanted to do the job i'd done before me, but he had come across with the priority list obstacle when he tried to talk with giray and other guys.

it's more easy to estimate results of your moves in a more hierarchical structure or in a place with some rules, at least you'll know the guys who you're supposed to satisfy. but in an anarchic community you're supposed to satisfy everyone, and this is too ideal, not much possible practically. so there becomes no time without arguments, there is always someone complains about someone or something. we don't manage these complaints in our company, so even if you have nothing to do with a subject, you're finding yourself inside it. in fact, in zeitin there can't be a subject that you're not related with, because if you're working in zeitin, it means you've already accepted that you're ready to take responsibility of any thing related with the company.

i think the freedom we have in zeitin is not offered to any other company employee in the world, including google. people think that parkyeri and zeitin is inspired from google. if it's necessary to compare us with a company, i would prefer to be compared with SUN rather than google. what i can understand from outside, SUN is designed more like a school and employee initiative is really important. google seems like pretending to be some other thing rather than it really is. it says "don't do evil" but it forces android users to have a google account to activate their phones. it uses it's distribution power for monopolization. it makes propoganda by saying that "every employee is free to do what ever s/he wants at %20 of her/his work time", but it doesn't share the small detail that every employee need to take approval of their managers at first. the terms like "manager" is a sign of hierarchy inside. it opens two distinct projects with names chrome and chromium and tries to use community power one sidedly. this kind of things make me suspicious about company's sincerity.

in zeitin we prefer debian because it's the only distribution which could keep freedom as a reason to exist. generally other distributions try to use community power, they have other purposes but their way intersect with free software, so they are pretending to be some other thing than they really are like google does.

i know founders of zeitin, i was there when zeitin was started-up, i can assure you that zeitin is a company where freedom and happiness of employees has the highest priority, i hope it can go on like that.

but i can also assure you that freedom does not mean happiness, it's also not true to show the freedom as a cause of success, but i believe it's possible to be free and successful at the same time.

limitless freedom of everyone means anarchy, in anarchies it's normal to have disagreements that can ruin peace, democracy can be a solution to these problems. but it's not possible to adapt democracy to zeitin, because democracy brings rules with itself, laws means restrictions. in this system, by time, you're becoming subject to some rules you've never heard of.

instead of writing rules, keeping them as community morals in minds can be a solution to this problem, in practice you can allow public sanction to keep rules alive. in this case, there can't be any rule that you didn't hear. "knowing the rule" becomes precondition of a rule's existence.

in last example i experienced, i violated our moral of obeying the priority list, and the first obstacle i came across with was alper. nobody supported me so i was also defeated with public sanction. but i kept going on my way despite every thing, i said "let me do this job or just fire me", and i'd exceeded all the obstacles. most probably i'll get one more punishment with my low performance mark at the end of month, to prevent me doing something like this again. (for today performance evaluation job also is giray's responsibility, he gives marks to every one that affects their salary, so it's his choice to give this punishment or not. performance evaluation is the second point where anarchy breaks.)

the job i wanted to do was changing our releasing methodology, at parkyeri times we were using tarballs for our releases and i was despirately defending the idea that we should use deb packages instead, and we should have a private deb repository to keep our releases in and their dependencies that does not exist in official debian repositories. that's why it was a high-priority job for me but apparently giray was not in the same idea with me. if i'd obeyed giray's priorities we would probably not have a deb repository for the next 6 six years (if we think that parkyeri was a six-year old company which doesn't have a deb repository yet)

it was a critical job, because in an anarchic community, doing something in right time and right place will change too much thing for company's feature, it would trigger birth of a new moral and affect community culture. at first sight it seems like forcing others to your methods, but it's not. because if you force people to something unreasonable, they won't accept it and will resist. of course you must have smart and questioning people around to make it work. it's not enough to determine a methodology also you must stand with it to make it a cultural thing. i'm planning to take responsibility of all release jobs for a while, i'll tell my methodologies to people, i'll get feedback, i'll try to make new procedures mature enough. if i'm on the right way, people will accept them and begin to use them. otherwise i'll stay alone and my new procedures will not live long.

zeitin is a new company, people doesn't have some old habits, so my job will be relatively easy. if i let time pass, it wouldn't be enough to offer a reasonable method, i would also have to fight with old routines.

another way was trying to convince people before starting job. but this way is pretty expensive, it requires doing some meetings. for meetings you'll need to want people to stop what they're doing currently and you'll expect them to be focused on the subject as much as you are. and in this method, generally it becomes different people who offers the idea and who does the job actually. i see this a little risky, because ideas can be so unmature when it's offered, procedures should be kept improved during their life cycle not before.

if you'll do a job in an anarchic community, you should belive what you're doing, you should determine your motivations good, because you shouldn't allow what people says to break your motivation, you should make it a principle to finish what you're started.

as well as not letting people break your motivation, you should also not break other people's motivations. you should be careful about what you're saying, you should keep yourself away from purposeless sentences. but if you have a purpose, you should not avoid saying your word, because if you keep your silence, compony can go to a wrong way that you don't want. if you want some morals to be settled up, you should support them. (though i couldn't obey this rule, i was supporting existence of priority list normally, but i thought breaking rules for this time will have more incomes than the damage it'll create). you should also not avoid your word if you think someone doing something wrong, you should let people fix their mistakes, but always keep the possibility in your mind that you can also be wrong. it's not easy to care your words every time, i'm not so good at it as well, i can't control my patience if i feel someone attacks me, and i can break people's motivation when i become reactive.

actually there becomes many times i lost my peace in zeitin, i prefer peace rather than freedom. but when i look behind, in 3 month time we developed vidi which was definetly would last near 1 year if you try to make it done by a classical firm in same scale with us. and also we are inthe half way of gokada project that we did not officially announced yet, we are currently using thinclient solution of gokada in our company now.

apparently, in anarchy, with some moral rules, it's possible to get progress. as a developer, feel of success, feel of "doing something" is very important. i hope we can succeed to make some sales of our products and keep our company alive.