Sunday, April 4, 2010

after free software and linux days

one 'free software and linux days' organisation is behind again. for us, who are former parkyeri employees, it's also like a graduate gathering. thus we find chance to see old friends.

to list what i gained from this activity this year; i got aware of a new association for IT employees called 'bitder', and i catched two new keyword from brian king's presentation, one is jetpack and the other is drumbeat.

by the way, at brian king's presentation, the demonstration of what can be done with css on a video at firefox was quite impressive. i'll get a nightly built firefox to play with it a little bit at the earliest opportunity.

drumbeat is a platform leaded by mozilla, where philosophical discussions take place to decide how will internet's social profile be at next 10 years. in other words a non-technical one of W3C. as i understand, they're trying to create a community to get long term strategic decisions to protect and improve the internet as an open, participatory, decentralized and public resource, by keeping an eye at human values.

jetpack is an add-on devolopment kit letting devolopers to create new add-ons by just basic html, css, javascript knowledge without dealing with xul. and also add-ons devopeloped with jetpack do not require a restart of firefox at installation. of course they're not as powerful as xul ones, but jetpack is ideal for simple add-ons like my imkbizle. i guess it will cause number of people increase who develop add-ons.

(and a parkyeri advertising break :) now i understand the privileges of working at parkyeri better as i'm away. for example it's almost impossible not to be aware of something like jetpack if you work at parkyeri, for sure somebody will mention it, and you'll be an earwitness by some way. i saw that i'd been also away from rest of the world as i got away from parkyeri. by the way i'm afraid, if parkyeri keeps founding new companies at this speed, there will be no place to other sponsors' logos at organisation's poster.

if we look at activity generally this year, i think it was a little dead in compare to past years, i don't know maybe it's just same but i'm getting older. but when i look at the number of posts at turkish debian e-mail list, there is an obvious decrease. i guess growing of free software community slows down. on the other hand i feel an increase at the number of linux users at my around. maybe the connections between linux users are getting thinner by time, because people do not behave linux users as an alien today. and i think the linux users were more like each other, there was a more common profile, but today there is a more variety among linux guys.

after all it was an efficient and connective organisation, thanks to everyone who were participated in, i hope we can keep this activities go on forever.

Sunday, January 17, 2010

what the ... is siberfiber?

I am often exposed to this question and usually my attempt to tell people what siberfiber is ends with failure, and finally I decided to write it down.

siberfiber is an a computer-aided management software developed by intimatek, and supported by TUBITAK. it is aiming to create an environment for business analysts to make them capable of producing industry and company specific enterprise resource planning (ERP) and bussiness process management (BPM) solutions, without any need for a programmer.

what makes siberfiber a R & D project is it's original approach to some ancient problems. a major problem in an ERP software is reflecting changes in a resource, to the other resources that are affected by this change. programmers are the ones who analyses resources and designs the data model and they create necessary tables on database according to this design. but the general problem of classical database systems is to keep relationship between data models one-sided.

well what does that mean? assume that we have two resources 'teacher' and 'student', if there is a 1 to n relationship between these two resources (1 teacher can have multiple student, 1 student can have only 1 teacher), on conventional database systems, general method is adding a column to 'student' table to hold the ID of the teacher. in this case, by looking at the schema of 'student' table, it's possible to see realationship between 'teacher' and 'student' resources, but the 'teacher' table's schema won't have any information about this relationship. if you need the 'x teacher's students' knowledge, you have to hire a programmer to write a new sql query for you.

siberfiber is developed based on declarative programming rather than procedural programming, all the relations between resources on siberfiber is two-sided. this two approach solves basical problems from the root even without they born.

on siberfiber, the relationship between teacher and student can be seen on both 'student' and 'teacher' types' schemas, because all relations are established at definition level.

let's make our sample a little more complex, for example, we need 'x teacher's successful students' information, we need to add a new column with name 'grade' for 'student' table. and then give our programmer the definition of 'successful student' as 'students with a grade higher than 50'. in this case, our programmer will write the following sql for us:
select * from student, teacher where teacher.id=student.teacher_id and student.grade => 50;
and on siberfiber this logic will be expressed by this sfs [1], at the time when 'teacher' and 'student' types are created:
createsifter "teacher" "successful students" n "student" "succeeded teacher" 1 filtervaluesgreaterthan student.grade 50
(i'll not do any explanation about sfs now, unfortunately it's impossible to explain it by 3 words, I think at this stage it would be an unnecessary detail, but if I hadn't given this example it would be too abstract)

sfs script at above is creating a property (at classical database system, properties can be considered as columns in the table) both at 'student' type and 'teacher' type.

even though no one had asked us 'which teachers did this student passed from?' question, siberfiber's two-sided relations made this answer ready automatically and unavoidably. in fact, what we do here is binding two types with a rope, and there is a sign by this rope at student side on which 'goes to teachers whose lessons are passed' is written. and in the other side, teacher side, on the sign what's written is 'goes to students who are passed the lesson' text. it's possible to travel in both directions on this rope.

usually in conventional databases you can only go down by a rope, you can not go up. to get the 'teachers who are passed' knowledge you have to pull another rope, which means programmers have to write some more sql for you.

by two-way relationship, at siberfiber space it's possible to update all types that are affected by changes occured at any type instantly and automatically (in fact i mean 'thing' which means instance of types at siberfiber terminology by saying 'type' here).

at ERP software world, normally business analysts are the ones who identify process and resources, then they would tell programmers what to do, they would expect programmers to know what they are talking about as much as they are, then programmers began to code in the light of what they understood, and there becomes a software. or the programmer had done same kind of jobs for that sector too much, and (s)he'd turned into a bussiness analyst-programmer, and (s)he was thinking that (s)he is the god of that bussiness, (s)he doesn't need anyone in the world.

in siberfiber, definitions are given directly to the system, not to the programmer. of course, we don't expect business analysts to write sfs code, for this, an analyst designer interface is being written, siberfiber definitions can be expressed with a notation smilar to BPMN [2]. business analysts, by using this design interface, will be able to create these definitions, sfs codes created by these definitions will update system at real-time.

and of course, a CAM[3] sofware as siberfiber will need a user interface for it's users, well, who will create this interface then? answer is business analysts again, they will create this user interface by siberfiber definitions called masks. masks determine how and which part of a type will be shown in the interface. for example, grade property of 'student' will be shown as a read-only field at student interface, and it will be shown as an editable textbox on teacher interface. mask definitions are also done via analyst designer interface graphically, as it's done in type definitions.

I hope, I could succeed to give an idea about siberfiber. as you can estimate, this is the only visible part of the iceberg. if I'd tried to tell more, it would be unnecessarily confusing, and maybe it became so even for now.

[1] Siberfiber Script
[2] Bussiness Process Modeling Notation
[3] Computer Aided Management

need to patch qooxdoo selenium plugin

qooxdoo consumed from end of my life once again. I tried to write a simple selenium test, and I got lost in a hopeless struggle to click on a button.

first of all, in order to write a selenium test for your qooxdoo application you need this plug-in. and also you need to use an old version of firefox and qooxdoo framework in order to use this plug-in.

as a conclusion, my struggle resulted with this patch, in this way it became possible to work with firefox 3 and qooxdoo 0.8.3.

but tests are not applicable to a built (. / generate build) qooxdoo application, and it's not nice to be have to put test specific setUserData method calls into your application.

I don't know how long my struggle with qooxdoo will go on.

married and changed my job

since my last entry, a number of important changes happened in my life. finally esra and I succeeded to get married (we're still paying its debts) and I started to work at mynet.

i'm at mynet for 3 months, if we consider that the trial period was 2 months, either they forgot to fire me or i've become an official mynet employee.

this is my transfer story to mynet:

zeitin had to move to a new bussiness model in which employees need to do sacrifice to keep company exist. because of my marriage I was unable to make any sacrifice at that time, so I had to abandon the ship at a time that I wouldn't prefer much.

I couldn't return back to parkyeri, because I didn't want to ruin motivation of people at zeitin. and i had a desire to learn what's going at the world beyond parkyeri, as a person who opened his eyes at parkyeri. also i couldn't go on at intimatek, because in this way it would be a shame against mavci.

during my job search, i was specially looking for network engineering and test engineering positions. but for the salary level I need, employers were asking for experience on sector. and my marriage again forced me to keep hurry, i began to apply positions for which my background is suitable, and I found myself at mynet. it'd been nearly 4 months at mynet, as a software developer on adklik project.

in addition to the adklik project at mynet, I'm trying to continue with siberfiber at my spare time, as a hobby. siberfiber is a nice project supported by tubitak and devoloped by intimatek co. which is founded by giray (one of co-founders of parkyeri) and his siblings, it's impossible to explain siberfiber with 3 words, I want to make a separate entry for it, later.

to summarize, i left vidi and gokada projects devoloped by zeitin, and I've been continuing my life with adklik and siberfiber projects.

that's it.

Sunday, July 12, 2009

what i learned about red5, rtmp and flash

i'm living with red5 for last 5-6 months and i realised i've no entry about red5, i'll try to recover the loss now with one huge entry.

first of all, rtmp's specs are not open, so red5 is a reverse engineering product, and it's a pretty successful project if we consider thah facebook is using it. by the way, adobe had announced they'll open rtmp specs at first half of 2009, but yet they didn't.

rtmp is a protocol on tcp, i don't understand why it isn't udp. tcp is a lossless protocol, so in live video streaming it's hard to prevent delay if bandwith is insufficient. it's not possible to drop packets for incoming stream because you can't touch flash player code. it's only possible to drop packets in server side for stream subscribers, but you've to write some code for this, it's not a configuration thing.

if you need to do some scaling or re-broadcast incoming stream to other red5 servers, you've to be inside the code too. people prefer terracotta[1] for scaling. for re-broadcasting it's impossible to find sample code, but it's possible to find theorical solutions in email lists[2].

by default, red5 uses port 5080 for http, 1935 for rtmp, 8443 for rtmps and 8088 for rtmpt. so company firewalls are problem, generally only allowed port is 80. but we had to serve http for our webservice and rtmp for streaming from the same red5, this means two ports at the same time. we overcomed this obstacle with some servet settings in our web.xml[3]. rtmpt means http actually, transfers stream data with continous http requests. so it's possible to serve your webservice and stream server from the same port by adding this settings to your web.xml:


<servlet>
<servlet-name>rtmpt</servlet>
<servlet-class>org.red5.server.net.rtmpt.RTMPTServlet</servlet>
<load-on-startup>1</load>
</servlet>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/fcs/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/open/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/close/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/send/*</url>
</servlet-mapping>

<servlet-mapping>
<servlet-name>rtmpt</servlet>
<url-pattern>/idle/*</url>
</servlet-mapping>


it wasn't easy to make ssl work, we recognized that rtmps does not mean rtmp over ssl, in fact there is no such thing as rtmps, it's more appropriate to say rtmps=rtmpts=https. we were using 0.7.4 early, but it just didn't work with ssl, we moved to 0.8.X versions, after some tries we succeeded to make it work at exact version 0.8-RC2, neither earlier nor later.

and in flash side, you should look flash restrictions before designing anything. for example, sizes of permission boxes in flash is constant in flash, if you create a smaller flash, user can't give permission to access camera/microphone. if user doesn't check remember option in permission box, s/he need to give permission for every flash one by one. if in your screen, a part of the flash is not in eyesight, it's not possible to click on any button on flash or you can't make flash fullscreen without a user click. these are adobe's security precautions. and you have to deal with many security related requirements if you want to serve flash from a different server and your website from another, i advice you to read flash's security policy first. this kind of small things can cause whole design changes totally.

and using shared objects for communication between flash clients and server is not a good idea, red5 implementation has some problems with shared objects. we prefered to use remote function calls (flash.net.NetConnection.call) instead.

we're developing software on linux, so writing actionscript was a question mark for us, but with flex3sdk and ant we did it without problem. however i must warn you, if you want to do some design job there is no mature tool for linux, so it wouldn't be a right choice.

we had to push the lines about quality to see if it's possible to use red5 in closed-circuit broadcasts. we saw that it's possible to catch dvd quality (480 lines) with flash player without much effort. dvd quality requires 7-8 mbit/s bandwidth, by default we couldn't exceed 1 mbit/s limit in upload at first time, no one mentions about this limit in any where. you need to add some code to your application to activate bandwidth controls in order to exceed this limit.

flash player couldn't play incoming stream effectively after 480 lines. flash player is capable to encode camera data with h.263 and vp6 codecs, it can't encode h.264 but can decode it to play. to reduce the amount of data going to server it's important to use h.264. by the way it's possible to re-encode incoming stream in server side with xuggler[4] before broadcasting it. we didn't want to add cpu load in server side so we wanted to encode it in client side. we looked for other alternatives rather than flash player to stream camera data, we couldn't find much, probably because of rtmp's closed specs. there were many rtsp clients, if there were something converting rtmp to rtsp, alternatives would be increased in considerable amounts.

vlc version 0.9 and later have support for playing rtmp but no support for broadcast. meanwhile you need to remove some controls for rtmp connection handshake in vlc code and recompile it in order to make it work with red5. with vlc, playing quality was much better than flash player but with much delay. we couldn't investigate cause of delay yet. for broadcast we found a java code, but we didn't prefer to use it because it was not mature enough. we had to use adobe flash media encoder, that means windows dependency, something that we don't want. by the way you have to remove content type control in red5 codes and recompile it to be able to broadcast over rtmpt in adobe flash media encoder.

quality is more related with video interfaces you use rather than encode-decode issues. to be able to broadcast to red5 server from flash, your computer must recognize your video source as a webcam. if you use a capture card, you can use any source that you can connect to your TV (any TV channel, live screencasts, any camera with composite output, dvr cameras). it's important to move data over digital interfaces such as dvi and hdmi. component interface is the only interface supporting hd quality (1080 lines). but if you want high resolution and digital interfaces on a capture card, prices rises exponentially.

that's what i can remember so far, i can add new information as i remember. i mentioned about some key points casually, if you want some more detail about anything mentioned above, just leave a comment.

[1] http://www.terracotta.org/confluence/display/wiki/Red5+and+Terracotta+POC
[2] http://osflash.org/pipermail/red5_osflash.org/2007-December/017650.html
[3] http://gregoire.org/2009/01/28/rtmpt-and-red5/
[4] http://www.xuggle.com/xuggler/

Saturday, July 11, 2009

red5 sample for flowplayer bwchek plugin

to be able to make this rtmp sample work, i had to understand whole flowplayer structure. i want to share the settings i'd done, maybe it could save your time.

index.html:
<html>
<head>
<script type="text/javascript" src="flowplayer-min.js"></script>
</head>
<body>
<div id="info">click below to see most appropriate video to your bandwidth</div>
<a class="rtmp" id="red5"><img src="img/showme.png" /></a>
<script language="JavaScript">
$f("red5", "swf/flowplayer-3.1.0.swf", {
log: {
filter: 'org.flowplayer.bwcheck.*',
level: 'debug'
},

clip: {
url: 'skyandice.flv',
urlResolvers: 'bwcheck',
live: true,
provider: 'rtmp'
},

plugins: {

bwcheck: {
url: 'swf/flowplayer.bwcheck-3.1.0.swf',
netConnectionUrl: 'rtmp://red5ip:1935/bwcheck',
bitrates: [40, 150, 400, 700, 1000],
serverType: 'red5',
rememberBitrate: false,

onBwDone: function(url, chosenBitrate, bitrate) {
var el = document.getElementById("info");
el.innerHTML = "Your speed is: " +bitrate+ "<br />Video file served: " +url;
}
},

rtmp: {
url: 'swf/flowplayer.rtmp-3.1.0.swf',
netConnectionUrl: 'rtmp://red5ip:1935/oflaDemo'
}
}
});
</script>
</body>
</html>


files used:

http://static.flowplayer.org/img/player/btn/showme.png # img altina
http://static.flowplayer.org/video/skyandice-40.flv
http://static.flowplayer.org/video/skyandice-150.flv
http://static.flowplayer.org/video/skyandice-400.flv
http://static.flowplayer.org/video/skyandice-700.flv
http://static.flowplayer.org/video/skyandice-1000.flv
# skyandice*.flv'ler /usr/lib/red5/webapps/oflaDemo/streams altına (artık siz nereye kurduysanız)
http://flowplayer.org/releases/flowplayer.bwcheck/flowplayer.bwcheck-3.1.0.swf
http://flowplayer.org/releases/flowplayer.rtmp/flowplayer.rtmp-3.1.0.swf
http://flowplayer.org/releases/flowplayer/flowplayer-3.1.0.zip
# flowplayer-3.1.0.swf, flowplayer.controls-3.1.0.swf, flowplayer-3.1.0.min.js de bu zip dosyasından geliyor
# tum swfler swf dizini altına


of course you have to install bwcheck and oflaDemo applications at server side from 'http://red5ip:5080/installer' url.

i couldn't find this information when i googled around, i hope it will be useful for other googlers.

to be able to do a job in anarchy

in zeitin we are working in a pretty free environment, it's more close to an anarchy rather than a democracy. our priority list is the only point where anarchy ends. when we were in parkyeri, we had weekly coordination meetings that department heads and co-founders attend to put jobs in order for every department in that week.

we are a small team in zeitin now, so we began to do weekly meetings which everyone attends, and we were trying to solve prioritization problems. but our anarchic system didn't allow us to continue these meetings. so as a solution we preferred to leave this job to one volunteer, giray, probably it was the best solution for a small team.

this week, alper got crazy because of a job i'd done that was not in our priority list. this is a side effect of anarchy, it's not so easy to estimate when and who will react when you do something. afterwards i learned that alper had wanted to do the job i'd done before me, but he had come across with the priority list obstacle when he tried to talk with giray and other guys.

it's more easy to estimate results of your moves in a more hierarchical structure or in a place with some rules, at least you'll know the guys who you're supposed to satisfy. but in an anarchic community you're supposed to satisfy everyone, and this is too ideal, not much possible practically. so there becomes no time without arguments, there is always someone complains about someone or something. we don't manage these complaints in our company, so even if you have nothing to do with a subject, you're finding yourself inside it. in fact, in zeitin there can't be a subject that you're not related with, because if you're working in zeitin, it means you've already accepted that you're ready to take responsibility of any thing related with the company.

i think the freedom we have in zeitin is not offered to any other company employee in the world, including google. people think that parkyeri and zeitin is inspired from google. if it's necessary to compare us with a company, i would prefer to be compared with SUN rather than google. what i can understand from outside, SUN is designed more like a school and employee initiative is really important. google seems like pretending to be some other thing rather than it really is. it says "don't do evil" but it forces android users to have a google account to activate their phones. it uses it's distribution power for monopolization. it makes propoganda by saying that "every employee is free to do what ever s/he wants at %20 of her/his work time", but it doesn't share the small detail that every employee need to take approval of their managers at first. the terms like "manager" is a sign of hierarchy inside. it opens two distinct projects with names chrome and chromium and tries to use community power one sidedly. this kind of things make me suspicious about company's sincerity.

in zeitin we prefer debian because it's the only distribution which could keep freedom as a reason to exist. generally other distributions try to use community power, they have other purposes but their way intersect with free software, so they are pretending to be some other thing than they really are like google does.

i know founders of zeitin, i was there when zeitin was started-up, i can assure you that zeitin is a company where freedom and happiness of employees has the highest priority, i hope it can go on like that.

but i can also assure you that freedom does not mean happiness, it's also not true to show the freedom as a cause of success, but i believe it's possible to be free and successful at the same time.

limitless freedom of everyone means anarchy, in anarchies it's normal to have disagreements that can ruin peace, democracy can be a solution to these problems. but it's not possible to adapt democracy to zeitin, because democracy brings rules with itself, laws means restrictions. in this system, by time, you're becoming subject to some rules you've never heard of.

instead of writing rules, keeping them as community morals in minds can be a solution to this problem, in practice you can allow public sanction to keep rules alive. in this case, there can't be any rule that you didn't hear. "knowing the rule" becomes precondition of a rule's existence.

in last example i experienced, i violated our moral of obeying the priority list, and the first obstacle i came across with was alper. nobody supported me so i was also defeated with public sanction. but i kept going on my way despite every thing, i said "let me do this job or just fire me", and i'd exceeded all the obstacles. most probably i'll get one more punishment with my low performance mark at the end of month, to prevent me doing something like this again. (for today performance evaluation job also is giray's responsibility, he gives marks to every one that affects their salary, so it's his choice to give this punishment or not. performance evaluation is the second point where anarchy breaks.)

the job i wanted to do was changing our releasing methodology, at parkyeri times we were using tarballs for our releases and i was despirately defending the idea that we should use deb packages instead, and we should have a private deb repository to keep our releases in and their dependencies that does not exist in official debian repositories. that's why it was a high-priority job for me but apparently giray was not in the same idea with me. if i'd obeyed giray's priorities we would probably not have a deb repository for the next 6 six years (if we think that parkyeri was a six-year old company which doesn't have a deb repository yet)

it was a critical job, because in an anarchic community, doing something in right time and right place will change too much thing for company's feature, it would trigger birth of a new moral and affect community culture. at first sight it seems like forcing others to your methods, but it's not. because if you force people to something unreasonable, they won't accept it and will resist. of course you must have smart and questioning people around to make it work. it's not enough to determine a methodology also you must stand with it to make it a cultural thing. i'm planning to take responsibility of all release jobs for a while, i'll tell my methodologies to people, i'll get feedback, i'll try to make new procedures mature enough. if i'm on the right way, people will accept them and begin to use them. otherwise i'll stay alone and my new procedures will not live long.

zeitin is a new company, people doesn't have some old habits, so my job will be relatively easy. if i let time pass, it wouldn't be enough to offer a reasonable method, i would also have to fight with old routines.

another way was trying to convince people before starting job. but this way is pretty expensive, it requires doing some meetings. for meetings you'll need to want people to stop what they're doing currently and you'll expect them to be focused on the subject as much as you are. and in this method, generally it becomes different people who offers the idea and who does the job actually. i see this a little risky, because ideas can be so unmature when it's offered, procedures should be kept improved during their life cycle not before.

if you'll do a job in an anarchic community, you should belive what you're doing, you should determine your motivations good, because you shouldn't allow what people says to break your motivation, you should make it a principle to finish what you're started.

as well as not letting people break your motivation, you should also not break other people's motivations. you should be careful about what you're saying, you should keep yourself away from purposeless sentences. but if you have a purpose, you should not avoid saying your word, because if you keep your silence, compony can go to a wrong way that you don't want. if you want some morals to be settled up, you should support them. (though i couldn't obey this rule, i was supporting existence of priority list normally, but i thought breaking rules for this time will have more incomes than the damage it'll create). you should also not avoid your word if you think someone doing something wrong, you should let people fix their mistakes, but always keep the possibility in your mind that you can also be wrong. it's not easy to care your words every time, i'm not so good at it as well, i can't control my patience if i feel someone attacks me, and i can break people's motivation when i become reactive.

actually there becomes many times i lost my peace in zeitin, i prefer peace rather than freedom. but when i look behind, in 3 month time we developed vidi which was definetly would last near 1 year if you try to make it done by a classical firm in same scale with us. and also we are inthe half way of gokada project that we did not officially announced yet, we are currently using thinclient solution of gokada in our company now.

apparently, in anarchy, with some moral rules, it's possible to get progress. as a developer, feel of success, feel of "doing something" is very important. i hope we can succeed to make some sales of our products and keep our company alive.