Programming

On one of the last pages of the Dangerous book for boys there is a list of books which every boy should read. Having read none, except of course The Hitchhiker's Guide to the Galaxy, which is considered required reading if you're ever to join the club of nerd.

So my quest this summer was to read up on my classics. I've so far read "Brave new world", "Lord of the flies", "Of mice and men", "Animal farm", "I Robot" and "The Foundation series". I've never been much of a fantasy or sci-fi man but after the first foundation book I was sold. The minute I came home I ordered the lot of them. I finished "Foundation's Edge" today and on my way home something struck me.

I was thinking how we software/web/whatever-developers deal with ethics at work. Will you write software for weapons industry? Will you write code to block free speech? Dealing with these questions are all personal. But Asimov can help. I find the three laws of robotics apply to engineering (software in this case) just as well as they do to robots.

1. Software may not injure a human being or, through inaction, allow a human being to come to harm.

2. Software must obey orders given to it by human beings except where such orders would conflict with the First Law.

3. Software must protect its own existence as long as such protection does not conflict with the First or Second Law.

The third law sticks out as it does not apply as well to software as it does to "hardware" (robots). But in a way it does work when you consider security. "Software must protect itself against haxoring as long as such haxoring does not conflict with the first or second law".

These laws are, of course, very hard to maintain. How can anyone be sure if your code, open source and free software especially, isn't used for evil? But then again as long as code is not self-aware, the keeper of the laws should be us humans. We, as developers, have an obligation to do the right thing when it comes to "instructions" we create.

"Who'll join me!" I exclaimed when the news of a 24 hour 'lets see what can be done with php'-competition was announced. 3 brave colleges answered the call.

And so began what was to be a long day of code, sleep deprivation and caffeine.

We, the 'hot pink crusaders', signed up for the PHP Throwdown competition and desided to create a game. A planetarion-ish game called 'Incursion'. Mark and I actually graduated on a c / c++ / ncurses version of this game and we've since played with the idea of porting it to the web. And this competition, so it seemed, was a great chance to revitalize the concept.

The 'new' Incursion took 4 people 24 hours to get to a point that it nearly worked. It never made it beyond the suck phase though. It was a good exercise and we had lots of fun trying. There was just not enough time.

I believe I've spent a total of 7 hours trying to get some kind of battle calculation algorithm working. I had to 'invent' this from scratch and it proved to be extremely difficult during the wee hours. I got it working about an hour before the competition ended. A bit too late.

Our plan was to get something 'playable' before halftime and then just fix the bugs and tweak it so that it would be enjoyable to play. We never had a chance :).

The code we checked in at the end is full of fatal errors for which we just lacked the time to fix. Also, security wise, its quite a mess. We've made sure to filter all data going through Smarty but if register_globals is on, it will be easy to by-pass the 'dispatcher' file and go straight to the controllers which are wide open for all the world to see because none of us had the time to include a .htaccess that would prevent said access. Oh well.

Now we have to wait for the judging to commence.

The judging of our application is based on the following criteria:

Code Readability & Documentation (5 pts)
Installation/Configuration (5pts)
Functionality (20pts)
Performance (5pts)
Error handling & Security (15pts)
UI/Appearance (5pts)
Accessibility/Standards Compliance (5pts)
Code Innovation/Creativity (30pts)
Overall Quality of the App (10pts)

Oh boy.

My predictions are..

Code Readability & Docs
I say; 3 out of 5 points. We started out fine, trying to create a bit of structure, trying to be verbose. But that quickly faded into a code frenzy trying to get stuff working as fast as possible. So I give us 3 points.

Instalation and Configuration
3 points. We actually made an effort to create a database creation script and config file. The only thing needed for installation is a php 5.1 and higher, mysql 4, the cron and a little fiddling in the config.

Functionality
5 out of 20 points. Well the app barely works. So what it does, it does badly. I predict not very many points in this category.

Performance
4 out of 5 points. It's not fast, but its certainly not slow either. Incursion is made for 'smaller' groups of people. So some performance loss might happen when a lot of users are playing. The battle code for instance, does a lot of work.

Error handling & Security
5 out of 15 points. We have basically no error handling and security isn't much better. But if you have register_globals Off and close down all the private directories it should be pretty air-tight. So perhaps 5 points is a bit too low.

UI/Appearance
2 out of 5 points. Our UI sucks. Its very very basic. We've changed the way it looks a couple of time and we lacked the design-skill to make it look pretty. Basically we needed a hands on designer dude. But we didn't have anyone. Ever seen coders create a UI. Yeah.. thats how it looks. The tabs are nice though but there is just a sense of unity missing.

Accessibility/Standards Compliance
4 out of 5 points. We tried to adhere to the standards as much as possible. We tried making the html output conform with XHTML Transitional and CSS. But having done virtually no compliance/validation checks I can't say how we did. Also, the HTML is a mess. Did I say that already?

Code Innovation/Creativity
10 out of 30 points. Did we innovate? I don't know. We have some nice pieces of code, but the whole 'planetarion' concept is not very innovative. The thing where we differ from planetarion is that its made for small fast-pased games. We tried to make it into a 'planetarion for LAN gaming' kind of game where you can join a lobby and the administrator will start the game. The winner will then be determined by having the highest score. Which means having the most kills. What might also be innovative is the way we handle the battles, since we designed this from scratch.

Let me show you how its done (I'm quite proud of this, so bear with me):

Unit statistics:

Name            Offence Defense Type    Strength    Weakness

Marine          10      5       light   light       light

Commando        13      10      light   medium      medium

Raptor          25      20      medium  light       medium

Goliath         50      40      heavy   medium      heavy

Battlecruiser   100     150     heavy   heavy       light



Our fleet:

10  Marines

5   Commandos

10  Goliaths

1   Battlecruiser



Opponents fleet:

30  Marines

3   Commandos

10  Raptors

When they attack the 'attacking' fleet fires first, after which the defending fleet (if they are still standing) fire back. In Incursion this happens once after which the fleets return home.

Every unit has a weakness and strength against certain types of units. The attackers targets its 'strength' the its 'not strength, not weakness unit' and after that his 'weakness'. It also incurs a penalty for the amount of shots that hit target based on the type of unit it prefers.

The bonuses:

Shots that 'hit' based on its strength are 80% +- random(0..10)% amount of total shots. Shots that 'hit' based on its weakness are 30% +- random(0..10)% amount of total shots. Shots that 'hit' everything else are 50% +- random(0..15)% amount of total shots.

So basically, you always miss a few shots.

10 Marines fire 10 shots each. They prefer 'light' units so the 'light'-typed units of the opponent are targeted first. In this case the 'Marines' and the 'Commandos'.

The amount of shots that hit target are:

(80 + 5)% of a 100 shots = 85 Shots hit in total for all marines shooting.

The attacking marines target the opponents marines who have a collective 30*5=150 total defense.

So 85 shots hit 150 of total defense. This results in a total 65 defense left on the opponents side. 85 damage also means 85/5 losses on the opponents side, resulting in 17 dead and 30-17=13 marines left to see another day.

In this case all the 'hitting' shots are absorbed by the total defense of all the opponents marines. However! there is a possibility that the amount of damage being dealt exceeds the total defense the opponent provides. When this happens the 'hitting' shots will be transfered to hit the next group of units. In this case the 'Commandos' and going trough the same process as before.

After the marines fire, our commandos get a shot and everything goes through the motions again.

There. Thats what I spent the last 24 hours on trying to figure out.

Last but not least, quality.

Overall quality of the app
3 out of 10 points. I try to spend a lot of my time approving quality in code-bases and of the app in general. But in 24 hour, how much quality control can you have? If there are any sacrifices to save time, you bet it will be quality that pay's the price.

Anyhow, this post has become quite long enough. Let me just say that I've had lots of fun competing and hope I'll be around long enough to be able do it again. Just not within a month or two.

You know that strange thing that when you have to explicitly handle output filtering, things will inevitably go wrong? Well I do.

You see, I've been using Smarty as my templating buddy for some time now but once in a while I get bitten in the ass by me not handling output filtering and thus having HTML injection 'holes' in my applications.

The problem starts with the fact that I have to 'work' to get filtering included. What I actually want is filtering to be 'on by default', so that I can shut it off when needed. That way the default is 'safe' (there are always other way to get compromised of course) except in those cases that I explicitly turn the filtering off.

I want Smarty to do my filtering! "But Smarty already does this!" I hear you say, and I'd have to agree. Except Smarty does this by providing the "escape" modifier. But I don't want to write '|escape:"html"' after every bloody variable I use in my templates. I want Smarty to filter the little buggers by default.

I know what you're thinking. "No no no, you've got it all wrong. You should handle filtering at the logic layer and not at the presentation layer". Well.. then you'd be wrong. Since HTML (and Javascript) injection is only present in the presentation layer I want my presentation layer to handle the filtering. I can't really be bothered at the logic layer, since the problem does not exists there. I do, however, need to filter the input at the logic part of my code, but thats another story.

So what needed to be done was, reverse the way you handle escaping in Smarty. This means variables need to be escaped by default and can be 'unescaped' using the newly crafted 'unescape' modifier.

Ok, so how exacly does one handle this. Well, Smarty has a hand full of usefull API hooks with which you can 'plug' into the compiling process. I chose to use a 'postfilter' to process the template vars and encode them before they end up in the template.

You could also use a pre-filter since they both do their stuff after the all the variables are assigned and before the output is actually shown. I chose postfilter because this is the last possible point in the compilation process before the variables are forged into the template and pushed out to whatever device is doing the request.

So how does this look, code-wise:


Ok so now that the filters are in place, we need to test the stuff.

The php file:


The template file:


And voila, the output is nicely escaped and safe by default. Oh the joy of carelessness. I can't really show you a demo since I'm too lazy to set one up, so you need to test this for yourself.

Using this has some obvious advantages and some disadvantages. Unicode support, for instance, is an issue. Though htmlspecialchars can handle charset conversion quite nicely, it has to be provided. So you have to know the exact charset you are sending out for this to function like it should. It defaults to ISO-8859-1 so the not-so-internationalized applications can use this whithout worries.

Another thing which might be annoying is that all the HTML you use in variables will inadvertally be escaped. One place where we've had this problem is when dealing with language files. In some places we used HTML inside the language strings, these got escaped as well. Whoops!

Also performance could be slow when handling large sets of data. So you would need to do some benchmarking before putting stuff like this in production.

UPDATE: After reading some discussion on this very problem I've noticed another issue with my implementation.

Doing this:


Is doing a comparison on an encoded value and could possibly go wrong. Then again, this will probably not happen as long as you're not comparing to any 'weird' variables which contain quotes or other unholy characters. WHO DOES THAT ANYWAY.

Also my implementation does not function in sync with the current 'escape' functionality using options like html, htmlall, url etc for describing the type of escaping to be used.

A possible solution for the use of escaped vars in expressions would be changing the post-filter a bit. Instead of looping through all the template vars we could preg_replace the echo code in the actual compiled template. Personally I think that would be quite a gross hack. Not even thinking about the implications when using other modifiers. I like my first implementation better even though it requires an extra function and even though its not complete.

The stuff I read on the smarty forum gives some me hope that Smarty is going to have output escaping functionality of its own. Which is good and saves me the time hacking 'nice-yet-not-quite-right' solutions to these problems.

Since the release of PHP5, the object model has changed. While it 'fixes' a lot of 'wrongs' I still get confused as to the how it should be used and what the result should be.

My biggest gripe is that the 'where should references be defined' part is not enforced.

By example:



This confuses me. Which of these implementations is preferable over the other? Should I just pick one and 'not worry be happy'? Why isn't one of these enforced.

I would say that the second implementation is the 'prefered way', because it doesn't force the reference on you, and leaves the choice of $var being a reference to the person using the function.

Even more confusing is the fact that the following also works without a hitch:



I know the manual states:

They are not like C pointers; instead, they are symbol table aliases.

But its still confusing nonetheless. In my opinion E_STRICT should give me a notice (at least) to inform me of my questionable behaviour. But it doesn't, it just ignores me and goes on its merry way.

After I read Terry Chay's post about "PHP Coders", I wondered about what skill you would possibly need to call yourself a "professional" webdeveloper. I, for one, believe that there is a lot more to it then PHP/HTML/CSS/Javascript skill alone. So this is my addition to the "Grill your applicants!" question pool.

Mine are mostly about all the stuff non-php-javascript-html-css.

Has she ever worked with high traffic websites? How does one administer those? What can you do to speed up websites on both Win as Linux systems? How does one secure a Linux box? How does one work with a linux box securely? How well does she work the command line? Does she know shellscripting? Perl? Python? bash? Does she know vim or emacs or any ohter texteditor via commandline? How do you prevent data coruption? How do you handle data coruption? How do you set up backup systems? And in what situation is one backup setup preferable to the other? How do you make sure your data is transfered safely?

What about webservers? Has she ever administered Apache or IIS?. Has she ever compiled PHP? Could he tell me why Apache2 is not preferable for PHP and why Apache1.3 is? Does she know how to secure a apache server? How do you tune apache? Could she tell me how DNS servers work? Could she tell me how HTTP works?

How about database servers? Is she a phpmyadmin kinda girl (red flag!)? or does she use the command line? Has she ever installed and administered MySQL? How does one tune a MySQL server? How about PostgreSQL? What other database server does she have experience with? What are the specific defining features off both database servers? What are the situations for which you choose the one over the other? Why does or doesn't oracle suck for the web? When should you use sqllite, when should you use something else? How do MySQL indexes work? How do you optimize MySQL queries? How does PostgreSQL autovacuum work? Why is it useful? How does MySQL replication work? What does myisamchk do?

Also, Coding in general. What are best pratices when coding websites? Has she ever used version control? What kind? Why are those useful? Which editors does she use? Does she know how to write Unit-tests? Does she know how to write maintainable code? Does she know unicode? How do you handle internationalisation?

Doing webdevelopment is more then just pure coding skill. I value the whole "webdeveloper enviroment work experience" greatly. I, for one, would very much like to be sure that when the shit hits the fan (and it always hits.. badly too) that she would be able to handle the crisis accordingly. If you haven't had to simultaniously reinstate a mysql backup while making sure you're other database slaves arent crumbling under the extra load and handling multiple phone calls as to "WHY THE FUCKING SITE AINT WORKIN!" while your boss is stomping behind you because "WE'RE LOSING MONEY HERE DAMNIT! FIX IT NOW!".... Then you aint no webdeveloper.