Resource Intensive PHP Tasks part1

This is the first in  a five part session that will teach you the basics of designing a projects that is resource intensive.


So you now have a couple of applications under your belt, maybe even mastered a framework or two. Barring special circumstances you have come to expect response from the server in the order of milliseconds. Then baaaam comes a project with various resource intensive jobs and everything goes whack!!
Below are some of the problems that you are likely to encounter if you don’t plan for this resource hungry tasks:
1. Server will cut you off: The default limit for php scripts is 30 seconds. You could always modify this by changing the execution time and in fact for worker scripts this maybe inevitable but doing this is really just sweeping the problem under the carpet, which is just not a good idea, there are spiders there! Furthermore if you are in a shared scripting environment then things are a tad worse for you because you just don’t have access to php.ini file and are more likely than not unable to change global values.
2. You will run out of resources: Unlike most other scripting languages, php does not repeat file descriptors. For small to medium tasks this is hardly ever a problem for bigger projects however this can escalate very fast. See
3. Google will hate you: While Google’s ranking algorithm remains propriety, it is an open secret that page load does affect rank. Now unless you have a couple of millions to spare in advertising budget, you don’t want to find yourself in the netherworld that is page 2 of Google search!. For interested readers, you may read more here
4. The user will hate you: LOVE YOUR USERS. We can never over emphasize this. Slow and hanging apps amount to right out insulting them. Now you may argue that this is an internal app or that you, the developer, will be the only user but that is irrelevant and besides the point, when developing think of the user as a separate entity. You will thank yourself later.

There is a lot of argument as regards php and task intensive tasks I think the argument there is irrelevant besides we love PHP through the good and the bad right 🙂

Now that I have presented the challenge let me present the suggested solution. Note what I will be talking about here is design of your application and how to make it ready for long running tasks.

Watch out for part 2 of this series coming to you tomorrow.

If you like this, please follow me on twitter use the tabs to your right.

Thank you

by Jacob Chencha


Why your first app will fail

So you have just learned your new shiny language (or a couple) and now are wondering what to do next. For most, the options boil down to

  • Join a web development agency (get employed)
  • Become a freelancer
  • Create a new product and sell it.

If you are like me, you probably jumped right into no 3 and started coding your ideas. Hours will be burnt as you toil for many hours at a go until you finally have your product for publishing.

You publish your product on the app markets (if thats what you are into) or buy up a domain for your web app and start drumming up support all the while waiting for users.

Then it happens, Nothing. Not even one single user. What could have gone wrong?

Dunning–Kruger effect

You have just been hit by the Dunning-Kruger effect (D-K)!

The definition according to wikipedia for the D-K effect:

“The Dunning–Kruger effect is a cognitive bias in which unskilled individuals suffer from illusory superiority, mistakenly rating their ability much higher than average. This bias is attributed to a metacognitive inability of the unskilled to recognize their mistakes.”

Now I know you are probably that can’t be right, besides you can’t think of an application that you can’t engineer in you chosen language and you have practically read all the marketing blogs you could get your hands on.

Well unfortunately things are just not that easy and some of the following factors have certainly been playing again you:

  1. You will sub consiously over emphasize on the particular area of expertise that you are good at: For example developers good in scoping and requirements will come up with a very detailed SRS (Sofware requirements document) while those skilled in database design will create great designs but fail to focus on the user experience and so on and so forth.
  2. Feedback will be hard to get: If you have had to maintain someone else’s code or even your own code, you know how much more effort it takes to understand and process it. Couple this with the relatively low number of professional developers, you find yourself in a case where no one gets to see your code and its weaknesses, therego you never improve.
  3. You are smart and you know it: Most developers I have interacted with fall somewhere in the higher end of the bell curve. This probably served them well through their formal years of education. However you (the developer) will very soon realize that while raw intelligence counts for some, market experience counts for so much more, unfortunately it takes a flopped launch (or several) for you to get that.
  4. Low barriers of entry: This probably did serve you well as you we’re starting out. No one cared about your credentials and tutorials we’re a heap. Fortunately this resource is available to all other devs unfortunately most will just never use it. Thus you will have a lot of devs who couldn’t do the fizzbuzz test have already had a run in with you future clients. This is bad because by now they have already developed a negative stereotype to new devs.

Suggested ways to beat it

Here are some of my ideas on how to successfully navigate the muddy waters of D-K.


  1. Join a team: If you are a natural extrovert this should be easy however for the rest of us introverts this may take some effort. By joining a team, most of the problems I highlighted will most certainly disappear. If you don’t follow any other advice below, Follow this one.
  2. Give back to the community: Not every project you make needs to be commercial. Open source it! Yes it took time to make but the feedback about the quality of your code and the improvements that can be made will improve your worth as a developer dramatically.
  3. Help new developers: It may seem like a give-give relationship but in reality you may actually be getting more than you are giving. This is because by teaching you are forced to test the assumptions of all that you think you actually do know.
  4. Think small: If you have attended some formal training, you have probably been drummed with hype of huge pay. Even if you are an autodictat you have probably read success stories. The reality however is that what made this entrepreneurs rich was not their first project, something in the range of the 100th or worse. As such don’t pressure yourself too much, you will eventually make it but for now, build light applications and get them coming fast. This will help you build the necessary experience for your big one as well as allow you ample time to make the mistakes that will undoubredly show up.

The more discerning reader may have noticed I have mentioned nothing about spending more time coding your heart away. There is a reason for that. This industry just like all other industries is all about the people.

Your ability to connect with other developers, engineers, designers and most importantly clients is what will truly set you apart.

Thank you.

Other resources


If you liked this post and would like to recieve more please click on subscribe and like our page.




Resource Intensive PHP Tasks part 2

The requirements

Like the good developers we are we will kick of things by defining some requirements to help us in the design process.

But first some definitions.

Controller: PHP Script that directs receives and responds to requests

Worker: PHP Script that carries out resource/time intensive task

Model: PHP Script that provides the data framework. This can be anything from file system to the DB connection.

With that out of the way here are our requirements:

  1. System must respond within 5 seconds
  2. Controllers must not under any circumstances run for more than 30 seconds CPU time.
  3. Controllers should be able to report progress
  4. Controllers must be able to run on shared hosting service
  5. Workers must be able to operate independently.
  6. Workers should never have to interact with the file system.
  7. Workers should be easily exportable
  8. Workers should be RESTful
  9. Workers should have one verb to accept work. Common convention is to call this verb fire or index.
  10. Model must never return system resource handles
  11. Model must provide expected value types even on failure


A fair lot of the requirements are self explanatory. If you would like me to expound some more on any of this please comment as much below. Otherwise it will become obvious why we have the requirements in a bit.


Next article we will look at a sample work flow and the request life cycle. We will also get to see the relationship between the objects in our system.

by Jacob Chencha