Resource Intensive PHP Tasks part 5

Well we are finally here, end of this 5 part session.

By now you should have learnt principles that you will need in building your next project with long running task.

However if you are running enterprise application or otherwise an application that may need high volume you may need to consider a more formal method of serving your users.

Below you will find some of my recommended sites.

Please note I do not get paid anything when you follow this recommendations, I only have them here because I think they will be useful to you.

Open source queuing libraries

  • PHP-Resque: Built by the guys who built github, this is one of the most powerful yet easiest to implement queuing system out there. Its biggest weakness is that it borrows a lot from Ruby and may not be intuitive to use for hard core PHP devs. See more here
  • Mongo Queue: Schemaless databases are gaining a lot of ground in the dev community due to their relatively faster response times. If you are part of this wave, you may want to consider Mongo Queue for your job. It runs on Mongo db. See more here
  • Laravel Queue: Being a sucker for Laravel you all knew it had to make an appearance right 🙂 Seriously the Queue module for laravel is one of the easiest to configure and use. Plus it provides integration with third party commercial queue services if you need it. If there was no other reason to switch to Laravel this is it! See more

Commercial queuing services

  • At 10 million free API calls per month, this is the definitely the service you want. Not to mention integration is easy and is supported by numerous libraries. As a matter of fact Laravel supports this out of the box!! See more
  • Amazon SQS: Amazon is known for reliability and scalability. This is for you if you have some money to spend and reliability and scalability are an issue. With Amazon you get 1 million calls free
  • Google Queue: This is last on the list because its only relevant to Google App engine users. This is not yet supported in PHP as of today (July 2013) but beta tests are in way to support php. In the meantime it doesn’t hurt to check out the docs for python

I believe you now have the knowledge to build that next resource hungry task. Devs, go forth and build the world!!


by Jacob Chencha


Resource Intensive PHP Tasks part 4

Since telling is nothing without the showing. We have a very simple implementation of the above work.

The resource being created is a text document with time set every ten second apart for an iteration of ten.

Now this is a trivial application but should be able to show what we need. You can get a copy of the source files from github

Without further a do here are the files.

For this particular implementation I have chosen to choose one file for progress and one for storage, they are imaginatively named store.txt and progress.txt. For the application to work this files must be readable and writable to apache.

For your own application you definitely want to use a better database not to mention provide a way of identifying the different workers.

Next time we will talk about libraries and tools that can help you in your development process.

Remember to follow me on twitter/ subscribe to my feed.

Till next time happy coding!!

by Jacob Chencha



Resource Intensive PHP Tasks part 3

The design mock up


Simplified design for a long running php process
What we have above is the proposed app overview.

Here is a brief overview of what is actually happening:

  1. Requester (User) Sends a request for a resource to the system. The resource could be HTML, Report etc
  2. Controller receives the requests and returns a HTTP 202 response. See for more details on HTTP status codes.
  3. Controller sends job parameters to model
  4. Controllers sends the url that will call the appropriate worker to the queue manager
  5. Queue manager receives job request
  6. Queue manager evaluates systems resources and decides sends requests to workers. This can be done via curl. The queue manager should not wait for a response
  7. Worker does the time intensive task and on completion (successfully or not) sends output back to the Queue manager
  8. The Queue manager updates the progress to the model
  9. Requester(User) sends request for the current progress.
  10. Controller checks progress from the model, if complete returns the resource if not, returns the progress information that is available

Obviously not all work flows will fit this suggested work flow, regardless this should at least come close.

Next time we are going to go through a sample implementation of this design.

If you have any questions please drop a comment below. Remember to follow me on twitter.

by Jacob Chencha