Ensure that photographs were uploaded quickly without
failure and an ability to resume in case of loss of connections
Ensure that upload and download of photographs which
consume a lot of bandwidth and background server capacity
does not affect the functioning of the regular website activities
Ensure fast processing of user requests for information-particularly
on the editor's side who are sifting through hundreds of submissions
Ensure that bulk import of photographs from partner sites were carried out
seamlessly and without interrupting the user experience
Original Images from photographers are High Quality and therefore large sized.
It was imperative to ensure that the sizes were retained for final delivery to
publishers, but at the same time, the large size should not slow down the speed
at which the images are delivered to the editors while searching through the submissions
To meet the above challenges and to deliver the ideal solution for the client we looked to
the solutions available through the Amazon cloud.
The first task was to identify the resource intensive processes and identify how the load
could be split between the various activities. Image upload and download was identified as
the biggest consumer of resources in terms of server capacity and bandwidth. This was followed
by serving up of submissions to the editors.
The next task was to identify how the processes could be optimized to provide the best user experience. It was clear that
multiple servers would need to be in place to handle the activities. The split was decided as follows. The main application
server would only host the code and handle the requests from users that involved database interaction. There would be a
standalone database server (with replication implemented using a master-slave methodology) to handle all database
requests. Uploads and downloads would be handled on a stand-alone server which would be seamless to the user for all
interactive purposes. This would take the heavy image processing load off the main application server as these activities
do not require a real-time response and can be undertaken on a batch processing basis. All images would be stored on
the Amazon S3 cloud to take advantage of their CDN (Content Delivery Network) for quick delivery of images to users
without taxing the load on the main application server to serve up static content.
The final task was to create a background process to handle bulk processing activities like processing uploaded images to create web-friendly image sizes while retaining the original HD pictures for final delivery; batch processing of download requests and bulk imports from partner sites.