Scalable Analytics; High Traffic Website Measurement

Understanding Change

Now available: Real-time high volume traffic reporting. Think hundreds of thousands, millions of visitors and page views! We have made our Most Popular Pages report on-demand.

We start this article with a quote from the Xoriant blog which discusses architectural considerations for high traffic web portals:

“To withstand the heavy traffic, the system should primarily be scalable, be highly available and should be able to intelligently delegate/distribute the traffic to improve the overall performance.”

The Solution

This development project was accomplished with two technologies: Hadoop and Cassandra. The result is a fast, scalable, cutting-edge engine which is built on a platform characterised as “infinitely scalable”. We have tested this claim successfully with millions of pageviews per day on a single site.

This cloud-processing model is used by companies such as Twitter, Google, Amazon, and Facebook (it’s Facebook technology – Facebook started Cassandra). This technology is now available to all users of Opentracker.

Case Study

An online newspaper with millions of daily visitors spoke with us. They needed to know what stories are getting the most play on their website. They need this information now, in real-time, from minute to minute. This information will let them manage content and maximize advertising potential. The website content editors need to know what their readers are interested in.

Findings : 

Tools are needed to manage the traffic flow and also to manage the content people are looking for. For example, to produce or purchase more content if interest is growing. Websites also need to take advantage of traffic streams to generate revenue by displaying appropriate advertising. They also need to take advantage of opportunities by informing potential advertisers about opportunities.

The same scenario applies to social media sites, traffic portals, sports sites, television, media, video entertainment content providers (think Youtube, NFL, or HBO), gambling and adult content sites. This information is critical for managing content and advertising streams. Robust, scalable, traffic measurement is absolutely critical. Traffic movement can be volatile on the internet, the world’s attention span is dynamic, take a look at Twitter over a 24-hour period to get an idea. Millions of visitors can come and go within the space of an hour or less. Visitors are interested in different areas like international news, world sports championship events, large-scale media events, gossip, celebrities, movies, hollywood, etc.

Conclusion 

The reporting technology will deliver the capability to see where traffic attention shifts, as it happens. This reporting will give web masters the ability to monitor network traffic loads where scale is an issue.

Q: Who is this technology meant for?
A: Anybody who needs to know what is happening on a website while it happens. People who manage site with fluctuation and traffic spikes. People who need to make decisions based on visitor behaviour.

This technology is ready to deliver. Contact us for details and pricing