Friday 14 June 2013

Reference Architecture - Auto-scaling Moodle deployment on AWS

AWS provides a highly scalable and reliable cloud hosting environment for Moodle. With features like Auto Scaling Groups, Elastic Load Balancers and CloudWatch Monitoring services, Moodle deployments on AWS can be configured to automatically scale up and down seamlessly to meet the highs and lows in the demand curve in the most optimum and cost-effective manner. This post presents reference architecture for deploying Moodle Learning Management System over AWS Cloud to achieve high levels of Performance, Scalability, Availability, Security and Reliability.


Architecture Components

Moodle Web Tier
Moodle Web Tier is composed of multiple EC2 instances that act as Web Servers running Linux OS with Apache and PHP. The EC2 instances are bootstrapped using a pre-configured Amazon Machine Image containing necessary software stack along with Moodle Software.

These EC2 instances are launched automatically by Auto Scaling Groups based on an auto-scaling policy that utilizes data from CloudWatch Monitoring Services. CloudWatch monitoring services are configured to generate alerts when they see changes in the demand curve. Auto Scaling Groups subscribe to these alerts and increases / decreases EC2 instances to meet the demand curve. Auto-scaling groups are placed in different Availability Zones to provide high-availability. Even if a failure occurs in one availability zone, user’s would not experience any problems as they will automatically be served from the other availability zone.

HTTP Session state is maintained in Dynamo DB which is a highly available and scalable No-SQL database. This approach enables the Web Servers to be completely stateless and users can be served from any Web Server, which results in effective distribution of traffic.

Moodle Content Repository
Static content and resources including video, audio, images and multimedia objects embedded inside Moodle courses are stored in Amazon S3 Storage Service (registered as a repository in Moodle). Amazon S3 is a highly durable and scalable storage infrastructure that is highly available. Objects stored in Amazon S3 are delivered via Amazon CloudFront that automatically edge-caches these objects and routes requests to the nearest edge-location for speedier delivery.

MoodleData Folder is stored on S3FS mounted drives on Web Servers (EC2 Instances) backed by centralized storage in Amazon S3. Instead of S3FS mounted drives, you also have the option to setup a NAS (Network Attached Storage) Server using EC2 instance with attached EBS volumes in AWS.

Moodle Database Tier
Amazon RDS MySQL service is being used to host Moodle database. Amazon RDS is a managed Relational Database Service that automatically patches the database software and backs up database and support point-in-time recovery. It provides the flexibility to scale the compute resources or storage capacity associated with the Moodle database to achieve high levels of performance. RDS Multi-AZ deployment enables to maintain a stand-by replica of master Moodle database in another availability zone to fall back in case of any failures in the master database. RDS Service automatically syncs the standby copy with the master database.

Application Services
AWS provides a host of services that can be utilized in Moodle Deployment included Simple Email Service for sending emails, SNS for generating notifications, SQS for integrating Moodle with other applications via queues. Moodle also supports multiple authentication options including LDAP Integration that can be configured on AWS.

8 comments:

  1. This couldn't have come at a more perfect time. Do you know how many users this deployment supports? And Do you have the guide to setup that?
    Thanks for the great write-up

    ReplyDelete
    Replies
    1. Walter.. The architecture uses auto-scaling groups that are infinitely scalable. You can configure the max no of instances in your auto-scaling group to limit the scalability according to your budget. The number of concurrent users your deployment can handle is proportional to the number of web server instances you can provision to handle the load.

      Delete
    2. Would also be very interested in a more detailed guide on how to set this up. I understand all the individual pieces but not how you bring them all together.

      Delete
  2. Thanks for your Attention Mathur !!! Just more one question. Do you have details to setup DynamoDB to maintain Http Session state?

    ReplyDelete
    Replies
    1. Not sure about your particular use case, but if you are using PHP, refer http://docs.aws.amazon.com/aws-sdk-php-2/guide/latest/feature-dynamodb-session-handler.html

      If you are using ASP.NET, refer http://docs.aws.amazon.com/AWSSdkDocsNET/latest/DeveloperGuide/net-dg-dynamodb-session.html

      Delete
  3. Hi Shailendra,

    Excellent article. I found your design very interesting and worth giving it a try.

    Did you utilize this reference architecture for any large client with 1000+ concurrent users? I am interested in knowing whether this design is well tested in a production environment for general adoption. Typically, how many users does one EC2 instance supports (based on your personal testing)?

    Please help me out.

    Warm Regards
    Sathees Donthy

    ReplyDelete
  4. Have you actually set this configuration up?

    ReplyDelete

  5. It was great experience after reading this. thanks for sharing such good stuff with us.
    AWS Course in Delhi

    ReplyDelete