Sign Up!
* indicates required


Lambda Chat

AWS Lambda is the future, it's the new normal. Great! Awesome.

Can you do anything useful with it?

We took that as a challenge. Could we, using AWS Lambda, create a multi-user, dynamic web application without using EC2 or resorting to hacks? I mean no instances, no ELBs, not even a security group. And especially no hard-coded AWS credentials.

We did, and it's called Lambda Chat.

Live demo

http://lambda-chat.s3-website-us-west-2.amazonaws.com/

Code on GitHub

https://github.com/cloudnative/lambda-chat/

Lambda Chat is a simple chat application using AWS Lambda, SNS, DynamoDB and S3.

Static websites can be hosted on S3, so we use it to store the HTML, CSS and JavaScript. Each chat message becomes an SNS message, which is processed by AWS Lambda and stored in a DynamoDB table. Sounds simple enough but actually, this poses a few more challenges:

Challenge 1) How do we send SNS messages from Javascript?

Remember, one of our goals was no hacks. It turns out AWS has a solution for this: Web Identity Federation. This allows users to sign in using some existing account (Amazon, Google and Facebook (they just added Twitter too)). Signing in issues a token to the user, which can then be used to call AWS's Security Token Service (STS) to issue temporary AWS credentials. Those credentials only last an hour, and are used to publish messages to SNS. (Note: Check out the AWS Web Identity Federation Playground to learn more)

Challenge 2) How do we read the data back out?

We have temporary AWS credentials -- we could read the message out of DynamoDB directly. Hmmm, that means giving the front end more permissions. Not hard, but can it be avoided?

Why not use DynamoDB Streams to call another Lambda function. This function can read the data out of DynamoDB once, and write it to a JSON file stored on S3. DynamoDB is fast, but not for scans. By doing it this way, we can store the last 20 messages in S3, which acts as a DynamoDB cache, and it means we can keep the permissions down to just SNS publishing.

The resulting flow becomes:

    ◎ ◎
     ◡

     │(1)
     │                       ┏━━━━━━━━━━━━━━━━━━┓
     │                       ┃                  ┃
     │        (2)            ┃  Google OAuth2   ┃           (4)
     │        ┌─────────────▶┃       API        ┃◀────────────┐
     │        │              ┃                  ┃             │
     │        │              ┗━━━━━━━━━━━━━━━━━━┛             │
     ▼        ▼                                     ┏━━━━━━━━━━━━━━━━━━┓
    ┏━━━━━━━━━━━━━━━━━━┓                            ┃                  ┃
    ┃                  ┃ (3)                        ┃     AWS STS      ┃
 ┌─▶┃     Website      ┃◀──────────────────────────▶┃  AWS Web ID Fed  ┃
 │  ┃                  ┃                            ┃                  ┃
 │  ┗━━━━━━━━━━━━━━━━━━┛                            ┗━━━━━━━━━━━━━━━━━━┛
 │            │
 │            │        ┏━━━━━━━━━━━━━━━━━━┓
 │            │        ┃                  ┃
 │            └───────▶┃    SNS Topic     ┃
 │            (5)      ┃                  ┃
 │                     ┗━━━━━━━━━━━━━━━━━━┛
 │                               │
 │                               │       ┏━━━━━━━━━━━━━━━━━━┓
 │                               │       ┃                  ┃
 │                               └──────▶┃    Lambda fn1    ┃
 │                               (6)     ┃                  ┃
 │                                       ┗━━━━━━━━━━━━━━━━━━┛
 │                                                 │
 │                                                 │      ┏━━━━━━━━━━━━━━━━━━┓
 │                                                 │      ┃                  ┃
 │                                                 └─────▶┃  DynamoDB Table  ┃
 │                                                 (7)    ┃                  ┃
 │                                                        ┗━━━━━━━━━━━━━━━━━━┛
 │                                                                  │
 │                                    ┏━━━━━━━━━━━━━━━━━━┓          │
 │                                    ┃                  ┃          │
 │                                    ┃    Lambda fn2    ┃◀─────────┘
 │                                    ┃                  ┃        (8)
 │                                    ┗━━━━━━━━━━━━━━━━━━┛
 │                                              │
 │            ┏━━━━━━━━━━━━━━━━━━┓              │
 │            ┃                  ┃              │
 └────────────┃    S3 Object     ┃◀─────────────┘
 (10)         ┃                  ┃            (9)
              ┗━━━━━━━━━━━━━━━━━━┛

Diagram created with Monodraw

Steps:

  1. The user opens their browser and goes to the website which is hosted entirely on S3
  2. The user signs in with their Google account and gets back an id_token
  3. Using AWS Web Identity Federation in the Javascript SDK, the id_token is sent to get temporary AWS credentials from STS.
  4. STS verifies the token with Google
  5. The users types in a message, hits ENTER, and the website publishes the message to an SNS Topic.
  6. A Lambda function is triggered by the SNS message, which has the contents of the message, and...
  7. Stores the message in a DynamoDB table
  8. The process of adding a new chat message to the DynamoDB table triggers another Lambda function. This requires the currently-in-preview DynamoDB Streams feature. This second Lambda function reads the last 20 messages from DynamoDB, and...
  9. Writes them to an S3 object in JSON format
  10. The website polls the S3 object every second, and updates the chat box with any new messages it finds.

As mentioned above, we have open sourced this, and made it kinda-sorta easy to get up and running in your own account if you wish.

  • Our CloudFormation script (using troposphere) brings up all the AWS resources
  • An update script is used to push the website code to S3
  • Kappa packages and deploys both Lambda functions
  • Easy clean up process

Even though we suck at JavaScript, I think it is pretty easy to understand. I would love to see feedback or PRs on making the functions follow NodeJS best practices though.

Now a note on the latency: we purposely didn't fake the display of the message you wrote when you hit ENTER. We wanted you to see what the latency is. I have seen it as high at 13 seconds, and as low as 2. Interestingly, it gets faster the more people use it.

And now for the cost: This thing costs just a few dollars per month, and in theory scales near infinitely. We would need to increase the read/write capacity on the DynamoDB table, but not until a few 100 people are using it concurrently. Chat rooms tend to be self-limiting anyway, as humans can only read so many messages in a given time.

Lambda Chat is no Slack replacement. It is however, a well documented and open source example of how to do something useful using AWS Lambda. Thanks for reading.

Learn more about Mobot today!

Copyright © 2019 MinOps, Inc.