AWS

If you’ve read my previous post on AWS Lambda, you already know how magical it feels to just upload your code and let AWS handle the rest.

But here’s a question most beginners eventually ask:

“If Lambda can already run my code, why do people still use EC2?”

Good question. Let’s break it down in plain English.

–

EC2 — The Always-On Office PC

EC2 is like renting your own computer in the cloud.
You control everything — OS, software, uptime — but you also pay even when you’re not using it.

Perfect for:

• Apps that need to run 24/7.
• Consistent traffic or background services.
• Workloads needing custom setups (special libraries, daemons, etc.).

⚠️ Downside: You manage scaling, patching, and cost.

–

Lambda — The Cloud Vending Machine

Lambda, on the other hand, is event-driven magic.
You just drop in your code, and it runs only when something triggers it.

If you want to understand Lambda in detail (handlers, events, roles, limits, etc.),
check out my earlier post AWS Lambda Basics.

Here, let’s keep things simple.

You pay only when it runs, it scales automatically, and it shuts down when done.
But you give up some control — you can’t run long-lived processes or manage your own environment.

Real-Life Example: SQS → DynamoDB Forwarder

Imagine you have a small data forwarder:
it listens to messages in SQS, processes them, and stores results in DynamoDB.

Let’s see what happens with both EC2 and Lambda

–

EC2 Version

You set up an EC2 instance, install your app, and keep it running 24/7, polling SQS every few seconds.

✅ Pros

• Full control and visibility.
• Works great when messages keep coming in all day.
• You can fine-tune performance (threads, caching, retries).

❌ Cons

• Still running (and billing) even when the queue is empty.
• You manage scaling, logs, and health checks.

Billing vibe: Pay per hour. Idle? Still billed.

–

Lambda Version

You configure SQS as a trigger for your Lambda.
When a message arrives, AWS spins up your function, processes it, and shuts it down.

✅ Pros

• Pay only when messages arrive.
• No servers, no scaling worries.
• Handles bursty traffic automatically.

❌ Cons

• Time-limited execution (max 15 min).
• Cold starts add slight delay.
• Harder to debug long or stateful logic.

Billing vibe: No messages = no cost.

Which One Fits You?

Simple analogy:

• EC2 = rent a car → you maintain it.
• Lambda = GrabCar → you just ride when needed.

–

In real projects, both often coexist:
EC2 runs the main services, while Lambda handles small, event-based tasks.

Start simple — use Lambda for event-driven bits, and bring EC2 in when you need always-on power.
AWS gives you both tools so you can pick what fits the moment.

If you’ve been using Amazon DynamoDB for a while, you’ve probably noticed something: not all your data gets the same attention. Some of it is “hot” — frequently accessed, constantly updated. But a lot of it is “cold” — just sitting there, costing you money every month.

What if you could store that cold data somewhere cheaper without changing your code or losing availability?

That’s exactly what DynamoDB Standard-IA (Infrequent Access) is for. In this post, we’ll break down what it is, how it works, how it can save you money, and when it might not be the best idea.

Recap on previous post:

• AWS: DynamoDB & DAX Cost Factors
• AWS: DynamoDB Basics

DynamoDB Table Classes

DynamoDB offers different table classes to optimize costs based on your access patterns:

1. Standard – For data you access frequently.
   • Designed for low-latency access any time.
   • Best for your main, active application data.
   • The default table class when you create a new DynamoDB table.
   • Suitable for most workloads. Provides high availability and durability.
2. Standard-IA (Infrequent Access) – For data you rarely read or write.
   • Designed for data that is not accessed often but needs to be available when needed.
   • Offers lower storage costs compared to Standard.
   • Higher retrieval costs, so it’s best for data that you access less than once a month.

Both table classes work exactly the same way from a developer’s point of view:

• Same APIs
• Same queries
• Same AWS Console experience

The only difference? How AWS stores it behind the scenes and how much you pay.

What is Standard-IA?

Standard-IA is a table class designed for data that you access infrequently. It’s like a storage locker for your cold data — it’s still there when you need it, but it costs less to keep it around.

Think of it like moving your old books to a basement shelf:

• They’re still yours.
• You can still get them any time.
• But they’re not taking up expensive prime shelf space.

How can it save you money?

The main savings come from storage pricing:

That’s about 60% cheaper for storage.

Example: If you have 100 GB of archived order history:

• Standard = ~$25/month
• Standard-IA = ~$10/month

💡 That’s $15/month saved — or $180/year — just for one table.

The Catch

But it’s not all sunshine and rainbows. There are some important trade-offs to consider:

• Retrieval Costs – Around $0.01 per GB each time you read data from IA.
• Minimum 30-Day Storage Billing – You pay for at least 30 days even if you delete earlier.
• Not for Hot Data – If accessed often, retrieval fees can eat up savings.
• Whole-Table Setting – You can’t mix Standard and IA in one table.

Best Practice before Switching

• Check Access Patterns — Use CloudWatch metrics to see how often the table is read.
• Move Predictable Cold Data — Avoid sudden spikes in retrieval.
• Test on a Smaller Table First — See if retrieval costs are low enough to justify the switch.
• Combine With TTL — Automatically delete expired data to save more.

DynamoDB Standard-IA is like a budget-friendly storage locker for data you still need but rarely touch. It can cut storage costs by more than half — but only if you choose the right workloads.

Rule of thumb: If it’s predictable, cold, and still worth keeping — IA is your friend.

We discussed about DynamoDB in previous post, but let’s dive deeper into the cost factors associated with DynamoDB and its accelerator, DAX (DynamoDB Accelerator).

Recap

• Amazon DynamoDB (DDB) is AWS’s fully managed NoSQL database service, designed for applications that require consistent performance at any scale.

• Amazon DynamoDB Accelerator (DAX) is an in-memory caching service for DynamoDB. Think of it as a turbocharger — it reduces read latency from milliseconds to microseconds by storing frequently accessed data in memory.

Together, DDB and DAX can significantly improve application performance — but they also come with different cost models you’ll want to understand before adopting.

When to Use DAX?

DAX is particularly useful when:

• Your workload has high read traffic with repeated queries for the same items.
• You want microsecond read latency for real-time user experience.
• You aim to offload read traffic from DynamoDB to reduce provisioned read capacity usage.

Example: A database for AI model training, where the same training data is accessed repeatedly.

Skip DAX when:

• Your workload is write-heavy with low read repetition.
• Your queries are strongly consistent (DAX only supports eventually consistent reads).
• Your access patterns are highly dynamic and unpredictable — the cache hit rate might be low.

Understanding DynamoDB Costs

DynamoDB costs come from three main areas:

• Reading Data
  • Imagine a reading allowance — every time you check a page from a book, it uses part of your allowance.
  • You can either pay per read (On-Demand) or buy a monthly “reading subscription” (Provisioned) if you know your usual usage.
• Writing Data
  • Adding or updating books also uses an allowance — think of it as your “writing subscription” or “per-write” payment.
• Storing Data
  • This is your bookshelf space.
  • Regular storage (Standard) is always ready but costs more.
  • Cheaper storage (Standard-IA) is for books you rarely read, but you’ll pay a small fee each time you take one.

Extras You Might Pay For:

• Backups — like taking daily photos of your bookshelf.
• Copies in other regions — like having the same library in multiple cities.

Understanding DAX Costs

5. DAX Costs DAX pricing is per node-hour, depending on node type:

• Smallest node (dax.t3.small) is the cheapest, suitable for dev/test.
• Larger nodes (dax.r5.large, etc.) cost more but handle higher throughput.
• DAX clusters require at least 3 nodes for fault tolerance in production.

Note: DAX charges are separate from DynamoDB — even if your reads come from the cache.

Cost Comparison

If your cache hit rate is low, DAX might increase costs without much benefit.

Final Thoughts

• Use DAX if you have heavy, repeated reads and need lightning-fast results.
• Use Standard-IA storage for rarely accessed data — but don’t forget the retrieval cost.
• Always measure first: monitor read/write usage and cache hit rates before committing.

Serverless is one of the most exciting ways to build modern cloud applications — and AWS SAM makes it even easier.

What is AWS SAM?

AWS Serverless Application Model (SAM) is an open-source framework that helps you build and deploy serverless applications on AWS.

It’s designed to simplify your infrastructure-as-code, especially when working with:

• AWS Lambda
• API Gateway
• DynamoDB
• EventBridge, SQS, Step Functions, and more

At its core, SAM is just a shorthand syntax for AWS CloudFormation — making your templates cleaner, easier to write, and faster to iterate.

Why Use AWS SAM?

You should consider SAM when:

• You’re building serverless applications using AWS services like Lambda and API Gateway.
• You want to define infrastructure as code but find raw CloudFormation too verbose.
• You want to test Lambda functions locally using Docker.
• You prefer guided deployment over manually zipping and uploading code.

SAM is ideal for:

• Quick prototyping of serverless apps
• Developer teams who want simplicity without giving up AWS-native IaC
• Learning how serverless works with real AWS infrastructure

SAM vs. CloudFormation: what’s the difference?

SAM is built on top of CloudFormation, so it inherits all the benefits of CloudFormation while providing a simpler syntax for serverless applications. Here are some key differences:

In short: SAM is CloudFormation — just way easier for serverless.

You still get all the benefits of CloudFormation (rollback, drift detection, etc.), but with less effort and boilerplate.

SAM Main Components

1. template.yaml

Your SAM template is the blueprint of your application — it declares all the AWS resources your app needs.

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.12
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

2. samconfig.toml: Configuration for Reusability & Environments

When you run sam deploy --guided, SAM generates a samconfig.toml file. This file stores deployment settings like your S3 bucket, stack name, region, and parameter overrides — so you don’t need to type them every time.

But beyond that, you can define multiple environments using named configurations. Example:

version = 0.1
[staging.deploy.parameters]
stack_name = "my-sam-app-staging"
region = "ap-southeast-1"
s3_bucket = "my-sam-artifacts-staging"
capabilities = "CAPABILITY_IAM"
parameter_overrides = "Environment=staging"

[prod.deploy.parameters]
stack_name = "my-sam-app-prod"
region = "ap-southeast-1"
s3_bucket = "my-sam-artifacts-prod"
capabilities = "CAPABILITY_IAM"
parameter_overrides = "Environment=prod"

Now you can deploy using:

sam deploy --config-env staging
sam deploy --config-env prod

This allows:

• Cleaner separation between dev/staging/prod
• Safer deployment practices
• Per-env overrides for Lambda environment variables, tags, etc.

3. SAM CLI

A command-line tool that simplifies development and deployment:

• sam init – scaffold a new project
• sam build – package your code
• sam deploy – push it to AWS
• sam local invoke – test individual functions locally
• sam local start-api – emulate full API locally

If you’re starting your journey into serverless with AWS, SAM is one of the best tools to learn and use. It removes the friction of writing raw CloudFormation, supports local development, and lets you ship your ideas quickly.

It’s not just beginner-friendly — it’s also powerful enough to be used in production systems, especially when paired with other AWS services like DynamoDB, Step Functions, and EventBridge.

AWS Lambda is one of the most exciting services in the serverless world. It lets you write code that automatically responds to events — without needing to worry about provisioning servers or managing infrastructure.

In this post, I will cover the basics:

• What is Lambda?
• What are the core components?
• Why use it?
• A real use case: processing SQS messages in TypeScript
• Common limitations

What is AWS Lambda?

AWS Lambda is a serverless compute service that lets you run code in response to events without provisioning or managing servers. You can use Lambda to run code for virtually any type of application or backend service with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability.

You don’t manage servers. You just focus on the code, and Lambda takes care of:

• Running it
• Scaling it automatically
• Charging you only when it runs

Key Components of AWS Lambda

1. Handlers: The entry point for your Lambda function. It’s the method that AWS Lambda calls to start execution.
2. Events: Lambda functions are triggered by events, which can come from various AWS services like S3, DynamoDB, API Gateway, or even custom events.
3. Context: Provides runtime information to your Lambda function, such as the function name, version, and remaining execution time.
4. IAM Roles: AWS Identity and Access Management (IAM) roles define the permissions for your Lambda function, allowing it to access other AWS services securely.
5. Environment Variables: Key-value pairs that you can use to pass configuration settings to your Lambda function at runtime.
6. Timeouts and Memory: You can configure the maximum execution time and memory allocated to your Lambda function, which affects performance and cost.
7. CloudWatch Logs: Automatically logs the output of your Lambda function, which you can use for debugging and monitoring.

Why Use AWS Lambda?

• Cost-Effective: You only pay for the compute time you consume. There are no charges when your code is not running.
• Scalability: Automatically scales your application by running code in response to each event, so you don’t have to worry about scaling your infrastructure.
• Flexibility: Supports multiple programming languages (Node.js, Python, Java, Go, C#, Ruby, and custom runtimes) and can be used for a wide range of applications, from simple scripts to complex microservices.
• Event-Driven: Easily integrates with other AWS services, allowing you to build event-driven architectures that respond to changes in your data or system state.
• Zero Administration: No need to manage servers or runtime environments. AWS handles all the infrastructure management tasks, including patching, scaling, and availability.

Real Use Case: TypeScript Lambda to Process SQS → DynamoDB

In my current role, we use AWS Lambda to process messages from SQS queues. Here’s a simple example of how you can set up a Lambda function in TypeScript to process messages from an SQS queue and store them in DynamoDB.

Lets say we receive messages in SQS that contain user data, and we want to store this data in DynamoDB.

import { SQSHandler, SQSEvent, Context } from "aws-lambda";
import { DynamoDB } from "aws-sdk";

const dynamoDb = new DynamoDB.DocumentClient();

export const handler: SQSHandler = async (
  event: SQSEvent,
  context: Context
) => {
  for (const record of event.Records) {
    const userData = JSON.parse(record.body);
    const params = {
      TableName: "Users",
      Item: userData,
    };
    await dynamoDb.put(params).promise();
  }
};

In this example:

• We import necessary types from aws-lambda and the DynamoDB client from aws-sdk.
• The handler function processes each message in the SQS event.
• We parse the message body and store it in a DynamoDB table named Users.

This function will be uploaded to AWS Lambda, and you can configure it to trigger whenever new messages arrive in the SQS queue.

Common Limitations of AWS Lambda

• Execution Time: Lambda functions have a maximum execution time of 15 minutes. If your task takes longer, you may need to break it into smaller functions or use other services.
• Cold Starts: When a Lambda function is invoked after being idle, it may take longer to start due to the initialization time (cold start). This can affect performance, especially for latency-sensitive applications.
• Limited Resources: Each Lambda function has a maximum memory limit (up to 1024 MB) and a maximum package size (50 MB for direct upload, 250 MB when using layers). This can be a constraint for resource-intensive applications.
• Limited Runtime Environment: While Lambda supports multiple programming languages, you may encounter limitations with certain libraries or dependencies that require a specific runtime environment.
• State Management: Lambda functions are stateless, meaning they do not retain any state between invocations. If you need to maintain state, you will have to use external storage solutions like DynamoDB or S3.
• Concurrency Limits: There are limits on the number of concurrent executions for Lambda functions. If your application experiences a sudden spike in traffic, you may hit these limits, leading to throttling of requests.
• Vendor Lock-In: Using AWS Lambda ties you to the AWS ecosystem, which can make it challenging to migrate to other cloud providers or on-premises in the future.

Wrap-Up

AWS Lambda is a powerful tool for building serverless applications that can scale automatically and respond to events without the need for server management. By understanding its core components and limitations, you can effectively leverage Lambda to build efficient, cost-effective applications that meet your business needs.

Whether you’re processing SQS messages, building APIs with API Gateway, or integrating with other AWS services, Lambda provides a flexible and scalable solution that can adapt to your application’s requirements.

Datastore is always a crucial part of any application, and choosing the right database can significantly impact your application’s performance, scalability, and maintainability. In this post, we’ll explore AWS DynamoDB.

Database Types

There are two main types of databases:

• Relational Databases (RDBMS): These databases use structured query language (SQL) and are designed to handle structured data with predefined schemas. Examples include MySQL, PostgreSQL, and Oracle.
• NoSQL Databases: These databases are designed to handle unstructured or semi-structured data. They provide flexibility in data modeling and can scale horizontally. Examples include MongoDB, Cassandra, and DynamoDB.

If you’re coming from MySQL or PostgreSQL, imagine removing JOINs and replacing rows with JSON-like documents stored under a single key.

Key Features of DynamoDB

• Fully Managed: DynamoDB is a fully managed service, meaning AWS handles the operational aspects such as hardware provisioning, setup, configuration, and scaling.
• Performance at Scale: It automatically scales up and down to adjust for capacity and maintain performance.
• Flexible Data Model: Supports key-value and document data structures, allowing for a variety of use cases.
• Built-in Security: Offers encryption at rest and in transit, along with fine-grained access control.

So, what is DynamoDB? DynamoDB is a fully managed NoSQL database service provided by AWS that offers fast and predictable performance with seamless scalability. It is designed to handle large amounts of data and high request rates, making it ideal for applications that require low-latency data access.

Key Concepts

• Tables: The primary structure in DynamoDB, similar to tables in relational databases. Each table has a primary key that uniquely identifies each item.
• Items: Individual records in a table, similar to rows in a relational database.
• Attributes: The data fields in an item, similar to columns in a relational database.
• Primary Key: Uniquely identifies each item in a table. It can be a simple primary key (partition key) or a composite primary key (partition key and sort key).
• Indexes: Allow for efficient querying of data. DynamoDB supports both global secondary indexes (GSI) and local secondary indexes (LSI).

A simple item in a DynamoDB table might look like this:

{
  "UserId": "12345", # Unique identifier for the user. Primary key.
  "Name": "Hazriq",
  "Email": "hazriq@example.com"
}

Benefits of Using DynamoDB

• Scalability: Automatically scales to handle large amounts of data and high request rates without manual intervention.
• Performance: Provides low-latency data access, making it suitable for real-time applications.
• Flexibility: Supports various data models, allowing developers to choose the best fit for their application.
• Cost-Effective: Pay-as-you-go pricing model, where you only pay for the resources you use, making it cost-effective for applications with variable workloads.
• Integration with AWS Services: Seamlessly integrates with other AWS services like Lambda, API Gateway, and CloudWatch for monitoring and logging.
• TTL (Time to Live): Automatically deletes expired items, helping manage storage costs and data lifecycle.

Performance Considerations

• Provisioned Throughput: You can specify the read and write capacity units for your table, which determines how many reads and writes per second your table can handle.
• On-Demand Capacity: Automatically scales to accommodate workload changes, making it suitable for unpredictable workloads.
• Caching: Use DynamoDB Accelerator (DAX) for in-memory caching to improve read performance for read-heavy workloads.
• Batch Operations: Use batch operations for efficient processing of multiple items in a single request, reducing the number of round trips to the database.

Without DAX

• Reads and writes are directly from the DynamoDB table.
• Each read or write operation incurs a latency based on the network and DynamoDB’s processing time.

With DAX

• DAX acts as an in-memory cache, reducing the latency for read operations.
• DAX handles cache misses by fetching data from DynamoDB and storing it in memory for subsequent requests.
• This significantly speeds up read operations, especially for frequently accessed data.

When not to Use DynamoDB

While DynamoDB is a powerful tool, it may not be the best fit for every use case. Here are some scenarios where you might consider alternatives:

• Complex Queries: If your application requires complex queries with multiple joins or aggregations, a relational database might be more suitable.
• Transactional Support: If your application requires complex transactions involving multiple items or tables, consider using a relational database or a database that supports multi-item transactions.
• Large Binary Objects: If your application needs to store large binary objects (BLOBs), such as images or videos, consider using Amazon S3 for storage and DynamoDB for metadata.
• High Write Throughput: If your application requires extremely high write throughput, consider using Amazon S3 or a distributed database like Apache Cassandra.

DynamoDB shines when you need a fast, scalable, and fully managed database that just works — whether you’re powering a real-time leaderboard, handling millions of API requests, or storing user sessions with minimal latency. By understanding its core concepts and performance features like DAX, you can unlock a powerful tool that fits right into modern, serverless-first architectures.

Of course, like any tool, it’s not a one-size-fits-all solution. Knowing when and how to use DynamoDB effectively is key — and that journey starts with grasping its strengths.

At my new company, we rely heavily on AWS SQS (Simple Queue Service) and SNS (Simple Notification Service) to handle the large volume of records we need to ingest daily.

Think of a situation where the system experiences a temporary bottleneck or needs to go offline for maintenance or upgrades. In such cases, SQS acts as a buffer—safely storing incoming records in a queue so they’re not lost. Even during traffic spikes, the messages are queued and processed at our own pace. For example, our AWS Lambda function polls the queue and retrieves messages when it’s ready, allowing the system to remain responsive and scalable even under pressure.

Core Components of SQS and SNS

To better understand how SQS and SNS work, it helps to break down their main components.

📨 SQS Main Components:

• Queue: The main container where messages are stored temporarily until processed.
• Producer: The system or service that sends messages to the queue.
• Consumer: The service (e.g. Lambda, EC2) that polls the queue and processes the message.
• Visibility Timeout: A short period where the message becomes invisible to other consumers once picked up—helps avoid duplicate processing.
• DLQ (Dead-Letter Queue): A separate queue that stores messages that couldn’t be successfully processed after several retry attempts. Useful for debugging failed messages.

📣 SNS Main Components:

• Topic: The central component that receives messages from publishers.
• Publisher: The producer that sends a message to a topic.
• Subscriber: Services or endpoints (like Lambda, SQS queue, HTTPS endpoint, email) that receive messages pushed from the topic.
• Subscription Filter Policy: You can apply rules to decide which subscribers should receive which messages (useful for message routing).

SQS vs SNS – What’s the Difference?

While both SQS and SNS are messaging services provided by AWS, they serve very different purposes:

• Use SQS when you want to decouple your producer and consumer, especially if the consumer might be temporarily unavailable.
• Use SNS when you want to broadcast messages to multiple services (e.g., notify a Lambda, an email service, and an HTTP endpoint at the same time).