How to Build a Real-Time IoT Device Monitoring System on AWS (Step-by-Step Guide)

Introduction

IoT device monitoring is crucial for maintaining a healthy and efficient connected device ecosystem. With thousands of IoT devices deployed across different locations, understanding their performance becomes essential for business success. These devices operate across diverse environments and generate enormous amounts of data every day. Without an effective monitoring framework, even minor device malfunctions can escalate into large-scale operational disruptions.

The Importance of CPU Utilisation Monitoring

Among various IoT metrics like device connectivity, network latency, and system availability, CPU utilisation stands out as a fundamental performance indicator. High CPU usage can indicate:

• Resource-intensive processes running on the device
• Potential system overload leading to device crashes
• Need for system optimisation or hardware upgrades
• Early warning signs before complete device failure

Motivation

If you’ve ever wanted to monitor your IoT devices in real-time and receive alerts when something goes wrong then article is for you!

This project demonstrates a system where an IoT device (like a Raspberry Pi) publishes its CPU utilisation as a key performance parameter. By implementing a comprehensive monitoring solution using AWS services, we can:

• Detect performance issues before they impact operations
• Receive instant alerts when CPU usage exceeds safe thresholds
• Ensure proactive maintenance rather than reactive troubleshooting

The system provides a scalable foundation that can be extended to monitor additional metrics, making it valuable for managing IoT deployments of any size while keeping operational costs manageable through AWS’s pay-as-you-use model.

Prerequisites

• AWS account (free tier is fine)
• Basic Python knowledge
• A Raspberry Pi or a Laptop with Python installed
• Familiarity with AWS Console navigation

Understanding the Architecture

Before diving into the implementation, let’s explore how our IoT monitoring system works. This solution uses a cloud-native, event-driven architecture built entirely on AWS services, ensuring scalability, reliability, and cost-effectiveness. Here’s how the components work together:

1. IoT Device (Publisher): Your Raspberry Pi, laptop, or any device running our Python script.
2. AWS IoT Core: Managed MQTT broker that receives device messages securely.
3. IoT Rules: Serverless routing that triggers actions based on incoming data.
4. Lambda Function: Processes data, stores it, and decides when to send alerts.
5. DynamoDB: NoSQL database for storing all CPU metrics.
6. SNS: Simple Notification Service for sending email alerts.

Step-by-Step Setup Guide

To begin, sign in to the AWS Management Console using either the root user account or an IAM user with sufficient permissions to access AWS IoT Core. From the console, navigate to the IoT Core service and select the AWS Region in which you want to provision your IoT resources. In this project the US East (N. Virginia) [us-east-1] Region is used as an example. You may choose any available Region; however, ensure that the same Region is used consistently throughout the project to maintain resource alignment and prevent configuration errors.

STEP 1 : Create IoT Thing

AWS IoT Core manages devices as “Things” — digital twins of your physical devices.

• Navigate to AWS IoT Core → Manage → Things
• Click “Create things” → “Create a single thing”
• Thing name: iot_device
• Thing type: Leave blank (optional)
• Click “Next”

Generating Security Certificates

IoT devices need certificates for secure communication — think of them as digital passports.

• Choose “Auto-generate a new certificate”
• Click “Next”

⚠️ Important: Download ALL certificate files immediately — you can’t retrieve them later!

Files you’ll download:

• iot_device.cert.pem (Device certificate)
• iot_device.private.key (Private key)
• root-CA.crt (Amazon Root CA)

Creating Device Permissions

1. Click “Create policy”
2. Policy name: iot_device-policy
3. Policy document:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot:Connect",
        "iot:Publish",
        "iot:Subscribe",
        "iot:Receive"
      ],
      "Resource": "*"
    }
  ]
}

🔒 Security Note: In production, replace “*” with specific resource ARNs for better security

STEP 2: Getting Your Device Endpoint

Every AWS account has a unique IoT endpoint URL where your devices connect. Go to AWS IoT Core → Connect → Domain configurations

1. Copy your “Domain name”
2. Save it — you’ll need this in your Python code!

Example endpoint: a1b2c3d4e5f6g7-ats.iot.us-east-1.amazonaws.com

⚠️ Important: Always verify the AWS Region before copying your IoT device endpoint. Endpoints are Region-specific, and using the wrong Region may cause connection failures.

STEP 3: Python Code for Your IoT Device

Installing Dependencies

pip install psutil awsiotsdk

Here’s the code for your IoT device to publish CPU usage every 5 seconds:

import os
import time
import json
import psutil
from awscrt import mqtt, io
from awsiot import mqtt_connection_builder

# AWS IoT Core connection settings
ENDPOINT = "YOUR-DEVICE-ENDPOINT-HERE"
PATH_TO_CERT = "iot_device.cert.pem"
PATH_TO_KEY = "iot_device.private.key"
PATH_TO_ROOT = "root-CA.crt"
CLIENT_ID = "rpi_iot_device"
TOPIC= "iot/device/metrics"

# Create MQTT connection
mqtt_connection = mqtt_connection_builder.mtls_from_path(
    endpoint=ENDPOINT,
    cert_filepath=PATH_TO_CERT,
    pri_key_filepath=PATH_TO_KEY,
    ca_filepath=PATH_TO_ROOT,
    client_id=CLIENT_ID,
    clean_session=False,
    keep_alive_secs=30,
)
print(f"Connecting to {ENDPOINT} with client ID '{CLIENT_ID}'...")
mqtt_connection.connect().result()
print("Connected!")

# Publish CPU usage every 5 seconds
try:
    while True:
        message = {
            "time": int(time.time()),
            "quality": "GOOD",
            "hostname": os.uname().nodename,
            "value": psutil.cpu_percent()
        }
        mqtt_connection.publish(
            topic=TOPIC,
            payload=json.dumps(message),
            qos=mqtt.QoS.AT_LEAST_ONCE
        )
        print(json.dumps(message, indent=2))
        time.sleep(5)

except KeyboardInterrupt:
    print("Disconnecting...")
    mqtt_connection.disconnect().result()
    print("Disconnected!")

💡 Code Explanation: This script uses psutil to read CPU usage and awsiotsdk to securely publish data to AWS IoT Core every 5 seconds.

STEP 4: Testing MQTT Communication

AWS IoT → MQTT Test Client → Subscribe to iot/devices/metrics. run the python code and you can observe the incoming messages as follows

STEP 5: Creating DynamoDB Storage

DynamoDB will store all your device metrics for historical analysis and reporting.

• Go to DynamoDB → Tables → Create table

Settings:

• Table name: IoTDeviceMetrics
• Partition key: hostname (String)
• Sort key: time (Number)
• Table settings: Use default settings (On-demand billing)
• Click “Create table”

Why This Schema?

• Partition key (hostname): Groups data by device
• Sort key (timestamp): Orders metrics chronologically
• Efficient queries: Easy to get latest metrics or time ranges per device

💰 Cost Tip: On-demand billing means you only pay for actual reads/writes — perfect for IoT projects!

STEP 6: Create Lambda Function

Lambda will process each MQTT message, store it in DynamoDB, and send alerts when needed. In architecture diagram also it is shown.

Function Setup

• Go to Lambda → Functions → Create function
• Choose “Author from scratch”
• Function name: IotToDynamoSNS
• Runtime: Python 3.12
• Architecture: x86_64
• Click “Create function”

Lambda function code :

import json
import boto3
from decimal import Decimal

dynamodb = boto3.resource('dynamodb')
sns = boto3.client('sns')

TABLE_NAME = 'IoTDeviceMetrics'
SNS_TOPIC_ARN = 'YOUR-SNS-TOPIC-ARN:iot-alerts'
CPU_THRESHOLD = 70  # in percent

def lambda_handler(event, context):
    print("Received event:", json.dumps(event))
    
  try:
        message = event

        # Convert values to Decimal if needed
        hostname = message['hostname']
        timestamp = Decimal(str(message['time']))
        quality = message['quality']
        cpu_value = Decimal(str(message['value']))

        # Create a new item with proper types for DynamoDB
        item = {
            'hostname': hostname,
            'time': timestamp,
            'quality': quality,
            'value': cpu_value
        }

        # Store in DynamoDB
        table = dynamodb.Table(TABLE_NAME)
        table.put_item(Item=item)

        # Publish alert if CPU is high
        if cpu_value > CPU_THRESHOLD:
            sns.publish(
                TopicArn=SNS_TOPIC_ARN,
                Subject="High CPU Usage Alert",
                Message=f"Device {hostname} CPU usage is {cpu_value}% at time {timestamp}"
            )
        return {
            'statusCode': 200,
            'body': json.dumps("Success")
        }

    except Exception as e:
        print("Error:", str(e))
        return {
            'statusCode': 500,
            'body': str(e)
        }

🗒️ Note: SNS-ARN need to add in this code will add after configuring SNS in later steps.

🧠 Lambda Insights: This function handles the core business logic — storing data and intelligent alerting based on configurable thresholds.

Setting Up Permissions

Your Lambda needs permissions to access DynamoDB and SNS:

1. Go to your function → Configuration → Permissions
2. Click on the execution role name

After clicking it will redirect to IAM role of Lambda function .

Go to Add Permission → Attach Policies 3. Add these policies:

• AmazonDynamoDBFullAccess
• AmazonSNSFullAccess

Step 7: Setting up SNS Email Alert

SNS (Simple Notification Service) will send you email alerts when your devices need attention.

Creating the SNS Topic

1. Go to SNS → Topics → Create topic

Configuration:

• Type: Standard
• Name: iot-alerts
• Display name: IoT Device Alerts
• Click “Create topic”

Adding Email Subscription

1. From your topic page, click “Create subscription”
2. Protocol: Email
3. Endpoint: Your email address
4. Click “Create subscription”

📧 Important: Check your email and click the confirmation link to activate alerts! Copy SNS Topic ARN and paste in Lambda Function.

STEPS 8: Connecting Everything with IoT Rules

IoT Rules automatically trigger your Lambda function when new device data arrives — no servers needed!

Creating the IoT Rule

1. Go to AWS IoT Core → Message routing → Rules
2. Click “Create rule”

Rule Configuration:

• Rule name: ForwardCPUMetrics
• Description: Route CPU metrics to Lambda for processing

SQL Statement:

SELECT * FROM 'iot/device/metrics'

💡 SQL Explanation: This selects all fields from messages published to the iot/device/metrics topic.

Adding Lambda Action

1. Click “Add action”
2. Select “Send a message to a Lambda function”
3. Choose your IotToDynamoSNS function
4. Click “Add action”

Our project setup is complete, and now it’s time to test the entire system. To do this, we will simulate a scenario of high CPU usage and check if the alerting mechanism works as expected.

End-to-End Testing

Go to AWS IoT Core → Test → MQTT test client

Click “Publish to a topic”

Topic: iot/devices/metrics

Message:

{
  "time": 1755340680,
  "quality": "GOOD",
  "hostname": "Nimeshs-MacBook",
  "value": 85.7
}

Click “Publish”

Verification Steps

Check Lambda Logs: CloudWatch → Log groups → /aws/lambda/IoTToDynamoSNS

Check DynamoDB: Verify data appears in your IoTDeviceMetrics table

Check Email: You should receive an alert email!

Wrapping Up

Congratulations! You have built an IoT monitoring system using AWS serverless technologies.

This project demonstrates a secure, scalable, and cost-effective solution that delivers real-time alerting, all without the need for server management. This robust architecture is a solid foundation for future advanced analytics and unlock even more value from your IoT data.

Enjoyed this post?

I’d love to hear your thoughts in the comments! Don’t forget to follow for more practical IoT deep dives, and let’s keep exploring the future of connected technologies together. 🚀🌍