OpenSec Atlas

trailscraper: An Open Source Tool for Identity & Access Management (IAM) | OpenSec Atlas

Identity & Access Management (IAM)

Tool

trailscraper

TrailScraper is a command-line tool that extracts and analyzes AWS CloudTrail data to generate IAM policies and assist with identity and access management.

View on GitHub

About This Tool

A command-line tool to get valuable information out of AWS CloudTrail

Primary Use Case

This tool is designed for AWS administrators and security professionals who need to audit, analyze, and generate IAM policies based on CloudTrail logs. It helps users filter CloudTrail events, download logs, and automate policy creation to improve cloud security posture and compliance.

Key Features

Query CloudTrail events using filters directly from the CloudTrail API
Download CloudTrail logs from specified S3 buckets and organizational trails
Search and filter downloaded CloudTrail logs for specific events
Generate IAM policies based on CloudTrail event records
Extend existing IAM policies by guessing additional relevant actions
Supports running via native installation, pip, Homebrew, or Docker
Handles global service logs by including us-east-1 region
Integration with organizational AWS accounts for centralized log analysis

Insights & Recommendations

Requires AWS credentials configured locally to access CloudTrail API or S3 buckets. Including the us-east-1 region is important to capture global service events. Logs from organizational trails require specifying the organization ID. The tool can help automate least privilege policy creation but may need manual review for completeness.

Installation

For OSX, run: brew install trailscraper
Ensure Python version 3.5 or higher is installed
Install via pip: pip install trailscraper
Run directly using Docker with AWS environment variables and mounting ~/.aws directory
Use GitHub Container Registry for current Docker images (version 0.7.0 and later)
Older Docker versions available on DockerHub

Usage

trailscraper select --use-cloudtrail-api --filter-assumed-role-arn some-arn --from 'one hour ago' --to 'now'

Fetches CloudTrail events matching a filter directly from the CloudTrail API

trailscraper download --bucket some-bucket --account-id some-account-id --region us-east-1 --from 'two days ago' --to 'now'

Downloads CloudTrail logs from a specified S3 bucket and region, including global service logs

trailscraper download --bucket some-bucket --account-id some-account-id --region us-east-1 --org-id o-someorgid --from 'two days ago' --to 'now'

Downloads CloudTrail logs from organizational trails for centralized AWS accounts

trailscraper select --filter-assumed-role-arn some-arn --from 'one hour ago' --to 'now'

Filters and finds CloudTrail events matching criteria in previously downloaded logs

gzcat some-records.json.gz | trailscraper generate

Generates an IAM policy based on provided CloudTrail event records

cat minimal-policy.json | trailscraper guess

Extends an existing IAM policy by guessing additional relevant actions not present in logs

Smart Usage Notes

Integrate TrailScraper into continuous compliance pipelines to automate IAM policy refinement and reduce privilege creep.Use generated IAM policies to enforce least privilege access dynamically based on actual CloudTrail usage patterns.Leverage TrailScraper’s filtering capabilities to detect anomalous or suspicious IAM activity indicative of compromised credentials.Combine TrailScraper with SIEM tools for enriched context and automated alerting on risky identity behaviors.Employ TrailScraper in purple team exercises to simulate attacker IAM misuse and validate detection and response controls.