Miranda

Video Analysis Application

Dec 2024 - Present

Advanced video analysis application for material deformation tracking using computer vision and AI. Features real-time tracking, elasticity analysis, polygon selection, and comprehensive statistical visualization. Built as both research prototype and production-ready executable.

Stack

The Problem

Video analysis with modern computer vision models (such as dense tracking or advanced segmentation) requires large amounts of memory, GPU, and processing time. These limitations restrict this type of analysis to local machines with expensive hardware, hindering scalability and significantly slowing down workflows.

The Solution

I designed a distributed platform that allows executing high-computational-cost computer vision models in on-demand cloud infrastructure. This eliminates dependency on local hardware and enables flexible scaling of processing, optimizing costs and making advanced video analysis accessible.

Architecture

Frontend

The frontend is built with Next.js, React, TypeScript and Tailwind CSS, providing an interactive interface where users can upload videos, select tracking points, and monitor processing progress. It supports two main modes: Job Mode for individual analysis and Session Mode for batch processing of multiple videos, with real-time status updates via polling.

Backend

The backend, developed with FastAPI and Python, acts as central system API. It manages authentication via JWT, controls credit system for resource usage, and coordinates processing logic. It integrates with Supabase through SQLAlchemy for persistence of users, jobs, and sessions, and uses boto3 to communicate with AWS services.

Cloud Infrastructure (AWS)

The cloud infrastructure is where intensive processing executes. It uses Amazon S3 for video and results storage, AWS Lambda as the trigger system, and Amazon EC2 with GPU instances to run artificial intelligence models. The instances are launched via Lambda using AMIs (Amazon Machine Images) preconfigured with the miranda-aws code and Computer Vision AI models, ensuring the EC2s boot up with all necessary configuration ready to use. Each job launches one of these dedicated instances that downloads data from S3, executes processing, and uploads results before automatically shutting down. The entire infrastructure is secured under a least-privilege model using AWS IAM, employing specific roles for each service such as 'miranda-video-trigger-role' and 'miranda-cleanup-role' for Lambda functions, and 'miranda-ec2-role' along with 'aws-ec2-spot-fleet-tagging-role' for secure EC2 instance management and execution.

Data Flow

The process starts when a user uploads a video from the frontend. The backend stores it in S3 and triggers an event that activates a Lambda function. Lambda launches an EC2 instance that executes the processing, and once completed, the results are stored back in S3. The frontend detects completion and allows the user to download the results.

Deployment & CI/CD

Deployment is automated using Docker containers and a CI/CD pipeline in GitLab. Both the frontend and backend have their own Dockerfiles, orchestrated on the server via Docker Compose. The update process is managed by a .sh script that pulls the latest images from the GitLab registry, ensuring efficient continuous integration defined in the .gitlab-ci.yml file.