Nexoryx_Spark_Terraform

Production-ready Terraform automation for deploying an Apache Spark cluster on AWS with:

• 1 Spark Master
• 20 Spark Workers
• Ubuntu 24.04
• Docker-based Spark deployment
• Security Groups
• Auto provisioning
• Scalable infrastructure

---

Architecture

AWS VPC | +-- Spark Master EC2 | +-- 20 Spark Worker EC2 Instances

---

Features

• Terraform-based infrastructure automation
• Spark Master auto configuration
• Spark Worker auto join
• Security groups
• SSH access
• Ubuntu 24.04
• Docker installation
• Apache Spark cluster deployment

---

Requirements

• AWS Account
• Terraform >= 1.5
• Existing AWS key pair
• AWS CLI configured

---

Usage

Initialize

terraform init

Plan

terraform plan

Deploy

terraform apply -auto-approve

Destroy

terraform destroy -auto-approve

---

Outputs

• Spark Master Public IP
• Spark Master URL
• Worker Instance IDs

---

Spark Web UI

http://SPARK_MASTER_IP:8080

---

License

MIT License

Project Roadmap

• Kubernetes Helm charts
• GitOps support
• CI/CD improvements
• Monitoring dashboards
• Multi-cloud support
• Security hardening

GitHub Actions

This repository includes:

• Shell validation
• Markdown linting
• Terraform validation (where applicable)

Example Deployments

See:

• examples/
• docs/

Related Nexoryx Projects

This repository is part of the Nexoryx infrastructure ecosystem.