Installation Guide - SpecterOps Open Source

Prerequisites

Before installing Ghost Scout, ensure you have the following:

Required Software

Node.js: Version 14 or higher
npm: Comes with Node.js
Docker: For running Redis and MarkItDown-API containers
Git: For cloning the repository

API Keys

Hunter.io API Key: Required for contact discovery
- Sign up at hunter.io
- Obtain API key from account settings
Anthropic API Key: Required for AI profile and pretext generation
- Sign up at anthropic.com
- Obtain API key from console

System Requirements

Operating System: Linux, macOS, or Windows
RAM: Minimum 2GB, 4GB recommended
Disk Space: At least 500MB for application and dependencies
Network: Internet connection for API access

Installation Steps

Clone the Repository

Clone the Ghost Scout repository to your local machine:

git clone <repository-url>
cd ghost_scout

Replace <repository-url> with the actual Git repository URL for Ghost Scout

Install Dependencies

Install all required Node.js packages:

npm install

This will install all dependencies specified in package.json, including:

Fastify (web framework)
Socket.io (real-time updates)
Bee-Queue (job processing)
SQLite3 (database)
Alpine.js (frontend framework)
And other required packages

Configure Environment Variables

Create a .env file in the project root with your API credentials:

# Create .env file
touch .env

Add the following content to .env:

# Hunter.io API Key
HUNTER_API_KEY=your_hunter.io_api_key

# Anthropic API Key
ANTHROPIC_API_KEY=your_anthropic_api_key

Never commit your .env file to version control. Ensure it’s listed in .gitignore

Obtaining API Keys:

Hunter.io
Anthropic

Visit hunter.io
Create an account or sign in
Navigate to API settings
Copy your API key
Paste into .env file

Visit anthropic.com
Create an account or sign in
Navigate to API console
Generate an API key
Copy and paste into .env file

Start Redis Container

Ghost Scout requires Redis for job queue processing. Start Redis using Docker:

Basic Redis
Redis with Persistence

For development/testing (no persistence):

docker run --name redis -p 6379:6379 -d redis

This starts a Redis container without data persistence.

For production (with data persistence):

docker run --name redis -p 6379:6379 -v redis-data:/data -d redis redis-server --appendonly yes

This enables append-only file (AOF) persistence to prevent data loss.

Verify Redis is running:

docker ps | grep redis

Managing Redis Container:

# Stop Redis
docker stop redis

# Start Redis (after stopping)
docker start redis

# Restart Redis
docker restart redis

# View Redis logs
docker logs redis

# Remove Redis container
docker rm redis

Start MarkItDown-API Container

Ghost Scout uses MarkItDown-API for HTML to Markdown conversion. Start the container:

docker run -d --name markitdown-api -p 8490:8490 ghcr.io/fkasler/markitdown-api:sha-ee4fcafe2cf2f17fbbff77cc7f1b1c81a7c370d2

Verify MarkItDown-API is running:

docker ps | grep markitdown-api

Test the API:

curl http://localhost:8490/health

The MarkItDown-API container will be accessible on port 8490

Managing MarkItDown-API Container:

# Stop MarkItDown-API
docker stop markitdown-api

# Start MarkItDown-API (after stopping)
docker start markitdown-api

# View logs
docker logs markitdown-api

# Remove container
docker rm markitdown-api

Start Ghost Scout

Start the Ghost Scout application:

node index.js

You should see output indicating the server has started:

Server listening on port 3000

For production deployments, consider using a process manager like PM2

Access the Application

Open your web browser and navigate to:

http://localhost:3000

You should see the Ghost Scout web interface.

Verification

Verify your installation is working correctly:

Check Services
Test Database
Check Logs

Ensure all services are running:

# Check Ghost Scout
curl http://localhost:3000

# Check Redis
docker ps | grep redis

# Check MarkItDown-API
docker ps | grep markitdown-api
curl http://localhost:8490/health

Verify SQLite database was created:

ls -lh db/

You should see the database file.

Monitor application logs for errors:

# Ghost Scout logs (terminal where you ran node index.js)
# Watch for connection errors or API issues

# Redis logs
docker logs redis

# MarkItDown-API logs
docker logs markitdown-api

Post-Installation Configuration

Database Setup

Ghost Scout automatically initializes the SQLite database on first run. The database includes:

Domain - Target company domains
SourceDomain - Domains where source data is found
Target - Target individuals (prospects)
SourceData - URLs where target data was found
TargetSourceMap - Many-to-many relationship between targets and sources
Prompt - LLM prompts for pretext generation
Pretext - Generated phishing messages

No manual database initialization is required

Prompt Library

Ghost Scout includes a prompt library for AI generation:

ls prompt_library/

These YAML files contain templates for:

Profile generation prompts
Pretext generation prompts
Different phishing scenarios

You can customize these templates to fit your specific use cases.

Production Deployment

For production deployments, consider the following enhancements:

Process Manager

Use PM2 to manage the Node.js process:

# Install PM2 globally
npm install -g pm2

# Start Ghost Scout with PM2
pm2 start index.js --name ghost-scout

# Configure PM2 to start on boot
pm2 startup
pm2 save

PM2 Commands:

# Status
pm2 status

# Logs
pm2 logs ghost-scout

# Restart
pm2 restart ghost-scout

# Stop
pm2 stop ghost-scout

Reverse Proxy

Use Nginx as a reverse proxy:

server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket support for Socket.io
    location /socket.io/ {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

SSL/TLS

Use Let’s Encrypt for HTTPS:

# Install Certbot
sudo apt-get install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d your-domain.com

Docker Compose

Consider creating a Docker Compose setup (currently on project TODO list):

version: '3.8'
services:
  ghost-scout:
    build: .
    ports:
      - "3000:3000"
    environment:
      - HUNTER_API_KEY=${HUNTER_API_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    depends_on:
      - redis
      - markitdown-api

  redis:
    image: redis:latest
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data

  markitdown-api:
    image: ghcr.io/fkasler/markitdown-api:sha-ee4fcafe2cf2f17fbbff77cc7f1b1c81a7c370d2
    ports:
      - "8490:8490"

volumes:
  redis-data:

Troubleshooting

Port Already in Use

Error: Port 3000 is already in useSolutions:

Stop the process using port 3000: lsof -ti:3000 | xargs kill -9
Change the port in Ghost Scout configuration
Identify and stop conflicting service

Redis Connection Failed

Error: Cannot connect to RedisSolutions:

Verify Redis container is running: docker ps | grep redis
Check Redis logs: docker logs redis
Ensure port 6379 is not blocked by firewall
Restart Redis container: docker restart redis

API Key Invalid

Error: Invalid API key or Authentication failedSolutions:

Verify API keys in .env file
Check for extra spaces or newlines in .env
Ensure .env is in the project root directory
Restart Ghost Scout after updating .env
Verify API keys are active in respective portals

MarkItDown-API Not Responding

Error: Cannot connect to MarkItDown-APISolutions:

Verify container is running: docker ps | grep markitdown-api
Check logs: docker logs markitdown-api
Test endpoint: curl http://localhost:8490/health
Restart container: docker restart markitdown-api
Ensure port 8490 is available

Module Not Found

Error: Cannot find module 'package-name'Solutions:

Run npm install again
Delete node_modules/ and package-lock.json, then run npm install
Check Node.js version compatibility
Verify package.json is intact

Database Errors

Error: SQLite database errorsSolutions:

Ensure db/ directory exists and is writable
Check disk space availability
Verify SQLite3 module is installed
Delete database and let Ghost Scout recreate it

Updating Ghost Scout

To update Ghost Scout to the latest version:

# Pull latest changes
git pull origin main

# Update dependencies
npm install

# Restart the application
node index.js

If using PM2:

git pull origin main
npm install
pm2 restart ghost-scout

Always backup your database before updating

Uninstallation

To completely remove Ghost Scout:

Stop the Application

# If running directly
# Press Ctrl+C in the terminal

# If using PM2
pm2 stop ghost-scout
pm2 delete ghost-scout

Stop and Remove Containers

# Stop containers
docker stop redis markitdown-api

# Remove containers
docker rm redis markitdown-api

# Remove volumes (optional - deletes data)
docker volume rm redis-data

Remove Application Files

cd ..
rm -rf ghost_scout

Security Considerations

Protect your Ghost Scout installation and API keys

API Key Security

Never commit .env to version control
Use environment-specific API keys
Rotate API keys regularly
Limit API key permissions where possible
Monitor API usage for anomalies

Network Security

Use firewall rules to restrict access
Deploy behind VPN for production use
Use HTTPS in production (SSL/TLS)
Restrict database access
Secure Redis with password (Redis AUTH)

Access Control

Implement authentication for web interface
Use role-based access control
Audit user actions and data access
Restrict to authorized users only
Use separate instances per client/campaign

Next Steps

Overview

Learn about Ghost Scout features

Usage Guide

Practical usage and workflows

Hunter.io Docs

Hunter.io API documentation

​Prerequisites

​Installation Steps

​Verification

​Post-Installation Configuration

​Database Setup

​Prompt Library

​Production Deployment

​Troubleshooting

​Updating Ghost Scout

​Uninstallation

​Security Considerations

​Next Steps

Overview

Usage Guide

Hunter.io Docs

Prerequisites

Installation Steps

Verification

Post-Installation Configuration

Database Setup

Prompt Library

Production Deployment

Troubleshooting

Updating Ghost Scout

Uninstallation

Security Considerations

Next Steps