Skip to main content

​
Prerequisites

Before installing Ghost Scout, ensure you have the following:
  • Node.js: Version 14 or higher
  • npm: Comes with Node.js
  • Docker: For running Redis and MarkItDown-API containers
  • Git: For cloning the repository
  • Hunter.io API Key: Required for contact discovery
    • Sign up at hunter.io
    • Obtain API key from account settings
  • Anthropic API Key: Required for AI profile and pretext generation
  • Operating System: Linux, macOS, or Windows
  • RAM: Minimum 2GB, 4GB recommended
  • Disk Space: At least 500MB for application and dependencies
  • Network: Internet connection for API access

​
Installation Steps

1

Clone the Repository

Clone the Ghost Scout repository to your local machine:
git clone <repository-url>
cd ghost_scout
Replace <repository-url> with the actual Git repository URL for Ghost Scout
2

Install Dependencies

Install all required Node.js packages:
npm install
This will install all dependencies specified in package.json, including:
  • Fastify (web framework)
  • Socket.io (real-time updates)
  • Bee-Queue (job processing)
  • SQLite3 (database)
  • Alpine.js (frontend framework)
  • And other required packages
3

Configure Environment Variables

Create a .env file in the project root with your API credentials:
# Create .env file
touch .env
Add the following content to .env:
# Hunter.io API Key
HUNTER_API_KEY=your_hunter.io_api_key

# Anthropic API Key
ANTHROPIC_API_KEY=your_anthropic_api_key
Never commit your .env file to version control. Ensure it’s listed in .gitignore
Obtaining API Keys:
  • Hunter.io
  • Anthropic
  1. Visit hunter.io
  2. Create an account or sign in
  3. Navigate to API settings
  4. Copy your API key
  5. Paste into .env file
4

Start Redis Container

Ghost Scout requires Redis for job queue processing. Start Redis using Docker:
  • Basic Redis
  • Redis with Persistence
For development/testing (no persistence):
docker run --name redis -p 6379:6379 -d redis
This starts a Redis container without data persistence.
Verify Redis is running:
docker ps | grep redis
Managing Redis Container:
# Stop Redis
docker stop redis

# Start Redis (after stopping)
docker start redis

# Restart Redis
docker restart redis

# View Redis logs
docker logs redis

# Remove Redis container
docker rm redis
5

Start MarkItDown-API Container

Ghost Scout uses MarkItDown-API for HTML to Markdown conversion. Start the container:
docker run -d --name markitdown-api -p 8490:8490 ghcr.io/fkasler/markitdown-api:sha-ee4fcafe2cf2f17fbbff77cc7f1b1c81a7c370d2
Verify MarkItDown-API is running:
docker ps | grep markitdown-api
Test the API:
curl http://localhost:8490/health
The MarkItDown-API container will be accessible on port 8490
Managing MarkItDown-API Container:
# Stop MarkItDown-API
docker stop markitdown-api

# Start MarkItDown-API (after stopping)
docker start markitdown-api

# View logs
docker logs markitdown-api

# Remove container
docker rm markitdown-api
6

Start Ghost Scout

Start the Ghost Scout application:
node index.js
You should see output indicating the server has started:
Server listening on port 3000
For production deployments, consider using a process manager like PM2
7

Access the Application

Open your web browser and navigate to:
http://localhost:3000
You should see the Ghost Scout web interface.

​
Verification

Verify your installation is working correctly:
  • Check Services
  • Test Database
  • Check Logs
Ensure all services are running:
# Check Ghost Scout
curl http://localhost:3000

# Check Redis
docker ps | grep redis

# Check MarkItDown-API
docker ps | grep markitdown-api
curl http://localhost:8490/health

​
Post-Installation Configuration

​
Database Setup

Ghost Scout automatically initializes the SQLite database on first run. The database includes:
  • Domain - Target company domains
  • SourceDomain - Domains where source data is found
  • Target - Target individuals (prospects)
  • SourceData - URLs where target data was found
  • TargetSourceMap - Many-to-many relationship between targets and sources
  • Prompt - LLM prompts for pretext generation
  • Pretext - Generated phishing messages
No manual database initialization is required

​
Prompt Library

Ghost Scout includes a prompt library for AI generation: 
ls prompt_library/
These YAML files contain templates for:
  • Profile generation prompts
  • Pretext generation prompts
  • Different phishing scenarios
You can customize these templates to fit your specific use cases.

​
Production Deployment

For production deployments, consider the following enhancements:
Use PM2 to manage the Node.js process:
# Install PM2 globally
npm install -g pm2

# Start Ghost Scout with PM2
pm2 start index.js --name ghost-scout

# Configure PM2 to start on boot
pm2 startup
pm2 save
PM2 Commands:
# Status
pm2 status

# Logs
pm2 logs ghost-scout

# Restart
pm2 restart ghost-scout

# Stop
pm2 stop ghost-scout
Use Nginx as a reverse proxy:
server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket support for Socket.io
    location /socket.io/ {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
Use Let’s Encrypt for HTTPS:
# Install Certbot
sudo apt-get install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d your-domain.com
Consider creating a Docker Compose setup (currently on project TODO list):
version: '3.8'
services:
  ghost-scout:
    build: .
    ports:
      - "3000:3000"
    environment:
      - HUNTER_API_KEY=${HUNTER_API_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    depends_on:
      - redis
      - markitdown-api

  redis:
    image: redis:latest
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data

  markitdown-api:
    image: ghcr.io/fkasler/markitdown-api:sha-ee4fcafe2cf2f17fbbff77cc7f1b1c81a7c370d2
    ports:
      - "8490:8490"

volumes:
  redis-data:

​
Troubleshooting

Error: Port 3000 is already in useSolutions:
  • Stop the process using port 3000: lsof -ti:3000 | xargs kill -9
  • Change the port in Ghost Scout configuration
  • Identify and stop conflicting service
Error: Cannot connect to RedisSolutions:
  • Verify Redis container is running: docker ps | grep redis
  • Check Redis logs: docker logs redis
  • Ensure port 6379 is not blocked by firewall
  • Restart Redis container: docker restart redis
Error: Invalid API key or Authentication failedSolutions:
  • Verify API keys in .env file
  • Check for extra spaces or newlines in .env
  • Ensure .env is in the project root directory
  • Restart Ghost Scout after updating .env
  • Verify API keys are active in respective portals
Error: Cannot connect to MarkItDown-APISolutions:
  • Verify container is running: docker ps | grep markitdown-api
  • Check logs: docker logs markitdown-api
  • Test endpoint: curl http://localhost:8490/health
  • Restart container: docker restart markitdown-api
  • Ensure port 8490 is available
Error: Cannot find module 'package-name'Solutions:
  • Run npm install again
  • Delete node_modules/ and package-lock.json, then run npm install
  • Check Node.js version compatibility
  • Verify package.json is intact
Error: SQLite database errorsSolutions:
  • Ensure db/ directory exists and is writable
  • Check disk space availability
  • Verify SQLite3 module is installed
  • Delete database and let Ghost Scout recreate it

​
Updating Ghost Scout

To update Ghost Scout to the latest version: 
# Pull latest changes
git pull origin main

# Update dependencies
npm install

# Restart the application
node index.js
If using PM2: 
git pull origin main
npm install
pm2 restart ghost-scout
Always backup your database before updating

​
Uninstallation

To completely remove Ghost Scout:
1

Stop the Application

# If running directly
# Press Ctrl+C in the terminal

# If using PM2
pm2 stop ghost-scout
pm2 delete ghost-scout
2

Stop and Remove Containers

# Stop containers
docker stop redis markitdown-api

# Remove containers
docker rm redis markitdown-api

# Remove volumes (optional - deletes data)
docker volume rm redis-data
3

Remove Application Files

cd ..
rm -rf ghost_scout

​
Security Considerations

Protect your Ghost Scout installation and API keys
  • Never commit .env to version control
  • Use environment-specific API keys
  • Rotate API keys regularly
  • Limit API key permissions where possible
  • Monitor API usage for anomalies
  • Use firewall rules to restrict access
  • Deploy behind VPN for production use
  • Use HTTPS in production (SSL/TLS)
  • Restrict database access
  • Secure Redis with password (Redis AUTH)
  • Implement authentication for web interface
  • Use role-based access control
  • Audit user actions and data access
  • Restrict to authorized users only
  • Use separate instances per client/campaign

​
Next Steps

Overview

Learn about Ghost Scout features

Usage Guide

Practical usage and workflows

Hunter.io Docs

Hunter.io API documentation