Lyubitori

Download images from Your X(Twitter) likes using Selenium & Chrome WebDriver.

Requirements

• Python 3.13+

Installation

pip install -r requirements.txt
pip install -r dev-requirements.txt  # For development

Configuration

1. Copy the environment template:

cp .env.example .env

2. Edit .env with your Twitter credentials:

USERNAME=your_twitter_username
PASSWORD=your_twitter_password
BACKUP_CODE=your_2fa_backup_code(optional, if you want use login with backup code)

Usage

Quick Start

Run remote Chrome

chromedriver --allowed-ips=LIST_OF_IPS

Basic Commands

# Check configuration and status
python main.py status

# Login to Twitter (saves session)
python main.py login

# Download images from your likes
python main.py download --max-scroll 100

# Start API server
python main.py serve --port 5000

If you're running remote chrome, even no saved session, you could still continue download without login next time(if the chrome profile is not cleared).

Available Commands

# Authentication
python main.py login              # Login and save session
python main.py logout             # Logout and clear session
python main.py status             # Check authentication status

# Downloading
python main.py download                   # Download from your likes
python main.py download --max-scroll 200 # Limit scroll iterations

# API Server
python main.py serve              # Start API server on localhost:5000
python main.py serve --host 0.0.0.0 --port 8080  # Custom host/port


# Enable verbose logging for any command
python main.py -v <command>

# Enable debug mode (captures screenshots and HTML for each step)
python main.py --debug <command>

RESTful API

Start the API server:

python main.py serve --port 5000

Or run directly:

python app.py

API Endpoints

• GET /status - Get authentication and system status
• POST /download - Start download task
• POST /refresh - Refresh page and download new content
• GET /tasks - Get all tasks
• GET /tasks/<id> - Get specific task status
• POST /tasks/<id>/cancel - Cancel running task
• GET /health - Health check

Note: Authentication is handled via CLI. Users should login using python main.py login before using the API.

API Usage Examples

# Check status
curl http://localhost:5000/status

# Start download task
curl -X POST http://localhost:5000/download \
  -H "Content-Type: application/json" \
  -d '{"max_scroll": 100}'

# Start download task with debug mode
curl -X POST http://localhost:5000/download \
  -H "Content-Type: application/json" \
  -d '{"max_scroll": 100, "debug": true}'

# Check task progress
curl http://localhost:5000/tasks/<task-id>

# Refresh and download new content
curl -X POST http://localhost:5000/refresh \
  -H "Content-Type: application/json" \
  -d '{"max_scroll": 50}'

Session Management

The tool automatically saves your authentication session to avoid frequent logins:

• Sessions are saved in session/cookies.json
• Browser profile data in session/chrome_profile/
• Sessions restore automatically on subsequent runs
• Use python main.py logout to clear saved sessions

Directory Structure

lyubitori2/
├── twitter_scraper/          # Main package
│   ├── auth.py              # Authentication handling
│   ├── cli.py               # Click CLI interface
│   ├── config.py            # Configuration management
│   ├── driver.py            # WebDriver management
│   ├── scraper.py           # Image scraping logic
│   └── session.py           # Session persistence
├── main.py                  # CLI entry point
├── run.py                   # Legacy script (preserved)
├── downloaded/              # Scraped images (auto-created)
├── session/                 # Session data (auto-created)
├── screenshots/             # Debug screenshots (auto-created)
├── debug/                   # Debug captures (auto-created when --debug used)
│   ├── screenshots/         # Step-by-step screenshots
│   └── html/                # HTML source captures
└── logs/                    # Application logs (auto-created)

Development

Code Quality

The project uses ruff for linting and formatting:

# Install development dependencies
pip install -r dev-requirements.txt

# Run linter
ruff check .

# Auto-fix linting issues
ruff check --fix .

# Format code
ruff format .

Debug Mode

The --debug flag enables comprehensive debugging capabilities:

# Enable debug mode for any command
python main.py --debug download --max-scroll 50
python main.py --debug login

Debug Output Location:

• debug/screenshots/ - Step-by-step screenshots with timestamps
• debug/html/ - HTML source captures and element information
• debug/debug_summary.txt - Summary of the debug session

API Debug Mode:

# Start download with debug enabled via API
curl -X POST http://localhost:5000/download \
  -H "Content-Type: application/json" \
  -d '{"debug": true}'