Bitrix infrastructure as a code

This repository contains infrastructure code behind Bitrix-based site of my father's metal decking business operating in multiple cities.

It's a Bitrix website completely enclosed within docker-compose to be as portable and maintainable as possible, and a set of scripts around its maintenance like dev site redeploy or production site backup.

Is it fast?

You bet! Here is a performance on Yandex.Cloud server with Intel Cascade Lake 8 vCPUs, 16Gb of RAM and 120Gb SSD 4000 read\write IOPS and 60Mb/s bandwidth.

Getting Started

Follow these steps to get your Bitrix environment up and running:

1. Clone the repository:

   git clone https://github.com/paskal/bitrix.infra.git
   cd bitrix.infra

2. Create Environment Files: Navigate to the private/environment/ directory. You will need to create several environment files based on the templates or examples provided. These include:

   • mysql.env
   • ftp.env
   • dnsrobocert.env (Note: this file is for the certbot service, which uses DNSroboCert technology)
   • zabbix.env
   • updater.env Detailed information about the required variables for each file can be found in the "File structure > /private > private/environment" section of this Readme.

3. Set File Permissions: Before starting the containers for the first time, it's crucial to set the correct file and directory permissions. Run the provided script:

   sudo ./scripts/fix-rights.sh

   This script ensures that services like MySQL, PHP, and Nginx have the necessary access rights. See the "File system permissions" section for more details.

4. Start the Services: Use Docker Compose to start the services. For a basic setup with only core services, run:

   docker-compose up -d

   This project primarily uses pre-built Docker images from a container registry (like GitHub Container Registry - GHCR). Therefore, running docker-compose build or adding the --build flag is generally not necessary for standard usage. Docker Compose will automatically pull the specified images if they are not present locally. You would typically only need to use --build if you have made custom modifications to the Dockerfiles and need to rebuild the images locally.

   To manage optional services, refer to the "Managing Optional Services with Profiles" section.

How to make use of it

You couldn't use it as-is without alterations. However, I tried to make everything as generic as possible to make adoption for another project easy. To use it, read through docker-compose.yml and then read the rest of this Readme. For information about maintenance and utility scripts, see scripts/README.md.

bitrixdock (Russian) project was an inspiration for this one and had way better setup instructions. Please start with it if you don't know what to do with many files in that repo.

File system permissions

All files touched by MySQL use UID/GID 1001, and PHP and Nginx use UID/GID 1000. It is crucial to run the sudo ./scripts/fix-rights.sh script after cloning the repository and creating your environment files, and before running docker-compose up for the first time. This script sets the permissions appropriately for all containers to run correctly.

It might be easier to switch everything to User and Group 1000 for consistency later.

Relevant parts of Bitrix config

Documentation: sessions 1 2 (ru 1, 2), cache (ru)

bitrix/php_interface/dbconn.php

define('BX_CRONTAB_SUPPORT', true);

define("BX_USE_MYSQLI", true);
define("DBPersistent", true);
define("DELAY_DB_CONNECT", true);
$DBType = "mysql";
$DBHost = "localhost";
$DBName = "<DBNAME>";
$DBLogin = "<DBUSER>";
$DBPassword = "<DBPASSWORD>";
define('BX_TEMPORARY_FILES_DIRECTORY', '/tmp');

define("BX_CACHE_TYPE", "memcache");
define("BX_CACHE_SID", $_SERVER["DOCUMENT_ROOT"]."#01");
define("BX_MEMCACHE_HOST", "memcached");
define("BX_MEMCACHE_PORT", "11211");
define('BX_SECURITY_SESSION_MEMCACHE_HOST', 'memcached');
define('BX_SECURITY_SESSION_MEMCACHE_PORT', 11211);

bitrix/.settings.php

  'session' => array (
  'value' =>
  array (
    'mode' => 'separated',
    'lifetime' => 14400,
    'handlers' =>
    array (
      'kernel'  => 'encrypted_cookies',
      'general' =>
      array (
        'type' => 'memcache',
        'host' => 'memcached',
        'port' => '11211',
      ),
    ),
  ),
  'readonly' => true,
  ),
  'connections' =>
  array (
    'value' =>
    array (
      'default' =>
      array (
        'className' => '\\Bitrix\\Main\\DB\\MysqliConnection',
        'host' => 'localhost',
        'database' => '<DBNAME>',
        'login' => '<DBUSER>',
        'password' => '<DBPASSWORD>',
        'options' => 2.0,
      ),
    ),
    'readonly' => true,
  ),

bitrix/.settings_extra.php

<?php
return array(
  'cache' => array(
    'value' => array(
      'type' => 'memcache',
      'memcache' => array(
        'host' => 'memcached',
        'port' => '11211',
      ),
      'sid' => $_SERVER["DOCUMENT_ROOT"]."#01"
    ),
  ),
);
?>

What's inside?

Core

• Nginx with brotli proxying requests to php-fpm and serving static assets directly
• php-fpm (7 8 8.1 8.2 ) for bitrix with msmtp for mail sending
• Percona MySQL because of it's monitoring capabilities
• memcached for bitrix cache and user sessions

Optional

• PHP cron container (php-cron) with same settings as PHP serving web requests
• adminer (adminer) as phpmyadmin alternative for work with MySQL
• pure-ftpd (ftp) for ftp access
• DNSroboCert (certbot) for Let's Encrypt HTTPS certificate generation using the adferrand/dnsrobocert image.
• zabbix-agent2 (zabbix-agent) for monitoring
• Webhooks server (updater) for automated tasks.

File structure

/config

• cron/php-cron.cron is a list of cron tasks to run in php-cron container, only cron_events.php is required for Bitrix and others are specific to this site, must be owned by root:root and have access rights 0644 - fixable by running scripts/fix-rights.sh

• cron/host.cron is a list of cron tasks to run on the host machine

• mysql/my.cnf is a MySQL configuration, applied on top of package-provided my.cnf

• nginx directory contains the build Dockerfile, as well as following (HTTPS) configuration:

  • bitrix proxy, separate for dev and prod
  • adminer proxy
  • HTTP to HTTPS redirects
  • stub status page listening on localhost for Zabbix monitoring

• php directory contains the build Dockerfiles (e.g., Dockerfile.8.1, Dockerfile.8.2, Dockerfile.8.3) and php configuration, applied on top of package-provided one.

/logs

mysql, nginx, php logs. cron and msmtp logs will be written to the php directory.

/scripts

Maintenance and utility scripts for the infrastructure. See scripts/README.md for detailed documentation of each script.

/web

Site files in directories web/prod and web/dev.

/private

• private/environment is a directory with environment files for docker-compose

  • private/environment/mysql.env should contain the following variables:

    MYSQL_ROOT_PASSWORD=mysql_root_password
    MYSQL_USER=bitrix_user
    MYSQL_PASSWORD=bitrix_mysql_password

  • private/environment/ftp.env should contain the following variables:

    FTP_USER_NAME=ftp_username
    FTP_USER_PASS=ftp_password

  • private/environment/dnsrobocert.env should contain Yandex Cloud DNS API key for the certbot service (which uses the adferrand/dnsrobocert image):

    # Run `yc components update` once to get the key, and `update-dns-token.sh` script will renew it automatically afterwards
    AUTH_KEY=insert_key_there
    DNS_ZONE_ID=insert_zone_id_there
    

  • private/environment/updater.env should contain a secret key for the updater service:

    KEY=your_secret_key_here

  • private/environment/zabbix.env should contain the following variables:

    ZBX_HOSTNAME=myhostname
    ZBX_SERVER_HOST=zabbix.example.com

    MySQL setup if you want to use Zabbix for monitoring of the database:

    drop user if exists 'zbx_monitor'@'localhost';
    create user if not exists `zbx_monitor`@`localhost` identified by 'generate_random_password_here';
    grant process, replication client, show databases, show view on *.* to `zbx_monitor`@`localhost`;

• private/letsencrypt directory will be filled with certificates after the certbot service (using DNSroboCert technology) runs.

• private/mysql-data directory will be filled with database data automatically after the start of mysql container

• private/mysqld directory will contain MySQL unix socket for connections without network

• private/msmtprc is a file with msmtp configuration

Managing Optional Services with Profiles

This project uses Docker Compose profiles to manage optional services. This allows you to run only the services you need, saving resources. The core services (nginx, php, php-cron, mysql, memcached, memcached-sessions) will always start.

⚠️ Breaking Change Notice: If you were previously running services like adminer, zabbix-agent, updater, or ftp, they will no longer start automatically with docker-compose up -d. You must now explicitly enable them using profiles (see examples below) or set the COMPOSE_PROFILES environment variable.

Here are the available profiles and the services they enable:

• certs: Enables the certbot service (using DNSroboCert technology via the adferrand/dnsrobocert image) for managing SSL certificates.
• monitoring: Enables zabbix-agent for Zabbix monitoring.
• dbadmin: Enables adminer for database administration.
• hooks: Enables updater for handling webhooks.
• ftp: Enables ftp for FTP access.

Examples:

• To run only the core services:

  docker-compose up -d

• To run core services plus adminer and ftp:

  docker-compose --profile dbadmin --profile ftp up -d

• Alternatively, you can set profiles using the COMPOSE_PROFILES environment variable:

  COMPOSE_PROFILES=dbadmin,ftp docker-compose up -d

  Or export it for the session:

  export COMPOSE_PROFILES=dbadmin,ftp
  docker-compose up -d

• To run all services, including all defined profiles:

  docker-compose --profile "*" up -d

  As mentioned in "Getting Started," this project uses pre-built images. If you've made custom changes to Dockerfiles or need to ensure you have the absolute latest build not yet reflected in the pre-built images, you can add the --build flag (e.g., docker-compose --profile "*" up --build -d).

Advanced Usage

Switching PHP Versions

This project is configured to support multiple PHP versions. Dockerfiles for different versions (e.g., 8.1, 8.2, 8.3) are available in the config/php/ directory.

To switch the PHP version used by the php and php-cron services:

1. Edit docker-compose.yml:

   • Locate the php service definition.
   • Modify the build.context and build.dockerfile to point to the desired Dockerfile. For example, to switch to PHP 8.3:

     php:
       build:
         context: ./config/php
         dockerfile: Dockerfile.8.3 # Changed from Dockerfile.8.2
       image: ghcr.io/paskal/bitrix-php:8.3 # Update image tag
       # ... rest of the service definition

   • Repeat the same changes for the php-cron service definition, ensuring the image tag is also updated.

2. Rebuild the PHP images: This is a scenario where you would need to build the images:

   docker-compose build php php-cron
   # Or, if you are starting the services at the same time:
   # docker-compose up -d --build php php-cron 
   # (or simply 'docker-compose up -d --build' if you want to ensure all buildable services are updated)

   After building, you can start the services as usual:

   docker-compose up -d

For a more dynamic approach to switching PHP versions, you could consider:

• Using an environment variable (e.g., PHP_VERSION) in your docker-compose.yml to specify the Dockerfile path and image tag. You would then set this variable in your shell or a .env file.
• Utilizing Docker Compose override files to specify different PHP configurations.

Routine operations

Disaster recovery

To start the recovery you should have a machine with the latest Ubuntu with static external IP with DDoS protection attached to it, created in the Yandex.Cloud. It should be created with 100Gb of disk space, 12Gb of RAM and 8 cores.

SSH to the machine you want to set up as a new server and then execute the following, then follow the instructions of the script:

# that is preparation for backup restoration
sudo mkdir -p /web
sudo chown $USER:$(id -g -n) /web
sudo apt-get update >/dev/null
sudo apt-get -y install git >/dev/null
git clone https://github.com/paskal/bitrix.infra.git /web
cd /web
# backup restoration, it's safe to run it multiple times
sudo ./scripts/disaster-recovery.sh

Recovery of files

Presume you have a machine with problems, and you want to roll back the changes:

# restore to directory /web/prod2
# -t 2D means restore from the backup made 2 days
# last argument /web/web/prod2 is the directory to restore to, we're not restoring to the original dir
# so that you can rename it first and then rename this directory to prod
sudo HOME="/home/$(logname)" duplicity -t 2D \
    --no-encryption \
    --s3-endpoint-url https://storage.yandexcloud.net \
    --log-file /web/logs/duplicity.log \
    --archive-dir /root/.cache/duplicity \
    --file-to-restore web/prod  "boto3+s3://favor-group-backup/duplicity_web_favor-group" /web/web/prod2

Cleaning (mem)cache

There are two memcached instances in use, one for site cache and another for sessions. Here are the commands to clean them completely:

# to flush site cache
echo "flush_all" | docker exec -i memcached /usr/bin/nc 127.0.0.1 11211
# to flush all user sessions
echo "flush_all" | docker exec -i memcached-sessions /usr/bin/nc 127.0.0.1 11211

Here is the complete list of commands you can send to it.

Manual certificate renewal

DNS verification of a wildcard certificate is set up automatically through Yandex Cloud DNS via the certbot service (which uses DNSroboCert technology via the adferrand/dnsrobocert image).

To renew the certificate manually, if needed, you can run the following command which uses the certbot command available within the certbot service's container (which runs adferrand/dnsrobocert):

# Note: The service is certbot, and the command inside is also certbot
docker-compose run --rm --entrypoint "\
  certbot certonly \
    --email [email protected] \
    -d example.com -d *.example.com \
    --agree-tos \
    --manual \
    --preferred-challenges dns" certbot

To add required TXT entries, head to DNS entries page of your provider (Yandex Cloud). The certbot service is configured to handle renewals automatically.