Autolab + Tango Docker Compose Installation
The Autolab Docker Compose installation is a fast and easy production-ready installation and deployment method. It uses a MySQL database for the Autolab deployment, and comes with SSL support. This is now the preferred way of installing Autolab.
The Docker Compose installation method is relatively new and so you may likely run into issues. If you are stuck or find issues with the installation process please join our Slack here and let us know and we will try our best to help. Also see the debugging section for tips on how to diagnose problems.
Installation
First ensure that you have Docker and Docker Compose installed on your machine. See the official Docker docs for the installation steps.
- Clone this repository and its submodules:
git clone --recurse-submodules -j8 git://github.com/autolab/docker.git autolab-docker
- Enter the project directory:
cd autolab-docker
- Update submodules:
make update
- Create initial configs:
make
- Build the Dockerfiles:
docker-compose build
- Run the containers:
docker-compose up -d
. Note at this point Nginx will still be crash-looping in the Autolab container because SSL has not been configuired/disabled yet. - Ensure that the newly created config files have the right permissions:
make set-perms
- Perform migrations:
make db-migrate
- Create initial root user:
make create-user
- Stop all containers:
docker-compose stop
- Continue with TLS setup as outlined in the next section
- Build the autograding image(s) that you want to use in Tango (see the docs for more information). For this setup we will stick to the default Ubuntu 14.04 autograding image:
docker build -t autograding_image Tango/vmms/
. Note that we can just run this directly on the host because we are mapping the Docker socket to the Tango container (i.e they are using the same Docker server). - Start up everything:
docker-compose up -d
. Autolab should now be accessible on port 80 (and 443 if you configured SSL)!
Configuring SSL/TLS
There are three options for TLS: using Let's Encrypt (for free TLS certificates), using your own certificate, and not using TLS (not recommended for production deployment).
Option 1: Let's Encrypt
- Ensure that your DNS record points towards the IP address of your server
- Ensure that port 443 is exposed on your server (i.e checking your firewall, AWS security group settings, etc)
- Get initial SSL setup script:
make ssl
- In
ssl/init-letsencrypt.sh
, changedomains=(example.com)
to the list of domains that your host is associated with, and changeemail
to be your email address so that Let's Encrypt will be able to email you when your certificate is about to expire - If necessary, change
staging=0
tostaging=1
to avoid being rate-limited by Let's Encrypt since there is a limit of 20 certificates/week. Setting this is helpful if you have an experimental setup. - Run your modified script:
sudo sh ./ssl/init-letsencrypt.sh
Option 2: Using your own TLS certificate
- Copy your private key to ./ssl/privkey.pem
- Copy your certificate to ./ssl/fullchain.pem
- Generate your dhparams, i.e
openssl dhparam -out ./ssl/ssl-dhparams.pem 4096
- Uncomment the following lines in
docker-compose.yml
:
# - ./ssl/fullchain.pem:/etc/letsencrypt/live/test.autolab.io/fullchain.pem;
# - ./ssl/privkey.pem:/etc/letsencrypt/live/test.autolab.io/privkey.pem;
# - ./ssl/ssl-dhparams.pem:/etc/letsencrypt/ssl-dhparams.pem
Option 3: No TLS (not recommended, only for local development/testing)
- In
docker-compose.yml
(for all the subsequent steps), comment out the following:
# Comment the below out to disable SSL (not recommended)
- ./nginx/app.conf:/etc/nginx/sites-enabled/webapp.conf
Also uncomment the following:
# Uncomment the below to disable SSL (not recommended)
# - ./nginx/no-ssl-app.conf:/etc/nginx/sites-enabled/webapp.conf
Lastly set DOCKER_SSL=false
:
environment:
- DOCKER_SSL=true # set to false for no SSL (not recommended)
Updating Autolab/Tango Deployment
- Stop your running instances:
docker-compose stop
- Run
make update
to update your Autolab and Tango repositories - Run
docker-compose build
to rebuild the images with the latest code - Re-deploy your containers with
docker-compose up
Debugging your Deployment
In the (very likely) event that you run into problems during setup, hopefully these steps will help you to help identify and diagnose the issue. If you continue to face difficulties or believe you discovered issues with the setup process please join our Slack here and let us know and we will try our best to help.
Better logging output for Docker Compose
By default, docker-compose up -d
runs in detached state and it is not easy to immediately see errors:
$ docker-compose up -d
Starting certbot ... done
Starting redis ... done
Starting mysql ... done
Starting tango ... done
Recreating autolab ... done
Use docker-compose up
instead to get output from all the containers in real time:
$ docker-compose up
Starting certbot ... done
Starting mysql ... done
Starting redis ... done
Starting tango ... done
Starting autolab ... done
Attaching to redis, mysql, certbot, tango, autolab
mysql | [Entrypoint] MySQL Docker Image 8.0.22-1.1.18
tango | 2020-11-11 04:33:19,533 CRIT Supervisor running as root (no user in config file)
redis | 1:C 11 Nov 2020 04:33:19.032 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 11 Nov 2020 04:33:19.032 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
redis | 1:C 11 Nov 2020 04:33:19.032 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
mysql | [Entrypoint] Starting MySQL 8.0.22-1.1.18
redis | 1:M 11 Nov 2020 04:33:19.033 * Running mode=standalone, port=6379.
redis | 1:M 11 Nov 2020 04:33:19.033 # Server initialized
tango | 2020-11-11 04:33:19,539 INFO RPC interface 'supervisor' initialized
tango | 2020-11-11 04:33:19,539 CRIT Server 'unix_http_server' running without any HTTP authentication checking
mysql | 2020-11-11T04:33:19.476749Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.22) starting as process 22
--- output truncated ---
Checking Autolab logs
If the Autolab instance is not working properly, taking a look at both the application logs as well as the Nginx logs in the container will be helpful.
First, find the name of the container. This should be just autolab
by default:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
765d35962f52 autolab-docker_autolab "/sbin/my_init" 31 minutes ago Up 22 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp autolab
a5b77b5267b1 autolab-docker_tango "/usr/bin/supervisor…" 7 days ago Up 22 minutes 0.0.0.0:3000->3000/tcp tango
438d8e9f73e2 redis:latest "docker-entrypoint.s…" 7 days ago Up 22 minutes 6379/tcp redis
da86acc5a4c3 mysql/mysql-server:latest "/entrypoint.sh mysq…" 7 days ago Up 22 minutes (healthy) 3306/tcp, 33060-33061/tcp mysql
88032e85d669 a2eb12050715 "/bin/bash" 9 days ago Up 2 days compiler
Next get a shell inside the container:
$ docker exec -it autolab bash
root@be56be775428:/home/app/webapp#
By default we are in the project directory. Navigate to the logs
directory and cat
or tail
production.log
. This contains logs from the Autolab application itself.
root@be56be775428:/home/app/webapp# cd log
root@be56be775428:/home/app/webapp/log# tail -f -n +1 production.log
We can also check out our Nginx logs in /var/log/nginx/
:
root@be56be775428:/home/app/webapp/log# cd /var/log/nginx/
root@be56be775428:/var/log/nginx# ls
access.log error.log
Accessing the Rails console
Obtain a shell in the autolab
container as described previously, and do RAILS_ENV=production bundle exec rails c
:
root@be56be775428:/home/app/webapp# RAILS_ENV=production bundle exec rails c
Loading production environment (Rails 5.2.0)
2.6.6 :001 > User.all.count
=> 1
In the example above, if you performed make create-user
you should have at least one user in your database. If there are errors connecting to a database here it is likely that the database was misconfigured.
Checking Tango Logs
Get a shell in the Tango instance, similar to the instructions mentioned previously. The logs are stored in the parent folder (/opt/TangoService
) of the project directory:
$ docker exec -it tango bash
root@a5b77b5267b1:/opt/TangoService/Tango# cd ..
root@a5b77b5267b1:/opt/TangoService# ls
Tango tango_job_manager_log.log tango_log.log
root@a5b77b5267b1:/opt/TangoService# tail -f -n +1 tango_job_manager_log.log tango_log.log
Troubleshooting Autolab/Tango Connection
In the Autolab container, try to curl Tango:
root@be56be775428:/home/app/webapp# curl tango:3000
Hello, world! RESTful Tango here!
In the Tango container, try to curl Autolab:
root@a5b77b5267b1:/opt/TangoService/Tango# curl autolab
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.14.0 (Ubuntu)</center>
</body>
</html>
Permission issues in Autolab
Run make set-perms
again
Restarting Autolab Passenger Server
This is useful when you might want to test out some code change within the Autolab container without having to rebuild everything again. These changes can be applied by just restarting the Passenger service that is serving Autolab.
Run passenger-config restart-app
:
root@8b56488b3fb6:/home/app/webapp# passenger-config restart-app
Please select the application to restart.
Tip: re-run this command with --help to learn how to automate it.
If the menu doesn't display correctly, press '!'
‣ /home/app/webapp (production)
Cancel
Restarting /home/app/webapp (production)