Autolab + Tango Docker Compose Installation
The Autolab Docker Compose installation is a fast and easy production-ready installation and deployment method. It uses a MySQL database for the Autolab deployment, and comes with TLS/SSL support. This is now the preferred way of installing Autolab.
If you are stuck or find issues with the installation process you can either file an issue on our Github repository, or join our Slack here and let us know and we will try our best to help. Also see the debugging section for tips on how to diagnose problems and check out the troubleshooting section if you run into any issues.
First ensure that you have Docker and Docker Compose installed on your machine. See the official Docker docs for the installation steps.
Clone this repository and its Autolab and Tango submodules:
git clone --recurse-submodules -j8 git://github.com/autolab/docker.git autolab-docker
Enter the project directory:
Update our Autolab and Tango submodules to ensure that you are getting the latest versions:
Create initial default configs:
Build the Dockerfiles for both Autolab and Tango:
Run the Docker containers:
docker-compose up -d
Note at this point Nginx will still be crash-looping in the Autolab container because TLS/SSL has not been configuired/disabled yet.
Ensure that the newly created config files have the right permissions, as it may have been modified during the building process:
Perform database migrations for Autolab, which will initialize your database schema:
Create administrative user for Autolab:
This user has full permissions on Autolab and will be able to create other users and designate other admins.
docker-compose.ymlto be the absolute path to the Tango
/<path-to-docker-compose-installation>/Tango/volumes. This is so that Tango knows where to put the output files of its autograded jobs.
# in docker-compose.yml # Modify the below to be the path to volumes on your host machine - DOCKER_TANGO_HOST_VOLUME_PATH=/home/your-user/autolab-docker/Tango/volumes # example path
Stop all containers, as we are going to setup/disable TLS:
If you intend to use TLS later, in
nginx/app.conf, change instances of
<REPLACE_WITH_YOUR_DOMAIN>to your real domain name. Otherwise, if you are not using TLS, in
server_nameto your real domain name.
Continue with TLS setup as outlined in the next section
- Build the autograding image(s) that you want to use in Tango (see the docs for more information). For this setup we will stick to the default Ubuntu 18.04 autograding image:
docker build -t autograding_image Tango/vmms/
Note that we can just run this directly on the host because we are mapping the Docker socket to the Tango container (i.e they are using the same Docker server).
- Start up everything:
docker-compose up -d
Autolab should now be accessible on port 80 (and 443 if you configured TLS)!
Having TLS/SSL configured is important as it helps to ensure that sensitive information like user credentials and submission information are encrypted instead of being sent over in plaintext across the network when users are using Autolab. We have made setting up TLS as easy and pain-free as possible. Using TLS is strongly recommended if you are using Autolab in a production environment with real students and instructors.
There are three options for TLS: using Let's Encrypt (for free TLS certificates), using your own certificate, and not using TLS (suitable for local testing/development, but not recommended for production deployment).
Option 1: Let's Encrypt
- Ensure that your DNS record points towards the IP address of your server
- Ensure that port 443 is exposed on your server (i.e checking your firewall, AWS security group settings, etc)
- Get initial SSL setup script:
domains=(example.com)to the list of domains that your host is associated with, and change
- If necessary, change
staging=1to avoid being rate-limited by Let's Encrypt since there is a limit of 20 certificates/week. Setting this is helpful if you have an experimental setup.
- Run your modified script:
sudo sh ./ssl/init-letsencrypt.sh
Option 2: Using your own TLS certificate
- Copy your private key to
- Copy your certificate to
Generate your dhparams:
openssl dhparam -out ssl/ssl-dhparams.pem 4096
Uncomment the following lines in
# - ./ssl/fullchain.pem:/etc/letsencrypt/live/test.autolab.io/fullchain.pem; # - ./ssl/privkey.pem:/etc/letsencrypt/live/test.autolab.io/privkey.pem; # - ./ssl/ssl-dhparams.pem:/etc/letsencrypt/ssl-dhparams.pem
Option 3: No TLS (not recommended, only for local development/testing)
docker-compose.yml(for all the subsequent steps), comment out the following:
# Comment the below out to disable SSL (not recommended) - ./nginx/app.conf:/etc/nginx/sites-enabled/webapp.conf
Also uncomment the following:
# Uncomment the below to disable SSL (not recommended) # - ./nginx/no-ssl-app.conf:/etc/nginx/sites-enabled/webapp.conf
environment: - DOCKER_SSL=true # set to false for no SSL (not recommended)
Updating Your Docker Compose Deployment
Stop your running instances:
Update your Autolab and Tango repositories:
Rebuild the images with the latest code:
Re-deploy your containers:
Debugging your Deployment
In the (very likely) event that you run into problems during setup, hopefully these steps will help you to help identify and diagnose the issue. If you continue to face difficulties or believe you discovered issues with the setup process please join our Slack here and let us know and we will try our best to help.
Better logging output for Docker Compose
docker-compose up -d runs in detached state and it is not easy to immediately see errors:
$ docker-compose up -d Starting certbot ... done Starting redis ... done Starting mysql ... done Starting tango ... done Recreating autolab ... done
docker-compose up instead to get output from all the containers in real time:
$ docker-compose up Starting certbot ... done Starting mysql ... done Starting redis ... done Starting tango ... done Starting autolab ... done Attaching to redis, mysql, certbot, tango, autolab mysql | [Entrypoint] MySQL Docker Image 8.0.22-1.1.18 tango | 2020-11-11 04:33:19,533 CRIT Supervisor running as root (no user in config file) redis | 1:C 11 Nov 2020 04:33:19.032 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo redis | 1:C 11 Nov 2020 04:33:19.032 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1, just started redis | 1:C 11 Nov 2020 04:33:19.032 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf mysql | [Entrypoint] Starting MySQL 8.0.22-1.1.18 redis | 1:M 11 Nov 2020 04:33:19.033 * Running mode=standalone, port=6379. redis | 1:M 11 Nov 2020 04:33:19.033 # Server initialized tango | 2020-11-11 04:33:19,539 INFO RPC interface 'supervisor' initialized tango | 2020-11-11 04:33:19,539 CRIT Server 'unix_http_server' running without any HTTP authentication checking mysql | 2020-11-11T04:33:19.476749Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.22) starting as process 22 --- output truncated ---
Checking Autolab logs
If the Autolab instance is not working properly, taking a look at both the application logs as well as the Nginx logs in the container will be helpful.
First, find the name of the container. This should be just
autolab by default:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 765d35962f52 autolab-docker_autolab "/sbin/my_init" 31 minutes ago Up 22 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp autolab a5b77b5267b1 autolab-docker_tango "/usr/bin/supervisor…" 7 days ago Up 22 minutes 0.0.0.0:3000->3000/tcp tango 438d8e9f73e2 redis:latest "docker-entrypoint.s…" 7 days ago Up 22 minutes 6379/tcp redis da86acc5a4c3 mysql/mysql-server:latest "/entrypoint.sh mysq…" 7 days ago Up 22 minutes (healthy) 3306/tcp, 33060-33061/tcp mysql 88032e85d669 a2eb12050715 "/bin/bash" 9 days ago Up 2 days compiler
Next get a shell inside the container:
$ docker exec -it autolab bash root@be56be775428:/home/app/webapp#
By default we are in the project directory. Navigate to the
logs directory and
production.log. This contains logs from the Autolab application itself.
root@be56be775428:/home/app/webapp# cd log root@be56be775428:/home/app/webapp/log# tail -f -n +1 production.log
We can also check out our Nginx logs in
root@be56be775428:/home/app/webapp/log# cd /var/log/nginx/ root@be56be775428:/var/log/nginx# ls access.log error.log
Accessing the Rails console
Obtain a shell in the
autolab container as described previously, and do
RAILS_ENV=production bundle exec rails c:
root@be56be775428:/home/app/webapp# RAILS_ENV=production bundle exec rails c Loading production environment (Rails 5.2.0) 2.6.6 :001 > User.all.count => 1
In the example above, if you performed
make create-user you should have at least one user in your database. If there are errors connecting to a database here it is likely that the database was misconfigured.
Checking Tango Logs
Get a shell in the Tango instance, similar to the instructions mentioned previously. The logs are stored in the parent folder (
/opt/TangoService) of the project directory:
$ docker exec -it tango bash root@a5b77b5267b1:/opt/TangoService/Tango# cd .. root@a5b77b5267b1:/opt/TangoService# ls Tango tango_job_manager_log.log tango_log.log root@a5b77b5267b1:/opt/TangoService# tail -f -n +1 tango_job_manager_log.log tango_log.log
Troubleshooting Autolab/Tango Connection
In the Autolab container, try to curl Tango:
root@be56be775428:/home/app/webapp# curl tango:3000 Hello, world! RESTful Tango here!
In the Tango container, try to curl Autolab:
root@a5b77b5267b1:/opt/TangoService/Tango# curl autolab <html> <head><title>301 Moved Permanently</title></head> <body bgcolor="white"> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx/1.14.0 (Ubuntu)</center> </body> </html>
Permission issues in Autolab
Run the following again:
Restarting Autolab Passenger Server
This is useful when you might want to test out some code change within the Autolab container without having to rebuild everything again. These changes can be applied by just restarting the Passenger service that is serving Autolab.
root@8b56488b3fb6:/home/app/webapp# passenger-config restart-app Please select the application to restart. Tip: re-run this command with --help to learn how to automate it. If the menu doesn't display correctly, press '!' ‣ /home/app/webapp (production) Cancel Restarting /home/app/webapp (production)
error: unable to unlink old 'db/schema.rb': Permission denied
If you obtain the following error when attempting to perform
error: unable to unlink old 'db/schema.rb': Permission denied fatal: Could not reset index file to revision 'HEAD'.
This is due to the fact that
db/schema.rb is updated whenever migrations are performed.
db/schema.rb documents the database schema, which depends on the database that you are using, its version, and when the migrations were run. It is likely that your
db/schema.rb will diverge from the one generated by the devs.
You can resolve this by changing the owner of the files to be your current user, and then running
make set-perms afterwards when you start the containers again.