DesignSafe-CI Portal

Prequisites for running the portal application

The DesignSafe-CI Portal can be run using Docker and Docker Compose. You will need both Docker and Docker Compose pre-installed on the system you wish to run the portal on.

• Install Docker
• Install Docker Compose

If you are on a Mac or a Windows machine, the recommended method is to install Docker Toolbox, which will install both Docker and Docker Compose as well as Docker Machine, which is required to run Docker on Mac/Windows hosts.

First time setup

1. Clone the repo

   $ git clone https://github.com/DesignSafe-CI/portal.git
   $ cd portal
   

2. Build the container(s)

   $ docker-compose build
   

3. Configure environment variables

   Make a copy of designsafe.env.sample and rename it to designsafe.env. Configure variables as necessary. See designsafe.env.sample for details of configuration.

   Required variables:

   • DJANGO_DEBUG: should be set to True for development
   • DJANGO_SECRET: should be changed for production
   • TAS_*: should be set to enable direct access to django.contrib.admin
   • AGAVE_*: should be set to enable Agave API integration (authentication, etc.)
   • RT_*: should be set to enable ticketing

4. Set up local/testing database

   $ docker-compose up -d
   $ docker exec -it portal_django_1 bash
   # ./manage.py migrate
   # ./manage.py createsuperuser
   

5. Open in browser

   Navigate to http://localhost:8000 in your browser.

   Note: On Mac/Windows hosts running Docker Machine you will need to navigate to the IP Address of the Docker Machine VM. You can find this using the command docker-machine ip <machine-name>. For example:

   $ docker-machine ip default
   192.168.99.100
   

   Then, navigate to: http://192.168.99.100:8000

Next steps

Importing data from production

If you need or want to import data from production to a local development instance running SQLite, you will need to create a datadump.json file using the Django dumpdata command.

To dump data from the production database you will first need an environment/configuration file that is configured for the production database. Then, run the following command:

docker run -it --rm -v $(pwd):/datadump \
    --env-file /path/to/production/designsafe.env \
    designsafeci/portal:latest bin/dumpdata.sh

This will created a file named datadump-YYYYMMDD.json in the current directory.

DO NOT RUN THE FOLLOWING COMMAND WITH THE PRODUCTION CONFIGURATION. IT WILL DESTROY THE DATABASE.

You can load the datadump-YYYYMMDD.json file into your local instance with the command:

docker run -it --rm -v $(pwd):/datadump \
    --env-file /path/to/local/designsafe.env \
    designsafeci/portal:latest bin/loaddata.sh

The result will be a SQLite database file db.sqlite3 in the current directory loaded with the contents of datadump-YYYYMMDD.json.

See this wiki page for additional details.

Developing DesignSafe-CI Portal

Apps

DesignSafe custom apps should be put into designsafe/apps. You can then enable them in the Django settings.py with designsafe.apps.{app_name}.

CSS/Styling

See the DesignSafe Styles Reference for style reference and custom CSS documentation.

Testing

The easiest way to run the tests is from inside a running Docker container. While you can install all the Python/Django/npm dependencies locally (ideally within a virtualenv), this is already done in the docker container.

We assume you have the image built or checked out locally and it is called portal_django.

Django tests

Django tests should be written according to standard Django testing procedures.

You can run Django tests with the following command:

$ docker run -it --rm portal_django python manage.py test --settings=designsafe.test_settings

Frontend tests

Frontend tests are Jasmine tests executed using the Karma engine. Testing guidelines can be found in the AngularJS Developer Guide on Unit Testing.

To run frontend tests, ensure that all scripts and test scripts are configured in karma-conf.js and then run the command:

$ docker run -it --rm portal_django bin/run-tests.sh

Development setup

Use docker-compose to run the portal in development. The default compose file, docker-compose.yml runs the main django server in development mode with a redis service for websockets support. You can optionally enable the EF sites for testing.

$ docker-compose build
$ docker-compose up

When using this compose file, your Agave Client should be configured with a callback_url of http://$DOCKER_HOST_IP:8000/auth/agave/callback/.

For developing some services, e.g. Box.com integration, https support is required. To enable an Nginx http proxy run using the docker-compose-http.yml file. This file configures the same services as the default compose file, but it also sets up an Nginx proxy secured by a self-signed certificate.

$ docker-compose -f docker-compose-http.yml build
$ docker-compose -f docker-compose-http.yml up

When using this compose file, your Agave Client should be configured with a callback_url of https://$DOCKER_HOST_IP/auth/agave/callback/.

Agave filesystem setup

1. Delete all of the old metadata objects using this command:

metadata-list Q '{"name": "designsafe metadata"}' | while read x; do metadata-delete $x; done; 2. Run dsapi/agave/tools/bin/walker.py to create the metadata objects for the existing files in your FS.

python portal/dsapi/agave/tools/bin/walker.py <command> <api_server> <token> <systemId> <base_folder>

• base_folder is your username, if you want to fix everything under your home dir.
• command:
  • files: Walk through the files and print their path.
  • meta: Walk through the metadata objs in a filesystem-like manner and print their path.
  • files-fix: Check if there's a meta obj for every file, if not create the meta obj.
  • meta-fix: Check if there's a file for every meta obj, if not delete the meta obj.

Production setup

Production deployment is managed by ansible. See https://github.com/designsafe-ci/ansible.