This document provides an overview of the core service environments within the Access Platform (Canadiana, Heritage, Parl, GAC, NRCan), including Docker service containers and associated Solr/Swift data stores.
- We can either have 1 Solr server with 3 cores per environment, or 3 Solr servers with 1 core per environment.
- We have one Swift cluster, but we will try to break down content into separate Swift containers.
- We can either build a REST API over the PDF Swift container (additional Docker container) or expose it publicly, since we serve PDFs directly to end users.
- We can swap to different IIIF Image server from Cantaloupe - See: Loris, IIPImage, RAIS
- We are entirely removing our use of CouchDB for access
- We are sequentially rolling out the following environments. We will first work on Canadiana.
- We can think about having an additional -test environment per host environment
- There is no change in the schemas between environments for the data stores.
URL: https://www.canadiana.ca/
| Service | Docker Container | Data Stores |
|---|---|---|
| Blacklight Search | blacklight-canadiana, internal port: 3000, .env (access protected) | Solr core: blacklight_marc_canadiana |
| IIIF Content Search | iiif-cs-canadiana, internal port: 3000, .env (access protected) | Solr core: content_search_canadiana |
| IIIF Presentation | iiif-p-canadiana, internal port: xxxx, .env (access protected) | Swift container: iiif_pres_canadiana, Redis cache: iiif_pres_canadiana |
| IIIF Image | (Cantaloupe/Loris/etc.) | Swift container: iiif_image_canadiana |
| N/A | Swift container: pdf_canadiana | |
| ARK | ark-canadiana, internal port: xxxx, .env (access protected) | Solr core: ark_map_canadiana |
URL: https://heritage.canadiana.ca/
| Service | Docker Container | Data Stores |
|---|---|---|
| Blacklight Search | blacklight-heritage, port: xxxx, env (access protected) | Solr core: blacklight_marc_heritage |
| IIIF Content Search | iiif-cs-heritage, port: xxxx, env (access protected) | Solr core: content_search_heritage |
| IIIF Presentation | iiif-p-heritage, port: xxxx, env (access protected) | Swift container: iiif_pres_heritage, Redis cache: iiif_pres_heritage |
| IIIF Image | (Cantaloupe/Loris/etc.) | Swift container: iiif_image_heritage |
| N/A | Swift container: pdf_heritage | |
| ARK | ark-heritage, port: xxxx, env (access protected) | Solr core: ark_map_heritage |
URL: https://parl.canadiana.ca/
| Service | Docker Container | Data Stores |
|---|---|---|
| Blacklight Search | blacklight-parl, port: xxxx, env (access protected) | Solr core: blacklight_marc_parl |
| IIIF Content Search | iiif-cs-parl, port: xxxx, env (access protected) | Solr core: content_search_parl |
| IIIF Presentation | iiif-p-parl, port: xxxx, env (access protected) | Swift container: iiif_pres_parl, Redis cache: iiif_pres_parl |
| IIIF Image | (Cantaloupe/Loris/etc.) | Swift container: iiif_image_parl |
| N/A | Swift container: pdf_parl | |
| ARK | ark-parl, port: xxxx, env (access protected) | Solr core: ark_map_parl |
URL: https://gac.canadiana.ca/
| Service | Docker Container | Data Stores |
|---|---|---|
| Blacklight Search | blacklight-gac, port: xxxx, env (access protected) | Solr core: blacklight_marc_gac |
| IIIF Content Search | iiif-cs-gac, port: xxxx, env (access protected) | Solr core: content_search_gac |
| IIIF Presentation | iiif-p-gac, port: xxxx, env (access protected) | Swift container: iiif_pres_gac, Redis cache: iiif_pres_gac |
| IIIF Image | (Cantaloupe/Loris/etc.) | Swift container: iiif_image_gac |
| N/A | Swift container: pdf_gac | |
| ARK | ark-gac, port: xxxx, env (access protected) | Solr core: ark_map_gac |
URL: https://nrcan.canadiana.ca/
| Service | Docker Container | Data Stores |
|---|---|---|
| Blacklight Search | blacklight-nrcan, port: xxxx, env (access protected) | Solr core: blacklight_marc_nrcan |
| IIIF Content Search | iiif-cs-nrcan, port: xxxx, env (access protected) | Solr core: content_search_nrcan |
| IIIF Presentation | iiif-p-nrcan, port: xxxx, env (access protected) | Swift container: iiif_pres_nrcan, Redis cache: iiif_pres_nrcan |
| IIIF Image | (Cantaloupe/Loris/etc.) | Swift container: iiif_image_nrcan |
| N/A | Swift container: pdf_nrcan | |
| ARK | ark-nrcan, port: xxxx, env (access protected) | Solr core: ark_map_nrcan |
| Portal | Canvases | Image Source | OCR PDF | TOTAL |
|---|---|---|---|---|
| canadiana | 21,271,334 | 18.62 TiB | 11.63 TiB | 30.25 TiB |
| heritage | 43,195,033 | 8.61 TiB | 4.49 TiB | 13.11 TiB |
| parl | 4,728,789 | 4.28 GiB | 2.40 TiB | 2.40 TiB |
| gac | 1,389,764 | 1.82 TiB | 322.51 GiB | 2.13 TiB |
| nrcan | 261,403 | 651.74 MiB | 4.11 GiB | 4.74 GiB |
| Portal | Manifests | Manifest PDF |
|---|---|---|
| canadiana | 518,672 | 11.87 TiB |
| heritage | 24,834 | 4.20 TiB |
| parl | 6,640 | 2.39 TiB |
| gac | 35,584 | 270.42 GiB |
| nrcan | 688 | 4.17 GiB |
The end user goes to the Blacklight docker container to search the access platform. Blacklight calls the other web server docker containers to receive content to show the end user.
To see a list of Issues for the implementation of this plan, click here.