Motivation and goals
The Containerization Working Group aims to support running the Dataverse software within containers. Containers may be used in very different contexts such as development, testing, staging and production. They also are a way to run the Dataverse software and its dependencies and integrations in enclosed units on different platforms of operating systems and/or hardware. This working group aims to include many perspectives and use cases to make sure the Dataverse community’s needs are covered.
Where are the images?
- Docker Hub: https://hub.docker.com/u/gdcc
- GitHub Container Registry (GHCR): https://github.com/orgs/gdcc/packages?ecosystem=container
That said, to actually make use of the images and understand how they are tagged, you are encouraged to read the Container Guide, either from the latest release or a preview of the next release of Dataverse.
New features, bug fixes, and use cases 🎁
- 2023-04-02 Dataverse 6.2 was released and our images on Docker Hub have been updated (alpha tag).
- 2024-03-38 Configuration of SMTP using MPCONFIG: https://github.com/IQSS/dataverse/pull/9939
- 2024-02-08 A tutorial has been written on running Dataverse in containers for demo or evaluation purposes.
- 2023-12-12 Dataverse 6.1 was released and our images on Docker Hub have been updated (alpha tag).
- 2023-11-07 Balázs Pataki figured out fast redeploy of Dataverse code to our containers, as discussed and documented, and further enhancements are being pursued in https://github.com/IQSS/dataverse/issues/9590.
- 2023-10-13 The docroot is now configurable so logos can now be uploaded to collections: https://github.com/IQSS/dataverse/pull/9819
- 2023-10-04 Testcontainers for integration testing: https://github.com/IQSS/dataverse/pull/9273 and https://dataverse-guide--9273.org.readthedocs.build/en/9273/developers/testing.html#writing-and-using-a-testcontainers-test and an example test
- 2023-08-28 Images with “unstable” tag updated to Solr 9: https://github.com/IQSS/dataverse/pull/9787
- 2023-08-23 Images with “unstable” tag updated to Java 17: https://github.com/IQSS/dataverse/pull/9764
- 2023-08-11 Images with “unstable” tag updated to Payara 6: https://github.com/IQSS/dataverse/pull/9685
- 2023-08-04 Images with “alpha” tag are now available (currently equivalent to 5.14): https://hub.docker.com/r/gdcc/dataverse/tags
- 2023-07-25 Experimental API test runner: https://github.com/pdurbin/dataverse-api-test-runner
- 2023-07-18 Frontend end to end tests are using containers. See https://github.com/IQSS/dataverse-frontend/pull/131
- 2023-06-23 Lots of documentation enhancements, fulfilling milestone A as well See PR#9584 and dev usage
- 2023-06-12 Container-based dev environment for the new React frontend. See docs, PR#87, and PR#112.
- 2023-06-01 Config Baker image to streamline tasks in Dataverse setup like bootstrapping. See Config Baker docs and PR#9574.
- 2023-05-08 Enable publishing preview application images to ghcr.io for pull requests from forks (
/push-image). See PR#9575 for details. - 2023-04-25 Application images (Dataverse, not just the base image) are being pushed to registries (Docker Hub and GitHub Container Registry). See PR#9447 and related discussion.
- 2023-04-04 DOI JVM options can now be configured using MPCONFIG. See PR#8828.
- 2023-03-20 If you have Java and Maven installed, you can spin up Dataverse in Docker! See dev usage and PR#9439.
Roadmap
We presented a roadmap as a proposal as a series of milestone during the 2023-04-18 Dataverse Community Call (notes, recording):
- Milestone A: For backend (Java) developers (✅)
- Milestone B: For API client testing (✅)
- Milestone C: For an integration/frontend developer (w/o Java) (✅)
- Milestone D: Improve developer experience (80%)
- Milestone E: Demo or evaluation (60%)
- Milestone F: Demo with some configurability (20%)
- Milestone G: Run API tests in containers (40%)
As we merge pull requests, we will update “new features” list above. We’ll use the Containerization column of the Dataverse Global Backlog board to indicate specific issues or pull requests we plan to work on next.
Working group meetings
We welcome anyone to join our meetings! We meet on Thursdays at 9:30am Eastern Time (ET).
The Zoom link is https://harvard.zoom.us/j/91061519853?pwd=U1lQR1ExMlo3Ty9XUVJIM2ZPNW1mdz09
- 2024-04-25, 09:30 Eastern meeting notes
- 2024-04-18, 09:30 Eastern meeting notes, recording
- 2024-04-11, 09:30 Eastern meeting notes, recording
- 2024-04-04 no meeting! Phil, Oliver, and Jan speaking at Distribits
- 2024-03-28, 09:30 Eastern meeting notes, recording
- 2024-03-21, 09:30 Eastern meeting notes, recording
- 2024-03-14, 09:30 Eastern meeting notes, recording
- 2024-03-07, 08:00 CST meeting notes, recording
- 2024-02-29, 09:30 Eastern meeting notes, recording
- 2024-02-22, 09:30 Eastern meeting notes, recording
- 2024-02-15, 09:30 Eastern meeting notes, recording
- 2024-02-08, 09:30 Eastern meeting notes, recording
- 2024-02-01, 09:30 Eastern meeting notes, recording
- 2024-01-25, 09:30 Eastern meeting notes, recording
- 2024-01-18, 09:30 Eastern meeting notes, recording
- 2024-01-11, 09:30 Eastern meeting notes, recording
- 2024-01-04, 09:30 Eastern meeting notes, recording
- No meetings! Happy holidays!
- 2023-12-14, 09:30 Eastern meeting notes, recording
- 2023-12-07, 09:30 Eastern meeting notes, recording
- 2023-11-30, 09:30 Eastern meeting notes, recording
- 2023-11-23 skipping! Happy Thanksgiving! 🦃
- 2023-11-16, 09:30 Eastern meeting notes, recording
- 2023-11-09, 09:30 Eastern meeting notes, recording
- 2023-11-02, 09:30 Eastern meeting notes, recording
- 2023-10-26, 09:30 Eastern meeting notes, recording
- 2023-10-19, 09:30 Eastern meeting notes, recording
- 2023-10-12, 09:30 Eastern meeting notes, recording
- 2023-10-05, 09:30 Eastern meeting notes, recording
- 2023-09-28, 09:30 Eastern meeting notes, recording
- 2023-09-21, 09:30 Eastern meeting notes, recording
- 2023-09-14, 09:30 Eastern meeting notes, recording
- 2023-09-07, 09:30 Eastern meeting notes, recording
- 2023-08-31, 09:30 Eastern meeting notes, recording
- 2023-08-24, 09:30 Eastern meeting notes, recording
- 2023-08-17, 09:30 Eastern meeting notes, recording
- 2023-08-10, 09:30 Eastern meeting notes, recording
- 2023-07-27, 09:30 Eastern meeting notes, recording
- 2023-07-20, 09:30 Eastern meeting notes, recording
- 2023-07-13, 09:30 Eastern meeting notes, recording
- 2023-06-15, 09:30 Eastern meeting notes, recording
- 2023-06-07 meeting (in person at #dataverse2023) notes
- 2023-05-25, 09:30 Eastern meeting notes, recording
- 2023-05-18, 09:30 Eastern meeting notes, recording
- 2023-05-11, 09:30 Eastern meeting notes, recording
- 2023-05-04, 14:00 UTC meeting notes, recording
- 2023-04-27, 14:00 UTC meeting notes, recording
- 2023-04-18, 15:00 UTC meeting (during the Dataverse Community Call) notes, recording
- 2023-04-13, 14:00 UTC meeting notes, recording
- 2023-04-06, 14:00 UTC meeting notes, recording
- 2023-03-30, 14:00 UTC meeting notes, recording
- 2023-03-23, 14:00 UTC meeting notes, recording
- 2023-03-21, 15:00 UTC kickoff (during the Dataverse Community Call) notes, recording
Get in touch
We love to hear feedback from you about our goals and outputs not just during meetings, but also using chat.
Please join us in Zulip, linked from chat.dataverse.org.
References
Outputs:
- Dataverse Containerization Proposal/Roadmap
- Dataverse Container Guide (pre-release preview)
Existing prior containerization efforts:
- Dataverse on K8s: gdcc/dataverse-kubernetes, k8s-docs.gdcc.io
- Derived: EOSC-synergy/dataverse-kubernetes
- Archive in a Box: IQSS/dataverse-docker
- Docker All In One (AIO): IQSS/dataverse/conf/docker-aio
- KU Leuven RDM Repository: libis/rdm-build
- LIP Computing: lip-computing/dataverse
- Patrick Carlson: PR 8709
- NDS Labs Service Catalog: nds-org/ndslabs-specs
Improving this website
Please feel free open an issue or create a pull request at https://github.com/gdcc/ct.gdcc.io