Docker¶

Setup¶

Install Docker and Docker Compose. Now run:

docker-compose -f docker-compose-dev.yml build
docker-compose -f docker-compose-dev.yml up

This builds and runs the docker containers. You can now connect to localhost:28080 to see your Invenio installation. The admin user does not have any password.

boot2docker ssh -vnNT \
    -Llocalhost:29200:localhost:29200 \
    -Llocalhost:28080:localhost:28080 \
    -Llocalhost:26379:localhost:26379 \
    -Llocalhost:25673:localhost:25673 \
    -Llocalhost:25672:localhost:25672 \
    -Llocalhost:23306:localhost:23306

Should you require a fresh installation and therefore wipe all your instance data, run:

docker-compose -f docker-compose-dev.yml rm -f

Interactive sessions¶

For interactive sessions inside the container simply attach a shell to the running instance:

docker exec -it invenio_web_1 bash

export TERM=xterm

Debugging¶

The docker-compose-dev.yml enables Werkzeug, a debugger that automatically kicks in whenever an error occurs. Stacktraces and debugger terminal are available via web interface.

You can also insert a tracepoint into the code to start an interactive debugger session:

import ipdb; ipdb.set_tracepoint()

Furthermore you can debug MySQL at localhost:23306, Elasticsearch at localhost:29200, RabbitMQ via localhost:25672 (webinterface at localhost:25673) and Redis at localhost:26379. You might want to use flower for celery debugging and analysis as well. Just run the following command to open the webinterface at port 5555:

celery flower --broker=amqp://guest:guest@localhost:25672//

Should you require additional information about the behaviour of the different containers as well as the contained processes and their interaction with the system and other processes, the usage of classical Linux tools like Wireshark and sysdig might be helpful.

Code changes and live reloading¶

As long as you do not add new requirements (python and bower) and only change files inside the invenio package, it is not required to rebuild the docker images. Code changes are mirrored to the containers. If Flask supports it, on your system it will automatically reload the application when changes are detected. This sometimes might lead to timeouts in your browser session. Do not worry about this, but be aware to only save files when you are ready for reloading.

As of this writing changing template files do not lead to application reloading and do not purge caches. As a workaround you can simple alter one of the python files, e.g. by using touch.

Building documentation¶

You can also use the Docker container to build the documentation. This can be done by attaching to running container:

docker exec -it invenio_web_1 sphinx-build -nW docs docs/_build/html

Running tests¶

You can also run tests using the Docker containers. Wait until the containers finished setup and the webservice is running. Then use:

docker exec -it invenio_web_1 python setup.py test

Overlays¶

You might want to use build distribute overlays using Docker. Instead of creating an entire new image and rewrite everything from scratch, you can the Invenio Docker image. Start by building the image from a branch or release of your choice:

cd src/invenio
docker build -t invenio:2.0 .

Now go to your overlay and create a Dockerfile that suits your needs, e.g:

# based on the right Invenio base image
FROM invenio:2.0

# get root rights again
USER root

# optional:
# add new packages
# (update apt caches, because it was cleaned from the base image)
#   RUN apt-get update && \
#       apt-get -qy install whatever_you_need

# optional:
# add new packages from pip
#   RUN pip install what_suits_you

# optional:
# add new packages from npm
#   RUN npm update && \
#       npm install fun

# optional:
# make even more modifications

# add content
ADD . /code-overlay
WORKDIR /code-overlay

# fix requirements.txt and install additional dependencies
RUN sed -i '/inveniosoftware\/invenio[@#]/d' requirements.txt && \
    pip install -r requirements.txt --exists-action i

# build
RUN python setup.py compile_catalog

# optional:
# do some cleanup

# step back
# in general code should not be writeable, especially because we are using
# `pip install -e`
RUN mkdir -p /code-overlay/src && \
    chown -R invenio:invenio /code-overlay && \
    chown -R root:root /code-overlay/invenio_demosite && \
    chown -R root:root /code-overlay/scripts && \
    chown -R root:root /code-overlay/setup.* && \
    chown -R root:root /code-overlay/src

# finally step back again
USER invenio

Notice that this Dockerfile must be located in the directory of your overlay. For a full working example, please see invenio-demosite. Here is how to build the demo site:

cd ~/private/src/invenio
git checkout maint-2.0
docker build -t invenio:2.0 .
cd ~/private/src/invenio-demosite
git checkout maint-2.0
docker-compose -f docker-compose-dev.yml build
docker-compose -f docker-compose-dev.yml up

After all the daemons are up and running, you can populate the demo site with demo records:

docker exec -i -t -u invenio inveniodemosite_web_1 \
       inveniomanage demosite populate \
       --packages=invenio_demosite.base --yes-i-know

Done. Your Invenio overlay installation is now up and running.