1
\$\begingroup\$

I have a Dockerfile with ubuntu:22.04 as a base image and a manual Python installation(specific version i.e., 3.11.1) layer which takes a long time. How to cache this manual Python installation layer in the docker image in subsequent CI runs in GitHub actions?

Dockerfile:

FROM --platform=linux/amd64 ubuntu:22.04 as base USER root ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 ENV DEBIAN_FRONTEND noninteractive COPY ZscalerRootCertificate-2048-SHA256.crt /usr/local/share/ca-certificates/ZscalerRootCertificate-2048-SHA256.crt RUN apt-get update && \ apt-get upgrade -y && \ apt-get install -y software-properties-common ca-certificates && \ update-ca-certificates RUN mkdir /python && cd /python && \ wget https://www.python.org/ftp/python/3.11.1/Python-3.11.1.tgz && \ tar -zxvf Python-3.11.1.tgz && \ cd Python-3.11.1 && \ ls -lhR && \ ./configure --enable-optimizations && \ make install && \ rm -rf /python # Fail soon than later, if python wasn't installed RUN python --version COPY . /app WORKDIR /app RUN python -m pip install -U pip RUN python3.11 -m pip install --no-cache-dir --trusted-host pypi.org --trusted-host files.pythonhosted.org -r requirements/core-requirements.txt CMD ["gunicorn", "main:app", "--workers", "2", "--worker-class", "uvicorn.workers.UvicornWorker", "--bind", "0.0.0.0:80"] 

GitHub workflow,

CI.yml:

name: Docker Image CI env: CONTAINER_NAME: my-use-case on: workflow_dispatch: concurrency: group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} cancel-in-progress: true permissions: id-token: write contents: write jobs: integration-tests: name: Integration Test runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 - name: View directory files run: | pwd echo "$PATH" - name: Set up Python uses: actions/setup-python@v4 with: python-version: "3.11.1" - name: Copy Python binaries to Docker context run: | mkdir -p Python/3.11.1 cp -r $RUNNER_TOOL_CACHE/Python/3.11.1/* Python/3.11.1 - name: View directory files run: | ls -lhR echo "$PATH" - name: Install dependencies run: | python --version python -m pip install --upgrade pip - name: View directory files run: | ls -lhR - name: Build & push docker image uses: docker/build-push-action@v2 with: context: . push: false tags: ${{ env.CONTAINER_NAME }}:${{ github.run_number }} file: ./Dockerfile 
\$\endgroup\$
9
  • \$\begingroup\$This question is new but has something relevant to my old question\$\endgroup\$CommentedDec 9, 2023 at 16:22
  • 1
    \$\begingroup\$Why do you need the final two RUN commands after COPY . /app? ("RUN python -m pip install -U pip" and "RUN python3.11 -m pip install ...")\$\endgroup\$
    – Peilonrayz
    CommentedDec 9, 2023 at 18:24
  • \$\begingroup\$@Peilonrayz Yeah, now I realize that I can copy the requriements.txt file first, then the RUN command, and then the final COPY . /app command. Thanks for your details suggestion on my previous question.\$\endgroup\$CommentedDec 9, 2023 at 18:36
  • 1
    \$\begingroup\$If your code depends on the minor version of something you got a bigger problems than speeding up builds on CI.\$\endgroup\$CommentedDec 10, 2023 at 12:51
  • 2
    \$\begingroup\$@AjinkyaKamat Whilst you're probably right you shouldn't depend on minor versions. Python makes no guarantee the code will work across minor versions. You can just look at the typing module in 3.5 which radically changed the internals numerous times across minor versions. Additionally I've seen minor versions introduce bugs. Version pinning to a specific version (minor included) isn't really bad when Python makes no guarantee about minor version compatibility.\$\endgroup\$
    – Peilonrayz
    CommentedDec 10, 2023 at 17:50

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.