.env file values used as args not passed to DockerFile

[Geekimo]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID:

Expected behavior

I have to variables defined in a .env file at the same level as my docker-compose.yml file.
When running docker compose config, I can see the values that I defined in my .env file as build arguments.
But when I run the build, values are missing and default values defined using ARG MY_ARG=123is used instead of the ones that I defined.
It stopped working today as I updated my Docker setup. My stack haven't moved in weeks.
I tested using Docker Desktop on Windows (WSL2) and everything works as expected.

Actual behavior

Information

• Is it reproducible? Only on Mac M1
• Is the problem new? Yes, works on Windows, and previously worked in M1 preview
• Did the problem appear with an update? Yes, but doesn't know which, as I didn't rebuild my containers for weeks then.
• macOS Version: Big Sur 11.3.1
• Intel chip or Apple chip: M1
• Docker Desktop Version: 3.3.3

Steps to reproduce the behavior

Set a docker compose projet, with build args, using variables defined in a .env file. Check that variables are properly loaded using docker compose config, then try to use theses variables in you Dockerfile, and they won't be there.

Files:

• .env

USER_ID=501
GROUP_ID=502

• docker-compose.yml

version: '3.2'
services:
[...]
  api:
    build:
      context: ./app
      dockerfile: Dockerfile.api
      args:
        USER_ID: ${USER_ID:-0}
        GROUP_ID: ${GROUP_ID:-0}
    volumes:
      - ./project/api:/var/www
    depends_on:
      - db
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`api.myproject.local`)"
      - "traefik.docker.network=traefik_webgateway"
    networks:
      - web
      - local
[...]

• ./app/Dockerfile.api

FROM amd64/php:8.0-apache

ARG USER_ID
ARG GROUP_ID

ENV ACCEPT_EULA=Y

# Fix debconf warnings upon build
ARG DEBIAN_FRONTEND=noninteractive

RUN if [ ${USER_ID:-0} -ne 0 ] && [ ${GROUP_ID:-0} -ne 0 ]; then \
    userdel -f www-data &&\
    if getent group www-data ; then groupdel www-data; fi &&\
    groupadd -g ${GROUP_ID} www-data &&\
    useradd -l -u ${USER_ID} -g www-data www-data &&\
    install -d -m 0755 -o www-data -g www-data /home/www-data \
;fi

WORKDIR /var/www

RUN apt-get update && apt-get install -y \
        apt-transport-https \
        apt-utils \
        ca-certificates \
        curl \
        git \
        gnupg2 \
        libcurl4-gnutls-dev \
        libfreetype6-dev \
        libicu-dev \
        libjpeg-dev \
        libjpeg62-turbo-dev \
        libldap2-dev \
        libldb-dev \
        libmcrypt-dev \
        libonig-dev \
        libpng-dev \
        libxml2-dev \
        libzip-dev \
        lsb-release \
        unzip \
        wget

RUN pecl channel-update pecl.php.net
RUN pecl install apcu
RUN pecl install xdebug

RUN docker-php-ext-configure gd --with-freetype --with-jpeg --enable-gd
RUN docker-php-ext-install -j$(nproc) gd
RUN docker-php-ext-install -j$(nproc) iconv
RUN docker-php-ext-install -j$(nproc) calendar
RUN docker-php-ext-install -j$(nproc) dom
RUN docker-php-ext-install -j$(nproc) curl
RUN docker-php-ext-install -j$(nproc) intl
#RUN docker-php-ext-install -j$(nproc) json
RUN docker-php-ext-install -j$(nproc) mbstring
RUN docker-php-ext-install -j$(nproc) zip
RUN docker-php-ext-install -j$(nproc) opcache
RUN docker-php-ext-install -j$(nproc) pdo
RUN docker-php-ext-enable apcu \
    && docker-php-ext-enable xdebug

# Install MS ODBC Driver for SQL Server
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
    && curl https://packages.microsoft.com/config/debian/9/prod.list > /etc/apt/sources.list.d/mssql-release.list \
    && apt-get update \
    && apt-get -y --no-install-recommends install msodbcsql17 unixodbc-dev \
    && pecl install sqlsrv \
    && pecl install pdo_sqlsrv

RUN docker-php-ext-enable pdo_sqlsrv
RUN docker-php-ext-enable sqlsrv

RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
RUN php -r "if (hash_file('sha384', 'composer-setup.php') === file_get_contents('https://composer.github.io/installer.sig')) { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
RUN php composer-setup.php --install-dir=/usr/bin --filename=composer
RUN php -r "unlink('composer-setup.php');"

ADD php.ini /usr/local/etc/php/conf.d/
ADD 000-default.api.conf /etc/apache2/sites-available/000-default.conf

RUN a2enmod rewrite

USER www-data

VOLUME ["/var/www"]

[Geekimo]

Run from Windows correctly displays UID and GID swapped with env values (uid=1000,gid=1000)

 => [ 2/29] RUN if [ 1000 -ne 0 ] && [ 1000 -ne 0 ]; then     userdel -f www-data &&    if getent group www-data   0.3s

[Geekimo]

Same files on OSX on a M1 (uid:501,gid=502):

 => [..._api 4/5] RUN if [ ${USER_ID:-0} -ne 0 ] && [ ${GROUP_ID:-0} -ne 0 ]; then     userdel -f www-data &&    if getent group www-data ; then groupdel www-data; fi &&    groupadd -g ${GROUP_  0.2s

[stephen-turner]

@Geekimo Does it work if you use docker-compose instead of docker compose? (The latter is a new, experimental reimplementation, so it may still have some missing features).

[Geekimo]

Hello,
Sorry for the late answer.
After a quick test, it only occurs when using docker compose. Everything works fine with the old docker-compose command.

[stephen-turner]

OK, thanks, I'll pass it on to the correct team.