Skip to content

feat(in): export all git metadata to files for pipeline access #456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat(in): export all git metadata to files for pipeline access #456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 28 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -478,27 +478,43 @@ the case.

#### Additional files populated

* `.git/committer`: For committer notification on failed builds.
This special file `.git/committer` which is populated with the email address
of the author of the last commit. This can be used together with an email
resource like [mdomke/concourse-email-resource](https://github.com/mdomke/concourse-email-resource)
to notify the committer in an on_failure step.
* `.git/ref`: Version reference detected and checked out. It will usually contain the commit SHA-1 ref, but also the
detected tag name when using `tag_filter` or `tag_regex`.

* `.git/ref`: Version reference detected and checked out. It will usually contain
the commit SHA-1 ref, but also the detected tag name when using `tag_filter` or
`tag_regex`.
* `.git/commit`: Full SHA-1 commit hash.

* `.git/short_ref`: Short (first seven characters) of the `.git/ref`. Can be templated with `short_ref_format`
parameter.

* `.git/branch`: Name of the original branch that was cloned.

* `.git/short_ref`: Short (first seven characters) of the `.git/ref`. Can be templated with `short_ref_format` parameter.
* `.git/tags`: Comma-separated list of tags associated with this commit.

* `.git/author`: Commit author name.

* `.git/author_date`: Timestamp when the author originally created the commit.

* `.git/committer`: For committer notification on failed builds. This special file `.git/committer` which is populated
with the email address of the author of the last commit. This can be used together with an email resource
like [mdomke/concourse-email-resource](https://github.com/mdomke/concourse-email-resource) to notify the committer in
an on_failure step.

* `.git/committer_name`: Name of the commit author.

* `.git/committer_date`: Timestamp when the commit was added to the repository.

* `.git/commit_message`: For publishing the Git commit message on successful builds.

* `.git/commit_timestamp`: For tagging builds with a timestamp.

* `.git/describe_ref`: Version reference detected and checked out. Can be templated with `describe_ref_options` parameter.
By default, it will contain the `<latest annoted git tag>-<the number of commit since the tag>-g<short_ref>` (eg. `v1.6.2-1-g13dfd7b`).
If the repo was never tagged before, this falls back to a short commit SHA-1 ref.
* `.git/describe_ref`: Version reference detected and checked out. Can be templated with `describe_ref_options`
parameter. By default, it will contain the
`<latest annoted git tag>-<the number of commit since the tag>-g<short_ref>` (eg. `v1.6.2-1-g13dfd7b`). If the repo
was never tagged before, this falls back to a short commit SHA-1 ref.

* `.git/url`: Web URL to view the commit (if applicable).

* `.git/metadata.json`: Complete metadata object in JSON format containing all metadata fields.

### `out`: Push to a repository

Expand Down
24 changes: 20 additions & 4 deletions assets/in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,16 +254,32 @@ git --no-pager log -1 --pretty=format:"%an" > .git/committer_name
echo "${return_ref}" > .git/ref

metadata=$(git_metadata)

# Store original cloned branch in .git/branch
echo "${metadata}" | jq -r '.[] | select(.name == "branch") | .value' > .git/branch
echo "${metadata}" | jq '.' > .git/metadata.json

# Store short ref with templating. Useful to build Docker images with
# a custom tag
echo "${return_ref}" | cut -c1-7 | awk "{ printf \"${short_ref_format}\", \$1 }" > .git/short_ref

# Write individual metadata fields to separate files

# .git/commit - full SHA hash
echo "${metadata}" | jq -r '.[] | select(.name == "commit") | .value' > .git/commit
# .git/author - commit author name
echo "${metadata}" | jq -r '.[] | select(.name == "author") | .value' > .git/author
# .git/author_date - timestamp when the author originally created the commit
echo "${metadata}" | jq -r '.[] | select(.name == "author_date") | .value' > .git/author_date

# .git/tags - branch name(s) containing this commit (comma-separated if multiple)
echo "${metadata}" | jq -r '.[] | select(.name == "branch") | .value // ""' > .git/branch
# .git/tags - comma-separated list of tags
echo "${metadata}" | jq -r '.[] | select(.name == "tags") | .value // ""' > .git/tags
# .git/url - web URL to view commit (if applicable)
echo "${metadata}" | jq -r '.[] | select(.name == "url") | .value // ""' > .git/url
# .git/committer_date - timestamp when the commit was created in the repository
echo "${metadata}" | jq -r '.[] | select(.name == "committer_date") | .value // ""' > .git/committer_date

# Store commit message in .git/commit_message. Can be used to inform about
# the content of a successfull build.
# the content of a successful build.
# Using https://github.com/cloudfoundry-community/slack-notification-resource
# for example
git log -1 --format=format:%B > .git/commit_message
Expand Down
142 changes: 141 additions & 1 deletion test/get.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,6 @@ it_omits_empty_branch_in_metadata() {
"
}


it_returns_branch_in_metadata() {
local repo=$(init_repo)
local ref1=$(make_commit_to_branch $repo branch-a)
Expand Down Expand Up @@ -176,6 +175,146 @@ it_returns_list_of_tags_in_metadata() {
"
}

it_writes_complete_metadata_files() {
local repo=$(init_repo)

# Create a commit with specific author/committer info
git -C $repo \
-c user.name='alice' \
-c user.email='[email protected]' \
commit --allow-empty -m "test commit message"
local ref=$(git -C $repo rev-parse HEAD)

# Add multiple tags to test tag aggregation
git -C $repo tag v1.0.0
git -C $repo tag v1.0.0-rc1
git -C $repo tag latest

local dest=$TMPDIR/destination
get_uri $repo $dest | jq -e "
.version == {ref: $(echo $ref | jq -R .)}
"

# === Verify metadata.json ===
test -e $dest/.git/metadata.json || \
( echo ".git/metadata.json does not exist"; return 1 )

cat $dest/.git/metadata.json | jq -e '. | length > 0' > /dev/null || \
( echo ".git/metadata.json is not valid JSON or is empty"; return 1 )

# === Verify all metadata files ===

# .git/commit
test -e $dest/.git/commit || \
( echo ".git/commit does not exist"; return 1 )
test "$(cat $dest/.git/commit)" = "$ref" || \
( echo ".git/commit content mismatch"; return 1 )

# .git/author
test -e $dest/.git/author || \
( echo ".git/author does not exist"; return 1 )
test "$(cat $dest/.git/author)" = "alice" || \
( echo ".git/author content mismatch"; return 1 )

# .git/author_date
test -e $dest/.git/author_date || \
( echo ".git/author_date does not exist"; return 1 )
echo "$(cat $dest/.git/author_date)" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} [+-][0-9]{4}$" || \
( echo ".git/author_date format invalid: $(cat $dest/.git/author_date)"; return 1 )

# .git/committer
test -e $dest/.git/committer || \
( echo ".git/committer does not exist"; return 1 )
test "$(cat $dest/.git/committer)" = "[email protected]" || \
( echo ".git/committer content mismatch"; return 1 )

# .git/committer_name
test -e $dest/.git/committer_name || \
( echo ".git/committer_name does not exist"; return 1 )
test "$(cat $dest/.git/committer_name)" = "alice" || \
( echo ".git/committer_name content mismatch"; return 1 )

# .git/committer_date
test -e $dest/.git/committer_date || \
( echo ".git/committer_date does not exist"; return 1 )
echo "$(cat $dest/.git/committer_date)" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} [+-][0-9]{4}$" || \
( echo ".git/committer_date format invalid"; return 1 )

# .git/ref
test -e $dest/.git/ref || \
( echo ".git/ref does not exist"; return 1 )
test "$(cat $dest/.git/ref)" = "$ref" || \
( echo ".git/ref content mismatch"; return 1 )

# .git/branch
test -e $dest/.git/branch || \
( echo ".git/branch does not exist"; return 1 )
test "$(cat $dest/.git/branch)" = "master" || \
( echo ".git/branch content mismatch"; return 1 )

# .git/short_ref
test -e $dest/.git/short_ref || \
( echo ".git/short_ref does not exist"; return 1 )
local expected_short="test-$(echo $ref | cut -c1-7)"
test "$(cat $dest/.git/short_ref)" = "$expected_short" || \
( echo ".git/short_ref content mismatch"; return 1 )

# .git/commit_message
test -e $dest/.git/commit_message || \
( echo ".git/commit_message does not exist"; return 1 )
grep -q "test commit message" $dest/.git/commit_message || \
( echo ".git/commit_message content mismatch"; return 1 )

# .git/commit_timestamp
test -e $dest/.git/commit_timestamp || \
( echo ".git/commit_timestamp does not exist"; return 1 )
test -n "$(cat $dest/.git/commit_timestamp)" || \
( echo ".git/commit_timestamp is empty"; return 1 )

# .git/describe_ref
test -e $dest/.git/describe_ref || \
( echo ".git/describe_ref does not exist"; return 1 )

# .git/tags
test -e $dest/.git/tags || \
( echo ".git/tags does not exist"; return 1 )
local tags=$(cat $dest/.git/tags)
test -n "$tags" || \
( echo ".git/tags is empty when tags exist"; return 1 )
echo "$tags" | grep -q "v1.0.0" || \
( echo ".git/tags missing v1.0.0"; return 1 )
echo "$tags" | grep -q "v1.0.0-rc1" || \
( echo ".git/tags missing v1.0.0-rc1"; return 1 )
echo "$tags" | grep -q "latest" || \
( echo ".git/tags missing latest"; return 1 )

# .git/url
test -e $dest/.git/url || \
( echo ".git/url does not exist"; return 1 )
test -z "$(cat $dest/.git/url)" || \
( echo ".git/url should be empty for local repo"; return 1 )

# === Test edge case: no tags ===
rm -rf $dest
local repo_notags=$(init_repo)
local ref_notags=$(make_commit $repo_notags)
get_uri $repo_notags $dest

test -e $dest/.git/tags || \
( echo ".git/tags does not exist when no tags present"; return 1 )
test -z "$(cat $dest/.git/tags)" || \
( echo ".git/tags should be empty when no tags exist"; return 1 )

# === Verify metadata.json structure ===
local has_commit=$(cat $dest/.git/metadata.json | jq '[.[] | select(.name == "commit")] | length')
test "$has_commit" = "1" || \
( echo "metadata.json missing commit field"; return 1 )

local has_author=$(cat $dest/.git/metadata.json | jq '[.[] | select(.name == "author")] | length')
test "$has_author" = "1" || \
( echo "metadata.json missing author field"; return 1 )
}

it_can_use_submodules_without_perl_warning() {
local repo=$(init_repo_with_submodule | cut -d "," -f1)
local dest=$TMPDIR/destination
Expand Down Expand Up @@ -951,6 +1090,7 @@ run it_omits_empty_branch_in_metadata
run it_returns_branch_in_metadata
run it_omits_empty_tags_in_metadata
run it_returns_list_of_tags_in_metadata
run it_writes_complete_metadata_files
run it_honors_the_depth_flag
run it_can_get_from_url_at_depth_at_ref
run it_falls_back_to_deep_clone_if_ref_not_found
Expand Down