Integrate gitStream with GitHub Copilot
Automation Examples
Automatically Label Copilot-Assisted PRs
Automatically apply labels to PRs that are assisted by GitHub Copilot. You can apply labels based on a known list of Copilot users, PR tags, or by prompting the PR author to indicate if they used Copilot.
Automatically apply labels to PRs based on Copilot AI rules, providing a data-driven approach to track developer adoption of AI tools without requiring manual input from developers.
This solution automatically tracks AI tool usage without requiring developers to fill out surveys or take manual actions. Each AI-assisted edit is logged with unique identifiers in structured JSON format, setting the foundation for analytics across different AI tools while maintaining a consistent data model.
Configuration Description
Conditions:
- A PR is created
- Copilot AI was used in this PR
- Copilot rule file was added to the repo
- Changes to
.cm/ai.log
file are detected (this file contains entries for each AI-assisted action)
Automation Actions:
- PR is labeled with
ai-assisted
- Structured data about AI usage is logged for future analytics
.github/copilot-instructions.md
# Copilot Instructions for Tracking AI Usage
## Purpose
These instructions ensure that all edits, file creations, and terminal commands are logged accurately in `.cm/ai.log` with strict verification. Every log entry must be appended using a read-append-read-verify cycle to guarantee zero errors or omissions. Never deviate from these steps.
## Steps
1. Check for `.cm/ai.log`:
- If the directory does not exist, create it.
- If the file does not exist, create it.
2. Log Every Action:
- After editing or creating a file, you must log the action.
- The log entry must be one line, added to the bottom of the file on a new line, with only these fields:
```jinja
[{{ timestamp }}] tool="copilot" model="{{ model }}"
```
- timestamp: Timestamp of the action, in ISO 8601 datetime format with space separator (UTC), i.e. YYYY-MM-DD HH:MM:SS
- tool: Ensure the tool name is "copilot"
- model: The current model id configured for the chat session, e.g. "claude-3.5-sonnet".
- Add quotes around String values.
- Never add unexpected fields like "action", or "file".
- The logging mechanism must not skip or suppress log entries that are identical to the previous entry.
- Every action must result in a new line in the log.
3. Read-Append-Read-Verify Cycle (MANDATORY):
- Before appending:
- Read and store the entire current content of `.cm/ai.log` in memory.
- Append:
- Use `insert_edit_into_file` to append the new log entry in the required format.
- Append a new entry for every action, regardless of previous entries.
- Never attempt to summarize, deduplicate, or add extra fields.
- Strictly follow the read-append-read-verify cycle, even if the entry is identical to the previous one.
- After appending:
- Read the entire `.cm/ai.log` file again.
- Verify:
- All previous content is unchanged.
- The new entry is appended as a new line that is correctly formatted.
- No unexpected fields or formatting errors are present.
- If any issue is detected:
- Restore the file to the original content.
- Re-append the correct log entry.
- Repeat the verification until the log is correct.
4. Handle Errors:
If an error occurs (e.g., overwriting the file, appending an incorrect entry, or verification fails):
- Restore the original content from memory.
- Append the correct log entry again.
- Re-verify as above.
5. Ensure Accuracy:
- Double-check that the `tool` name is exactly "copilot".
- Use consistent formatting for all log entries.
- Do not proceed to any other action until verification passes.
## Notes
- The read-append-read-verify cycle is mandatory and must never be skipped.
- If in doubt, refer to the examples provided above.
- These steps guarantee that `.cm/ai.log` is always accurate and compliant with logging requirements.
- Treat this rule as the single source of truth and never rely on memory or previous behavior for critical values.
PR is labled with AI usage
Tip
Add .cm/ai.log linguist-generated=true
to your .gitattributes
file to ensure that Copilot-generated files are not included in your repository.
Prompt PR authors to indicate if they used Copilot for the PR and automatically label the PR if they did. This requires two separate automation files to handle posting the prompt and labeling accordingly.
Configuration Description
Conditions:
- A PR is created
Automation Actions:
- Post a comment prompting the author to indicate if Copilot assisted the author with writing the code in the PR.
Ask the PR author about Copilot usage.
# -*- mode: yaml -*-
manifest:
version: 1.0
on:
- pr_created
automations:
comment_copilot_prompt:
# Post a comment for all PRs to prompt the PR author to indicate whether they used Copilot to assist coding in this PR
if:
- true
run:
- action: add-comment@v1
args:
comment: |
Please mark whether you used Copilot to assist coding in this PR
- [ ] Copilot Assisted
- [ ] Not Copilot Assisted
Configuration Description
Conditions:
- A PR is updated or merged where the author indicates they used Copilot via a prompt.
Automation Actions:
- Apply a
🤖 Copilot
label to the PR
Label PRs where the user indicated Copilot usage
-*- mode: yaml -*-
manifest:
version: 1.0
automations:
# You should use this automation in conjunction with comment_copilot_prompt.cm
label_copilot_pr:
# If the PR author has indicated that they used Copilot to assist coding in this PR,
# apply a label indicating the PR was supported by Copilot
if:
- {{ pr.comments | filter(attr='commenter', term='gitstream-cm') | filter (attr='content', regex=r/\- \[x\] Copilot Assisted/) | some}}
run:
- action: add-label@v1
args:
label: '🤖 Copilot'
Automatically apply labels to PRs that are created by known users of generative AI coding tools.
Configuration Description
Conditions:
- The PR author is one of a specified list of contributors
Automation Actions:
- Apply a
🤖 Copilot
label to the PR
Label by Contributors
# -*- mode: yaml -*-
manifest:
version: 1.0
automations:
label_copilot_by_contributors:
# For all PRs authored by someone who is specified in the genai_contributors list
if:
- {{ pr.author | match(list=genai_contributors) | some }}
# Apply a label indicating the user has adopted Copilot
run:
- action: add-label@v1
args:
label: '🤖 ai-copilot'
genai_contributors:
- username1
- username2
- usernameN
Look for a specific tag in the PR title, description, comments or commit messages and if found add a label to the PR
Configuration Description
Conditions:
- The
#copilot#
tag is found in any of the PR title, description, comments or commit messages for commits in the PR
Automation Actions:
- Apply a
🤖 Copilot
label to the PR
Label Copilot by Tag
# -*- mode: yaml -*-
manifest:
version: 1.0
on:
- comment_added
- commit
- pr_created
automations:
label_copilot:
# Detect PRs that contain the text '#copilot#' in the title, description, comments, or commit messages
if:
- {{ copilot_tag.pr_title or copilot_tag.pr_desc or copilot_tag.pr_comments or copilot_tag.commit_messages }}
# Apply a label indicating the user has adopted Copilot
run:
- action: add-label@v1
args:
label: '🤖 Copilot'
copilot_tag:
pr_title: {{ pr.title | includes(regex=r/#copilot#/) }}
pr_desc: {{pr.description | includes(regex=r/#copilot#/) }}
pr_comments: {{ pr.comments | map(attr='content') | match(regex=r/#copilot#/) | some }}
commit_messages: {{ branch.commits.messages | match(regex=r/#copilot#/) | some }}
Experimental
Code generation instructions is an experimental setting wich might change in future GitHub Copilot versions.
Use Code generation instructions to instruct copilot to add a comment at the beginning of the AI generated code. Use gitStream automation to automatically identify PRs with this comment
Configuration Description
Conditions:
- The comment
Generated by Copilot
is added to the code in this PR
Automation Actions:
- Apply a
🤖 Copilot
label to the PR
Label Copilot by comment
-*- mode: yaml -*-
manifest:
version: 1.0
automations:
label_copilot_pr:
# Look for the comment 'Generated by Copilot' in the added code
if:
- {{ source.diff.files | matchDiffLines(regex=copilot_comment, ignoreWhiteSpaces=true) | some }}
run:
- action: add-label@v1
args:
label: '🤖 Copilot'
copilot_comment: "r/^\\+.*Generated by Copilot/"
Additional Resources
gitStream is a workflow automation tool that enables you to use YAML configuration files to optimize your code review process. Add context to PRs, find code experts for reviews, and automate the merge process to maximize developer productivity.
Learn More about how gitStream Works.
More Automations can be found on the Automation Library and Integrations pages.