Filter functions
Filters can change the look and format of the source data, or even generate new data derived from the input values. What's important is that the original data is replaced by the result of transformations, and that's what ends up in rendered templates.
Note
Items marked with are under development and are not available yet.
Overview
The following functions are supported in addition to the built-in functions provided by Nunjucks.
Low level functions
Function | Input | Args | Output |
---|---|---|---|
every Checks whether all element in the list are true | [Bool] | - | Bool |
filter Reduce list of items into a list of same items that match the specified term | [String] [Object] | regex , term , list , attr | [String] [Object] |
includes Check if substring match | String | regex , term , list | Bool |
map Maps each object in a list into their specified attribute value | [Object] | attr | [Object] |
match Maps list of items into a list of booleans that match the specified term | [String] [Object] | regex , term , list attr | [Bool] |
nope Checks whether all element in the list are false | [Bool] | - | Bool |
reject Inverse of filter , the result list contains non-matching items | [String] [Object] | regex , term , list , attr | [String] [Object] |
some Checks whether at least one element in the list is true | [Bool] | - | Bool |
High level functions
Function | Input | Args | Output |
---|---|---|---|
allDocs Checks the list includes only documents | files | - | Bool |
allImages Checks the list includes only images | files | - | Bool |
allTests Checks the list includes only tests | files | - | Bool |
codeExperts Get list of contributors based on expert reviewer model results | repo | gt , lt | [String] |
estimatedReviewTime Estimated review time in minutes | branch | - | Integer |
extensions Lists all the unique file extensions | [String] | - | [String] |
explainCodeExperts Short markdown text explaining codeExperts results | repo | gt , lt | [String] |
explainRankByGitBlame Short markdown text explaining rankByGitBlame results | repo | gt , lt | [String] |
isFirstCommit Checks if its the author first commit in the repo | repo.contributors | String | Bool |
isFormattingChange Checks that only formatting changed | [FileDiff ] | - | Bool |
matchDiffLines Match every line in diff | [FileDiff ] | regex , ignoreWhiteSpaces | [Bool] |
rankByGitActivity Get list of contributors based on git-commit activity | repo | gt , lt | [String] |
rankByGitBlame Get list of contributors based on git-blame results | repo | gt , lt | [String] |
Named arguments
Some functions supports named arguments, many of these repeat in different functions.
term
- a single string, used as substring to match with the matched item.
list
- a list of strings, trying to match any of the listed substrings with the matched item.
regex
- a single string, used as regular expression to with the matched item. A regular expression can be created just like JavaScript, but needs to be prefixed with r, for example r/^foo.*/g
, for more info see Nunjucks.
globs
- a key to an element in the .cm
that holds a list of strings, used as glob pattern test on the matched item. For more info, see Wikipedia.
attr
- a key in the element to use when doing the requested operation.
For example, the following expressions provide an identical result:
- {{ 'something' | includes(regex=r/^some.*/) }}
- {{ 'something' | includes(term='some') }}
- {{ 'something' | includes(list=['some']) }}
Reference
every
Checks whether all element in the list are true
. In case the list of elements is empty, it will return false
.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [Bool] | List of booleans |
- | Output | Bool | Returns true when all list items are true |
For example, check that all changes are in either 'src' or 'dest' directories:
filter
Creates a shallow copy of a portion of a given list, filtered down to just the elements that match the given term. You can use either a single term, regex, or a list of terms to match with.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [String] [Object] | The list of strings to match, or list of objects if attr is used |
term regex list | Input (either) | String String [String] | Search term to match with the input items |
attr | Input (optional) | String | match a named attribute in the input object |
- | Output | [String] [Object] | The list with only the matching items |
For example, check if all changes to JavaScript files are in tests directory:
For example, check if all changes to JavaScript files are formatting:
includes
Determines whether a string includes a certain substring. You can use either a single term, regex, or a list of terms to match with.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | String | The list of strings to match |
term regex list | Input (either) | String String [String] | Substring term to match |
- | Output | Bool | true if search terms matches |
Check string matches either of the terms:
map
Creates a new list populated with the values of the selected attribute of every element in the input list.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [Object] | The list of objects to map, see context for valid inputs |
attr | Input | String | Object attribute to select |
- | Output | [Object] | List of the selected object attributes |
For example, the source.diff.files
context holds a list of FileDiff
, each has new_file
attribute. You can create a list of all the new file names by mapping to the new_file
attribute and then check if there are changes to any handler.js
file:
match
Return true
for each element in the list that match the search term.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [String] [Object] | The list of strings or if attr used the list of objects |
term regex list | Input (either) | String String [String] | Search term to match |
attr | Input | String | match a named attribute in the input object |
- | Output | [Bool] | true for every matching item |
For example, to check if all code changes are in the tests
directory:
For example, to check if there are code changes with specific function call:
nope
The inverse of every
, checks whether all element in the list are false
. In case the list of elements is empty, it will return false
.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [Bool] | List of booleans |
- | Output | Bool | Returns true when all list items are false |
For example, check that no changes in either 'src' or 'dest' directories:
reject
Creates a shallow copy of a portion of a given list, filtered down to just the elements that does not match the given term. You can use either a single term, regex, or a list of terms to match with.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [String] [Object] | The list of strings to match, or list of objects if attr is used |
term regex list | Input (either) | String String [String] | Search term to match with the input items |
attr | Input (optional) | String | match a named attribute in the input object |
- | Output | [String] [Object] | The list with only the non-matching items |
For example, check if all changes, but JavaScript files are in tests directory:
For example, check if all changes except for config.json
files are formatting:
some
Checks whether any element in the list is true
. In case the list of elements is empty it will return false
.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [Bool] | List of booleans |
- | Output | Bool | Returns true when any of the items is true |
allDocs
Return true
if the input list includes only documents based on file extensions.
Doc files extensions are: md
, mkdown
, txt
, rst
, except for requirements.txt
.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | files | The list of changed files with their path |
- | Output | Bool | true if all file extensions are of docs |
In case you want to exclude more files, like all txt
under requirements
directory, add another check:
allImages
Return true
if the input list includes only images based on file extensions.
Image file extensions are: svg
, png
, gif
.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | files | The list of changed files with their path |
- | Output | Bool | true if all file extensions are of images |
allTests
Return true
if the input list includes only tests based on file's path and name.
To identify as test the file must include the substring test
or spec
in its name or path.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | files | The list of changed files with their path |
- | Output | Bool | true if all file tests based on name and path |
codeExperts
When requesting a review for a pull request, it's important to select a reviewer who has a deep understanding of the relevant code area, the domain problem, and the framework being used. This ensures that the reviewer can provide specific and informed feedback, rather than general comments that may not take into account the context in which the issue was solved.
The filter provides the list of most qualified contributors based on git-blame
and git-commit
results to determine who has been most active in the relevant code area, and then combines this information into a score between 0 and 100. The commit activity is scored higher for recent commits, which ensures that those who are actively contributing to the codebase are given higher priority as potential reviewers. The result will be limited to 2 users and shall not include the PR author.
The output lists the Git provider users, e.g., GitHub users, which are mapped from the Git users included in the git-blame
output. When gitStream cannot map the Git user to a Git provider user it will be dropped from the output list, hence the list may contain less than 100% of the lines.
Note
The codeExperts
filter function calls gitStream app API with the repo
context to calculate the estimated review time value.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | repo | The repo context variable |
lt | Input | Integer | Filter the user list, keeping those below the specified threshold |
gt | Input | Integer | Filter the user list, keeping those above the specified threshold |
- | Output | [String] | Up to 2 users, sorted by best match first (it won't include the PR author) |
For example:
automations:
code_experts:
if:
- true
run:
- action: add-reviewers@v1
args:
reviewers: {{ repo | codeExperts(gt=10) }}
estimatedReviewTime
Returns the estimated review time in minutes based on statistical model. The model uses the amount of additions and deletions statistics for each file type with additional information about the commits and base branch.
Note
The estimatedReviewTime
filter function calls gitStream app API with the branch
context to calculate the estimated review time value.
The following files are excluded when calculating this value:
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | branch | Branch meta data |
- | Output | Integer | the estimated time for review in minutes |
The following files are automatically excluded from the estimated review time calculation.
File type | Filter type | Values |
---|---|---|
Data | Extension | ini csv xls xlsx xlr doc docx txt pps ppt pptx dot dotx log tar rtf dat ipynb po profile object obj dxf twb bcsymbolmap tfstate pdf rbi pem crt svg png jpeg jpg ttf |
Data | Regex | .*dist/.*\.js$ .*public/assets/.*\.js$ |
Lock | Regex | .*package-lock|packages\.lock|package)\.json$ |
Lock | File | yarn.lock gemfile.lock podfile.lock cargo.lock composer.lock pipfile.lock gopkg.lock |
Lock | Regex | .*gradle\.lockfile$ .*lock\.sbt$ |
Pipeline | Regex | .*ci\.yml$ |
Tip
You can also filter more files, using config.ignore_files
.
extensions
Expects files
and provide a list of all unique file extensions.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | files | The list of changed files with their path |
- | Output | [String] | List of all unique file extensions |
For example, check that only one file type was changed:
explainCodeExperts
This filter helps to explain the results of codeExperts
, the output is in Markdown format that can be used in a PR comment.
Note
The explainCodeExperts
filter function calls gitStream app API with the repo
context to calculate the estimated review time value.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | repo | The repo context variable |
lt | Input | Integer | Filter the user list, keeping those below the specified threshold |
gt | Input | Integer | Filter the user list, keeping those above the specified threshold |
- | Output | String | Explaining codeExperts results in markdown format |
For example:
automations:
code_experts:
if:
- true
run:
- action: add-reviewers@v1
args:
reviewers: {{ repo | codeExperts(gt=10) }}
- action: add-comment@v1
args:
comment: |
{{ repo | explainCodeExperts(gt=10) }}
Note the comment starts with |
and a new-line
as explainCodeExperts
generates a multiline comment.
explainRankByGitBlame
This filter helps to explain the results of rankByGitBlame
, the output is in Markdown format that can be used in a PR comment.
The output lists the Git provider users, e.g., GitHub users, which are mapped from the Git users included in the git-blame
output. Git users that could not be automatically mapped are marked with *
. To map these users, you can add user_mapping
see instructions here.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | repo | The repo context variable |
lt | Input | Integer | Filter the user list, keeping those below the specified threshold |
gt | Input | Integer | Filter the user list, keeping those above the specified threshold |
- | Output | String | Explaining rankByGitBlame results in markdown format |
Note
Each contributor's result is rounded down to the nearest integer, so the total may add up to less than 100%.
For example:
automations:
the_right_reviewer:
if:
- true
run:
- action: add-reviewers@v1
args:
reviewers: {{ repo | rankByGitBlame(gt=50) }}
- action: add-comment@v1
args:
comment: |
{{ repo | explainRankByGitBlame(gt=50) }}
Note the comment starts with |
and a new-line
as explainRankByGitBlame
generates a multiline comment.
isFirstCommit
Return true
if it's the author first commit in the repo.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | repo.contributors | List of contributors in the repo |
- | Input | String | The contributor name |
- | Output | Bool | true if its the first commit of the selected contributor |
if:
- {{ repo.contributors | isFirstCommit(branch.author) }}
run:
- action: add-comment@v1
args:
comment: Welcome {{branch.author}}!
isFormattingChange
Return true
if all file diffs are validated as formatting changes.
Support source code languages: JavaScript, TypeScript, JSON, YAML and HTML.
If changes in other formats detected, the filter will return false
.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | source.diff.files | List of file diffs |
- | Output | Bool | true if the all code changes are non functional |
matchDiffLines
Checks diff for matching lines.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | [Object] | The list of objects |
regex | Input | String | Regex term to match with the input items, use \\ for \ |
ignoreWhiteSpaces | Input | Bool | false by default, match a named attribute in the input object |
caseSensitive | Input | Bool | true by default, ignore case when matching terms |
- | Output | [Bool] | true for every matching object |
For example, to check if all the changes are of adding prints and ignore white spaces:
{{ source.diff.files | matchDiffLines(regex=r/^\+.*console\.log/, ignoreWhiteSpaces=true) | every }}
rankByGitActivity
Get list of contributors based on git-commit
activity.
The repo
context includes all the changed files, for each file it includes each contributor number of lines changed every week over the last 52 weeks, based on git-commit
.
These functions compare each contributor changes per week and yield an average percentage of contribution for any given file. For example, in a certain week a file had 500 line changed, 200 by a first user, while 3 other users changed 100 lines each. So the score for the first user in that week will be 40 (200/500 in %). The function then average the score for each user for the selected time period.
Then you can use the thresholds to get the right reviewer.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | repo | The repo context variable |
weeks | Input | Integer | The number of last weeks to include |
lt | Input | Integer | Filter the user list, keeping those below the specified threshold |
gt | Input | Integer | Filter the user list, keeping those above the specified threshold |
- | Output | [String] | The list of users based on their code score comparison |
Check if the branch author is a rookie
rankByGitBlame
Get list of contributors based on git-blame
results
The repo
context includes all the changed files, for each file it includes the contributors' percentage of lines in the file, based on git-blame
.
This function sums all these percentages per user and yield an average percentage of contribution. Then you can use the thresholds to get the right reviewer.
The output lists the Git provider users, e.g., GitHub users, which are mapped from the Git users included in the git-blame
output. When gitStream cannot map the Git user to a Git provider user it will be dropped from the output list, hence the list may contain less than 100% of the lines.
Argument | Usage | Type | Description |
---|---|---|---|
- | Input | repo | The repo context variable |
lt | Input | Integer | Filter the user list, keeping those below the specified threshold |
gt | Input | Integer | Filter the user list, keeping those above the specified threshold |
- | Output | [String] | The list of users based on their code score comparison, sorted by rank - first has highest score |
Example of the filter output, note the output are GitHub users in the example:
Get the most significant contributors for the PR files:
Check if the branch author is a rookie