Compare commits

..

20 Commits
v2.0.3 ... v2

Author SHA1 Message Date
Colt
9f79830454 Fix: builds for Unity versions less than 2019 fail (#467) 2022-10-15 20:52:15 +02:00
Paul Pacheco
f054aaa35d use node 16 (#465)
node12 actions are deprecated.
See documentation here:
https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/

Currently, the unity builder action gives me the following warning:
```
Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: game-ci/unity-test-runner, game-ci/unity-builder
```

Co-authored-by: Webber Takken <webber@takken.io>
2022-10-13 23:35:23 +02:00
Paul Pacheco
0756841374 update actions core to 1.10.0 (#466)
* update actions core to 1.10.0

The current version of actions/core produces a lot of warning about deprecated set-output command,  like this:

```
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
```

the new version of actions/core uses the environment files instead, so this version bump should fix the warning

* reran yarn
2022-10-13 23:18:30 +02:00
dependabot[bot]
579daa93a6 Bump jose from 1.28.1 to 1.28.2 (#435)
Bumps [jose](https://github.com/panva/jose) from 1.28.1 to 1.28.2.
- [Release notes](https://github.com/panva/jose/releases)
- [Changelog](https://github.com/panva/jose/blob/v1.28.2/CHANGELOG.md)
- [Commits](https://github.com/panva/jose/compare/v1.28.1...v1.28.2)

---
updated-dependencies:
- dependency-name: jose
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Webber Takken <webber@takken.io>
2022-09-17 16:46:22 +02:00
Shane Engelman
1adfabf512 Pass Android parameters for Windows builds, fixes #434 (#436)
https://github.com/game-ci/unity-builder/issues/434 will show that no Windows builds are currently capable of sending `androidVersionCode` or any other Android parameters. This can be fixed very simply by adjusting the build script to pass these parameters. Compare the following two files:

- cdee7d1d9a/dist/platforms/windows/build.ps1 (L112)
- cdee7d1d9a/dist/platforms/ubuntu/steps/build.sh (L138)

It's clear that the current Windows build script is omitting certain parameters. This will resolve that issue (linked in commit message header).
2022-09-16 15:12:28 -05:00
Webber Takken
cdee7d1d9a Update FUNDING.yml (#431) 2022-09-06 16:54:29 +02:00
Webber Takken
fb633660a4 chore: add game-ci github sponsoring option (#429) 2022-08-29 23:49:54 +02:00
dependabot[bot]
c15f955669 Bump @actions/core from 1.6.0 to 1.9.1 (#426)
Bumps [@actions/core](https://github.com/actions/toolkit/tree/HEAD/packages/core) from 1.6.0 to 1.9.1.
- [Release notes](https://github.com/actions/toolkit/releases)
- [Changelog](https://github.com/actions/toolkit/blob/main/packages/core/RELEASES.md)
- [Commits](https://github.com/actions/toolkit/commits/HEAD/packages/core)

---
updated-dependencies:
- dependency-name: "@actions/core"
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-19 23:28:41 +02:00
David Finol
2684be2dee Fix androidTargetSdkVersion (#425)
* Fix androidTargetSdkVersion

Fix for newer versions of Android API levels that do not get correctly parsed.

* Update dist/default-build-script/Assets/Editor/UnityBuilderAction/Input/AndroidSettings.cs
2022-08-11 07:35:01 -05:00
Webber Takken
db2d8b6dbd fix: make v character in version tags optional (#423)
* fix: make v character in version tags optional

* fix: cross platform regex

* fix: test regex within grep.

* fix: add semantic tags prepended with v
2022-08-04 02:09:32 +02:00
Webber Takken
c068855899 chore: remove audit pre push (#422) 2022-08-03 00:57:22 +02:00
Webber Takken
5b8bf1c8b7 chore: update action-cats (#421) 2022-08-03 00:56:52 +02:00
David Finol
47b25cf3b1 Fixes #409 (#412) 2022-06-01 12:11:32 +02:00
David Finol
36891ec921 Fix missing Android custom keystore (#400) 2022-05-20 18:40:23 -05:00
David Finol
2652cb78a7 Fix mac return license (#397) 2022-05-08 06:18:37 -05:00
Frostebite
f77696efae Cloud runner develop v0.1 (#395)
* Correct aws logs link

* Correct aws logs link

* better aws cli commands and better cleanup for aws

* better aws cli commands and better cleanup for aws

* improved garbage collection cli options

* Only allow ephemeral runners when using cloud runner integration tests flag to avoid unexpected hangup

* Only allow ephemeral runners when using cloud runner integration tests flag to avoid unexpected hangup

* fix issue #393

* Extract follow log stream service

* consolidate into one pipeline file

* consolidate into one pipeline file
2022-05-05 00:25:17 +01:00
Khalid.Sani
4556fc4ff1 fix: git private token not being set correctly for https (#394) 2022-05-02 17:04:05 +02:00
David Finol
d066039c26 Fix for 2021.3 (#392)
As a workaround for an error in 2021.3, write logs to file, and display the logs later: https://forum.unity.com/threads/exception-occurred-inside-beedriver.1139905/
2022-04-27 20:02:18 -05:00
Frostebite
8abce48a48 Cloud runner v0.2 - continued quality of life improvements (#387)
* Update cloud-runner-aws-pipeline.yml

* Update cloud-runner-k8s-pipeline.yml

* yarn build

* yarn build

* correct branch ref

* correct branch ref passed to target repo

* Create k8s-tests.yml

* Delete k8s-tests.yml

* correct branch ref passed to target repo

* correct branch ref passed to target repo

* Always describe AWS tasks for now, because unstable error handling

* Remove unused tree commands

* Use lfs guid sum

* Simple override cache push

* Simple override cache push and pull override to allow pure cloud storage driven caching

* Removal of early branch (breaks lfs caching)

* Remove unused tree commands

* Update action.yml

* Update action.yml

* Support cache and input override commands as input + full support custom hooks

* Increase k8s timeout

* replace filename being appended for unknclear reason

* cache key should not contain whitespaces

* Always try and deploy rook for k8s

* Apply k8s files for rook

* Update action.yml

* Apply k8s files for rook

* Apply k8s files for rook

* cache test and action description for kuber storage class

* Correct test and implement dependency health check and start

* GCP-secret run, cache key

* lfs smudge set explicit and undo explicit

* Run using external secret provider to speed up input

* Update cloud-runner-aws-pipeline.yml

* Add nodejs as build step dependency

* Add nodejs as build step dependency

* Cloud Runner Tests must be specified to capture logs from cloud runner for tests

* Cloud Runner Tests must be specified to capture logs from cloud runner for tests

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* better defaults for new inputs

* better defaults

* merge latest

* force build update

* use npm n to update node in unity builder

* use npm n to update node in unity builder

* use npm n to update node in unity builder

* correct new line

* quiet zipping

* quiet zipping

* default secrets for unity username and password

* default secrets for unity username and password

* ls active directory before lfs install

* Get cloud runner secrets from

* Get cloud runner secrets from

* Cleanup setup of default secrets

* Various fixes

* Cleanup setup of default secrets

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* AWS secrets manager support

* less caching logs

* default k8s storage class to pd-standard

* more readable build commands

* Capture aws exit code 1 reliably

* Always replace /head from branch

* k8s default storage class to standard-rwo

* cleanup

* further cleanup input

* further cleanup input

* further cleanup input

* further cleanup input

* further cleanup input

* folder sizes to inspect caching

* dir command for local cloud runner test

* k8s wait for pending because pvc will not create earlier

* prefer k8s standard storage

* handle empty string as cloud runner cluster input

* local-system is now used for cloud runner test implementation AND correctly unset test CLI input

* local-system is now used for cloud runner test implementation AND correctly unset test CLI input

* fix unterminated quote

* fix unterminated quote

* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name

* remove head and heads from branch prefix

* fix reversed caching direction of cache-push

* fixes

* fixes

* fixes

* cachePull cli

* fixes

* fixes

* fixes

* fixes

* fixes

* order cache test to be first

* order cache test to be first

* fixes

* populate cache key instead of using branch

* cleanup cli

* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files

* import cli methods

* import cli files explicitly

* import cli files explicitly

* import cli files explicitly

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* log parameters in cloud runner parameter test

* log parameters in cloud runner parameter test

* log parameters in cloud runner parameter test

* Cloud runner param test before caching because we have a fast local cache test now

* Using custom build path relative to repo root rather than project root

* aws-garbage-collect at end of pipeline

* aws-garbage-collect do not actually delete anything for now - just list

* remove some legacy du commands

* Update cloud-runner-aws-pipeline.yml

* log contents after cache pull and fix some scenarios with duplicate secrets

* log contents after cache pull and fix some scenarios with duplicate secrets

* log contents after cache pull and fix some scenarios with duplicate secrets

* PR comments

* Replace guid with uuid package

* use fileExists lambda instead of stat to check file exists in caching

* build failed results in core error message

* Delete sample.txt

* cloud-runner-system prefix changed to cloud-runner

* Update cloud-runner-aws-pipeline.yml

* remove du from caching, should run manually if interested in size, adds too much runtime to job to include by default

* github ephemeral pipeline support

* github ephemeral pipeline support

* Merge remote-tracking branch 'origin/main' into cloud-runner-develop

# Conflicts:
#	dist/index.js.map
#	src/model/cloud-runner/providers/aws/aws-task-runner.ts
#	src/model/cloud-runner/providers/aws/index.ts

* garbage collection

* garbage collection

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* ephemeral runner pipeline

* ephemeral runner pipeline

* ephemeral runner pipeline

* download runner each time

* download runner each time

* download runner each time

* garbage collect all older than 1d as part of cleanup

* download runner each time

* number container cpu and memory for aws

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* Skip printing size unless cloudRunnerIntegrationTests is true

* transition zip usage in cache to uncompressed tar for speed

* transition zip usage in cache to uncompressed tar for speed

* transition zip usage in cache to uncompressed tar for speed

* transition zip usage in cache to uncompressed tar for speed

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* better aws commands

* better aws commands

* parse number for cloud formation template

* remove container resource defaults from actions yaml

* remove container resource defaults from actions yaml

* skip all input readers when cloud runner is local

* prefer fs/promises

* actually set aws cloud runner step as failure if unity build fails

* default to 3gb of ram - webgl fails on 2
2022-04-22 00:47:45 +01:00
Webber Takken
5ae03dfef6 Streamline code styles (#384)
* feat: streamline code styles

* feat: spacing for comments and return statements

* chore: enforce camelcase

* fix: remove npm lock file

* fix: add integrity test

* fix: remove logfile

* chore: update node in test workflow
2022-04-12 00:43:41 +02:00
84 changed files with 3598 additions and 8816 deletions

View File

@@ -1,10 +1,15 @@
{
"plugins": ["jest", "@typescript-eslint", "prettier", "unicorn"],
"extends": ["plugin:unicorn/recommended", "plugin:github/recommended", "prettier"],
"extends": ["plugin:unicorn/recommended", "plugin:github/recommended", "plugin:prettier/recommended"],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 9,
"sourceType": "module"
"ecmaVersion": 2020,
"sourceType": "module",
"extraFileExtensions": [".mjs"],
"ecmaFeatures": {
"impliedStrict": true
},
"project": "./tsconfig.json"
},
"env": {
"node": true,
@@ -12,9 +17,44 @@
"jest/globals": true
},
"rules": {
// Error out for code formatting errors
"prettier/prettier": "error",
"import/no-extraneous-dependencies": 0,
// Namespaces or sometimes needed
"import/no-namespace": "off",
"no-undef": "off" // TODO: REMOVE THIS LINE WHEN UPDATING ESLINT RULES
// Properly format comments
"spaced-comment": ["error", "always"],
"lines-around-comment": [
"error",
{
"beforeBlockComment": true,
"beforeLineComment": true,
"allowBlockStart": true,
"allowObjectStart": true,
"allowArrayStart": true,
"allowClassStart": true,
"ignorePattern": "pragma|ts-ignore"
}
],
// Mandatory spacing
"padding-line-between-statements": [
"error",
{ "blankLine": "always", "prev": "*", "next": "return" },
{ "blankLine": "always", "prev": "directive", "next": "*" },
{ "blankLine": "any", "prev": "directive", "next": "directive" }
],
// Enforce camelCase
"camelcase": "error",
// Allow forOfStatements
"no-restricted-syntax": ["error", "ForInStatement", "LabeledStatement", "WithStatement"],
// Continue is viable in forOf loops in generators
"no-continue": "off",
// From experience, named exports are almost always desired. I got tired of this rule
"import/prefer-default-export": "off",
// Unused vars are useful to keep method signatures consistent and documented
"@typescript-eslint/no-unused-vars": "off",
// For this project only use kebab-case
"unicorn/filename-case": ["error", { "cases": { "kebabCase": true } }],
// Allow Array.from(set) mitigate TS2569 which would require '--downlevelIteration'
"unicorn/prefer-spread": "off"
}
}

4
.github/FUNDING.yml vendored
View File

@@ -1,8 +1,8 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
github: game-ci
patreon: # Replace with a single Patreon username
open_collective: game-ci
open_collective: # replace with a single OpenCollective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry

View File

@@ -11,6 +11,6 @@ jobs:
name: A cat for your effort!
runs-on: ubuntu-latest
steps:
- uses: ruairidhwm/action-cats@1.0.1
- uses: ruairidhwm/action-cats@1.0.2
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -12,3 +12,26 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}
expire-in: 21 days
cleanupCloudRunner:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
if: github.event.event_type != 'pull_request_target'
with:
lfs: true
- uses: actions/setup-node@v2
with:
node-version: 12.x
- run: yarn
- run: yarn run cli --help
env:
AWS_REGION: eu-west-2
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: eu-west-2
- run: yarn run cli -m aws-list-all
env:
AWS_REGION: eu-west-2
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: eu-west-2

View File

@@ -1,114 +0,0 @@
name: Cloud Runner - AWS Tests
on:
push: { branches: [main, cloud-runner-develop] }
env:
GKE_ZONE: 'us-central1'
GKE_REGION: 'us-central1'
GKE_PROJECT: 'unitykubernetesbuilder'
GKE_CLUSTER: 'unity-builder-cluster'
GCP_LOGGING: true
GCP_PROJECT: unitykubernetesbuilder
AWS_REGION: eu-west-2
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: eu-west-2
AWS_BASE_STACK_NAME: game-ci-github-pipelines
CLOUD_RUNNER_BRANCH: ${{ github.ref }}
CLOUD_RUNNER_TESTS: true
DEBUG: true
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
jobs:
buildForAllPlatforms:
name: AWS Fargate Build
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
projectPath:
- test-project
unityVersion:
# - 2019.2.11f1
- 2019.3.15f1
targetPlatform:
#- StandaloneOSX # Build a macOS standalone (Intel 64-bit).
- StandaloneWindows64 # Build a Windows 64-bit standalone.
- StandaloneLinux64 # Build a Linux 64-bit standalone.
#- iOS # Build an iOS player.
#- Android # Build an Android .apk.
#- WebGL # WebGL.
# - StandaloneWindows # Build a Windows standalone.
# - WSAPlayer # Build an Windows Store Apps player.
# - PS4 # Build a PS4 Standalone.
# - XboxOne # Build a Xbox One Standalone.
# - tvOS # Build to Apple's tvOS platform.
# - Switch # Build a Nintendo Switch player
# steps
steps:
- name: Checkout (default)
uses: actions/checkout@v2
if: github.event.event_type != 'pull_request_target'
with:
lfs: true
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2
- run: yarn
- run: yarn run cli --help
- run: yarn run test "caching"
- run: yarn run test-i-aws
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
PROJECT_PATH: ${{ matrix.projectPath }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TARGET_PLATFORM: ${{ matrix.targetPlatform }}
cloudRunnerTests: true
versioning: None
- uses: ./
id: aws-fargate-unity-build
timeout-minutes: 25
with:
cloudRunnerCluster: aws
versioning: None
projectPath: ${{ matrix.projectPath }}
unityVersion: ${{ matrix.unityVersion }}
targetPlatform: ${{ matrix.targetPlatform }}
githubToken: ${{ secrets.GITHUB_TOKEN }}
postBuildSteps: |
- name: upload
image: amazon/aws-cli
commands: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile default
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile default
aws configure set region $AWS_DEFAULT_REGION --profile default
aws s3 ls
aws s3 ls game-ci-test-storage
ls /data/cache/$CACHE_KEY
ls /data/cache/$CACHE_KEY/build
aws s3 cp /data/cache/$CACHE_KEY/build/build-$BUILD_GUID.zip s3://game-ci-test-storage/$CACHE_KEY/build-$BUILD_GUID.zip
secrets:
- name: awsAccessKeyId
value: ${{ secrets.AWS_ACCESS_KEY_ID }}
- name: awsSecretAccessKey
value: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: awsDefaultRegion
value: eu-west-2
- run: |
aws s3 cp s3://game-ci-test-storage/${{ steps.aws-fargate-unity-build.outputs.CACHE_KEY }}/build-${{ steps.aws-fargate-unity-build.outputs.BUILD_GUID }}.zip build-${{ steps.aws-fargate-unity-build.outputs.BUILD_GUID }}.zip
ls
- run: yarn run cli -m aws-garbage-collect
###########################
# Upload #
###########################
# download from cloud storage
- uses: actions/upload-artifact@v2
with:
name: AWS Build (${{ matrix.targetPlatform }})
path: build-${{ steps.aws-fargate-unity-build.outputs.BUILD_GUID }}.zip
retention-days: 14

View File

@@ -1,7 +1,7 @@
name: Cloud Runner - K8s Tests
name: Cloud Runner
on:
push: { branches: [cloud-runner-develop] }
push: { branches: [cloud-runner-develop, main] }
# push: { branches: [main] }
# pull_request:
# paths-ignore:
@@ -26,6 +26,97 @@ env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
jobs:
awsBuild:
name: AWS Fargate Build
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
projectPath:
- test-project
unityVersion:
# - 2019.2.11f1
- 2019.3.15f1
targetPlatform:
#- StandaloneOSX # Build a macOS standalone (Intel 64-bit).
- StandaloneWindows64 # Build a Windows 64-bit standalone.
- StandaloneLinux64 # Build a Linux 64-bit standalone.
- WebGL # WebGL.
#- iOS # Build an iOS player.
#- Android # Build an Android .apk.
# - StandaloneWindows # Build a Windows standalone.
# - WSAPlayer # Build an Windows Store Apps player.
# - PS4 # Build a PS4 Standalone.
# - XboxOne # Build a Xbox One Standalone.
# - tvOS # Build to Apple's tvOS platform.
# - Switch # Build a Nintendo Switch player
# steps
steps:
- name: Checkout (default)
uses: actions/checkout@v2
if: github.event.event_type != 'pull_request_target'
with:
lfs: true
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2
- run: yarn
- run: yarn run cli --help
- run: yarn run test "caching"
- run: yarn run test-i-aws
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
PROJECT_PATH: ${{ matrix.projectPath }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TARGET_PLATFORM: ${{ matrix.targetPlatform }}
cloudRunnerTests: true
versioning: None
- uses: ./
id: aws-fargate-unity-build
timeout-minutes: 25
with:
cloudRunnerCluster: aws
versioning: None
projectPath: ${{ matrix.projectPath }}
unityVersion: ${{ matrix.unityVersion }}
targetPlatform: ${{ matrix.targetPlatform }}
githubToken: ${{ secrets.GITHUB_TOKEN }}
postBuildSteps: |
- name: upload
image: amazon/aws-cli
commands: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile default
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile default
aws configure set region $AWS_DEFAULT_REGION --profile default
aws s3 ls
aws s3 ls game-ci-test-storage
ls /data/cache/$CACHE_KEY
ls /data/cache/$CACHE_KEY/build
aws s3 cp /data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar s3://game-ci-test-storage/$CACHE_KEY/build-$BUILD_GUID.tar
secrets:
- name: awsAccessKeyId
value: ${{ secrets.AWS_ACCESS_KEY_ID }}
- name: awsSecretAccessKey
value: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: awsDefaultRegion
value: eu-west-2
- run: |
aws s3 cp s3://game-ci-test-storage/${{ steps.aws-fargate-unity-build.outputs.CACHE_KEY }}/build-${{ steps.aws-fargate-unity-build.outputs.BUILD_GUID }}.tar build-${{ steps.aws-fargate-unity-build.outputs.BUILD_GUID }}.tar
ls
- run: yarn run cli -m aws-garbage-collect
###########################
# Upload #
###########################
# download from cloud storage
- uses: actions/upload-artifact@v2
with:
name: AWS Build (${{ matrix.targetPlatform }})
path: build-${{ steps.aws-fargate-unity-build.outputs.BUILD_GUID }}.tar
retention-days: 14
k8sBuilds:
name: K8s (GKE Autopilot) build for ${{ matrix.targetPlatform }} on version ${{ matrix.unityVersion }}
runs-on: ubuntu-latest
@@ -105,8 +196,7 @@ jobs:
aws s3 ls
aws s3 ls game-ci-test-storage
ls /data/cache/$CACHE_KEY
echo "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.zip s3://game-ci-test-storage/$CACHE_KEY/$BUILD_FILE"
aws s3 cp /data/cache/$CACHE_KEY/build/build-$BUILD_GUID.zip s3://game-ci-test-storage/$CACHE_KEY/build-$BUILD_GUID.zip
aws s3 cp /data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar s3://game-ci-test-storage/$CACHE_KEY/build-$BUILD_GUID.tar
secrets:
- name: awsAccessKeyId
value: ${{ secrets.AWS_ACCESS_KEY_ID }}
@@ -115,7 +205,7 @@ jobs:
- name: awsDefaultRegion
value: eu-west-2
- run: |
aws s3 cp s3://game-ci-test-storage/${{ steps.k8s-unity-build.outputs.CACHE_KEY }}/build-${{ steps.k8s-unity-build.outputs.BUILD_GUID }}.zip build-${{ steps.k8s-unity-build.outputs.BUILD_GUID }}.zip
aws s3 cp s3://game-ci-test-storage/${{ steps.k8s-unity-build.outputs.CACHE_KEY }}/build-${{ steps.k8s-unity-build.outputs.BUILD_GUID }}.tar build-${{ steps.k8s-unity-build.outputs.BUILD_GUID }}.tar
ls
###########################
# Upload #
@@ -124,5 +214,5 @@ jobs:
- uses: actions/upload-artifact@v2
with:
name: K8s Build (${{ matrix.targetPlatform }})
path: build-${{ steps.k8s-unity-build.outputs.BUILD_GUID }}.zip
path: build-${{ steps.k8s-unity-build.outputs.BUILD_GUID }}.tar
retention-days: 14

View File

@@ -12,10 +12,10 @@ jobs:
name: Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 12.x
node-version: '16'
- run: yarn
- run: yarn lint
- run: yarn test --coverage

View File

@@ -2,5 +2,6 @@
"semi": true,
"singleQuote": true,
"trailingComma": "all",
"printWidth": 120
"printWidth": 120,
"proseWrap": "always"
}

View File

@@ -4,13 +4,12 @@
#### Code of Conduct
This repository has adopted the Contributor Covenant as it's
Code of Conduct. It is expected that participants adhere to it.
This repository has adopted the Contributor Covenant as it's Code of Conduct. It is expected that participants adhere to
it.
#### Proposing a Change
If you are unsure about whether or not a change is desired,
you can create an issue. This is useful because it creates
If you are unsure about whether or not a change is desired, you can create an issue. This is useful because it creates
the possibility for a discussion that's visible to everyone.
When fixing a bug it is fine to submit a pull request right away.
@@ -34,6 +33,11 @@ Please note that commit hooks will run automatically to perform some tasks;
- run tests
- build distributable files
#### Windows users
Make sure your editor and terminal that run the tests are set to `Powershell 7` or above with
`Git's Unix tools for Windows` installed. Some tests require you to be able to run `sh` and other unix commands.
#### License
By contributing to this repository, you agree that your contributions will be licensed under its MIT license.

View File

@@ -115,11 +115,11 @@ inputs:
required: false
description: 'Either local, k8s or aws can be used to run builds on a remote cluster. Additional parameters must be configured.'
cloudRunnerCpu:
default: '1.0'
default: ''
required: false
description: 'Amount of CPU time to assign the remote build container'
cloudRunnerMemory:
default: '750M'
default: ''
required: false
description: 'Amount of memory to assign the remote build container'
cachePushOverrideCommand:
@@ -175,5 +175,5 @@ branding:
icon: 'box'
color: 'gray-dark'
runs:
using: 'node12'
using: 'node16'
main: 'dist/index.js'

View File

@@ -1,13 +0,0 @@
Cloud Runner platform selected AWS
Cloud Runner is running in custom job mode
AWS Region: eu-west-2
Parsing build steps:
- name: 'step 1'
image: 'alpine'
commands: 'printenv'
secrets:
- name: 'testSecretName'
value: 'testSecretValue'
game-ci stack does not exist (["game-ci-github-automation-424-linux64-a9hz-cleanup","game-ci-github-automation-423-linux64-v34g-cleanup","game-ci-github-automation-423-linux64-v34g","game-ci-github-automation-422-linux64-7x6i-cleanup","game-ci-github-automation-422-linux64-7x6i","game-ci-github-automation-414-linux64-j21p-cleanup","game-ci-github-automation-414-linux64-j21p","game-ci-github-automation-413-linux64-tcih-cleanup","game-ci-github-automation-413-linux64-tcih","game-ci-github-automation-411-linux64-0s69-cleanup","game-ci-github-automation-411-linux64-0s69","game-ci-github-automation-410-linux64-1tli-cleanup","game-ci-github-automation-410-linux64-1tli","game-ci-github-automation-408-linux64-8pbw-cleanup","game-ci-github-automation-408-linux64-8pbw","game-ci-github-automation-407-linux64-21un-cleanup","game-ci-github-automation-407-linux64-21un","game-ci-github-automation-406-linux64-dizb-cleanup","game-ci-github-automation-406-linux64-dizb","game-ci-github-automation-405-linux64-9xj5-cleanup","game-ci-github-automation-405-linux64-9xj5","game-ci-github-automation-402-linux64-0bym-cleanup","game-ci-github-automation-402-linux64-0bym","game-ci-github-automation-400-linux64-arqv-cleanup","game-ci-github-automation-400-linux64-arqv","game-ci-github-automation-399-linux64-utkt-cleanup","game-ci-github-automation-399-linux64-utkt","game-ci-github-automation-397-linux64-xwfu-cleanup","game-ci-github-automation-397-linux64-xwfu","game-ci-github-automation-396-linux64-2g3q-cleanup","game-ci-github-automation-396-linux64-2g3q","game-ci-github-automation","game-ci-stack-integration-tests-390-linux64-mcdw-cleanup","game-ci-stack-integration-tests-390-linux64-mcdw","game-ci-stack-integration-tests-391-linux64-2arq-cleanup","game-ci-stack-integration-tests-391-linux64-2arq","game-ci-stack-integration-tests-390-linux64-awd0-cleanup","game-ci-stack-integration-tests-390-linux64-awd0","game-ci-stack-integration-tests"])
created stack (version: eedce7440581ab2e8a80cee59e34ed64)

View File

@@ -1,416 +0,0 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Fargate cluster that can span public and private subnets. Supports
public facing load balancers, private internal load balancers, and
both internal and external service discovery namespaces.
Parameters:
EnvironmentName:
Type: String
Default: development
Description: "Your deployment environment: DEV, QA , PROD"
Version:
Type: String
Description: "hash of template"
# ContainerPort:
# Type: Number
# Default: 80
# Description: What port number the application inside the docker container is binding to
Mappings:
# Hard values for the subnet masks. These masks define
# the range of internal IP addresses that can be assigned.
# The VPC can have all IP's from 10.0.0.0 to 10.0.255.255
# There are four subnets which cover the ranges:
#
# 10.0.0.0 - 10.0.0.255
# 10.0.1.0 - 10.0.1.255
# 10.0.2.0 - 10.0.2.255
# 10.0.3.0 - 10.0.3.255
SubnetConfig:
VPC:
CIDR: '10.0.0.0/16'
PublicOne:
CIDR: '10.0.0.0/24'
PublicTwo:
CIDR: '10.0.1.0/24'
Resources:
# VPC in which containers will be networked.
# It has two public subnets, and two private subnets.
# We distribute the subnets across the first two available subnets
# for the region, for high availability.
VPC:
Type: AWS::EC2::VPC
Properties:
EnableDnsSupport: true
EnableDnsHostnames: true
CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
EFSServerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "efs-server-endpoints"
GroupDescription: Which client ip addrs are allowed to access EFS server
VpcId: !Ref 'VPC'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 2049
ToPort: 2049
SourceSecurityGroupId: !Ref ContainerSecurityGroup
#CidrIp: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
# A security group for the containers we will run in Fargate.
# Rules are added to this security group based on what ingress you
# add for the cluster.
ContainerSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "task security group"
GroupDescription: Access to the Fargate containers
VpcId: !Ref 'VPC'
# SecurityGroupIngress:
# - IpProtocol: tcp
# FromPort: !Ref ContainerPort
# ToPort: !Ref ContainerPort
# CidrIp: 0.0.0.0/0
SecurityGroupEgress:
- IpProtocol: -1
FromPort: 2049
ToPort: 2049
CidrIp: "0.0.0.0/0"
# Two public subnets, where containers can have public IP addresses
PublicSubnetOne:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select
- 0
- Fn::GetAZs: !Ref 'AWS::Region'
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']
# MapPublicIpOnLaunch: true
PublicSubnetTwo:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select
- 1
- Fn::GetAZs: !Ref 'AWS::Region'
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']
# MapPublicIpOnLaunch: true
# Setup networking resources for the public subnets. Containers
# in the public subnets have public IP addresses and the routing table
# sends network traffic via the internet gateway.
InternetGateway:
Type: AWS::EC2::InternetGateway
GatewayAttachement:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref 'VPC'
InternetGatewayId: !Ref 'InternetGateway'
# Attaching a Internet Gateway to route table makes it public.
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref 'VPC'
PublicRoute:
Type: AWS::EC2::Route
DependsOn: GatewayAttachement
Properties:
RouteTableId: !Ref 'PublicRouteTable'
DestinationCidrBlock: '0.0.0.0/0'
GatewayId: !Ref 'InternetGateway'
# Attaching a public route table makes a subnet public.
PublicSubnetOneRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetOne
RouteTableId: !Ref PublicRouteTable
PublicSubnetTwoRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetTwo
RouteTableId: !Ref PublicRouteTable
# ECS Resources
ECSCluster:
Type: AWS::ECS::Cluster
# A role used to allow AWS Autoscaling to inspect stats and adjust scaleable targets
# on your AWS account
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'application-autoscaling:*'
- 'cloudwatch:DescribeAlarms'
- 'cloudwatch:PutMetricAlarm'
- 'ecs:DescribeServices'
- 'ecs:UpdateService'
Resource: '*'
# This is an IAM role which authorizes ECS to manage resources on your
# account on your behalf, such as updating your load balancer with the
# details of where your containers are, so that traffic can reach your
# containers.
ECSRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
# Rules which allow ECS to attach network interfaces to instances
# on your behalf in order for awsvpc networking mode to work right
- 'ec2:AttachNetworkInterface'
- 'ec2:CreateNetworkInterface'
- 'ec2:CreateNetworkInterfacePermission'
- 'ec2:DeleteNetworkInterface'
- 'ec2:DeleteNetworkInterfacePermission'
- 'ec2:Describe*'
- 'ec2:DetachNetworkInterface'
# Rules which allow ECS to update load balancers on your behalf
# with the information sabout how to send traffic to your containers
- 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
- 'elasticloadbalancing:DeregisterTargets'
- 'elasticloadbalancing:Describe*'
- 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
- 'elasticloadbalancing:RegisterTargets'
Resource: '*'
# This is a role which is used by the ECS tasks themselves.
ECSTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs-tasks.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: AmazonECSTaskExecutionRolePolicy
PolicyDocument:
Statement:
- Effect: Allow
Action:
# Allow the use of secret manager
- 'secretsmanager:GetSecretValue'
- 'kms:Decrypt'
# Allow the ECS Tasks to download images from ECR
- 'ecr:GetAuthorizationToken'
- 'ecr:BatchCheckLayerAvailability'
- 'ecr:GetDownloadUrlForLayer'
- 'ecr:BatchGetImage'
# Allow the ECS tasks to upload logs to CloudWatch
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: '*'
DeleteCFNLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service: ["lambda.amazonaws.com"]
Action: "sts:AssumeRole"
Path: "/"
Policies:
- PolicyName: DeleteCFNLambdaExecutionRole
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
Resource: "arn:aws:logs:*:*:*"
- Effect: "Allow"
Action:
- "cloudformation:DeleteStack"
- "kinesis:DeleteStream"
- "secretsmanager:DeleteSecret"
- "kinesis:DescribeStreamSummary"
- "logs:DeleteLogGroup"
- "logs:DeleteSubscriptionFilter"
- "ecs:DeregisterTaskDefinition"
- "lambda:DeleteFunction"
- "lambda:InvokeFunction"
- "events:RemoveTargets"
- "events:DeleteRule"
- "lambda:RemovePermission"
Resource: "*"
### cloud watch to kinesis role
CloudWatchIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [logs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'kinesis:PutRecord'
Resource: '*'
#####################EFS#####################
EfsFileStorage:
Type: 'AWS::EFS::FileSystem'
Properties:
BackupPolicy:
Status: ENABLED
PerformanceMode: maxIO
Encrypted: false
FileSystemPolicy:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "elasticfilesystem:ClientMount"
- "elasticfilesystem:ClientWrite"
- "elasticfilesystem:ClientRootAccess"
Principal:
AWS: "*"
MountTargetResource1:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref EfsFileStorage
SubnetId: !Ref PublicSubnetOne
SecurityGroups:
- !Ref EFSServerSecurityGroup
MountTargetResource2:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref EfsFileStorage
SubnetId: !Ref PublicSubnetTwo
SecurityGroups:
- !Ref EFSServerSecurityGroup
Outputs:
EfsFileStorageId:
Description: 'The connection endpoint for the database.'
Value: !Ref EfsFileStorage
Export:
Name: !Sub ${EnvironmentName}:EfsFileStorageId
ClusterName:
Description: The name of the ECS cluster
Value: !Ref 'ECSCluster'
Export:
Name: !Sub ${EnvironmentName}:ClusterName
AutoscalingRole:
Description: The ARN of the role used for autoscaling
Value: !GetAtt 'AutoscalingRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:AutoscalingRole
ECSRole:
Description: The ARN of the ECS role
Value: !GetAtt 'ECSRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:ECSRole
ECSTaskExecutionRole:
Description: The ARN of the ECS role tsk execution role
Value: !GetAtt 'ECSTaskExecutionRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:ECSTaskExecutionRole
DeleteCFNLambdaExecutionRole:
Description: Lambda execution role for cleaning up cloud formations
Value: !GetAtt 'DeleteCFNLambdaExecutionRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:DeleteCFNLambdaExecutionRole
CloudWatchIAMRole:
Description: The ARN of the CloudWatch role for subscription filter
Value: !GetAtt 'CloudWatchIAMRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:CloudWatchIAMRole
VpcId:
Description: The ID of the VPC that this stack is deployed in
Value: !Ref 'VPC'
Export:
Name: !Sub ${EnvironmentName}:VpcId
PublicSubnetOne:
Description: Public subnet one
Value: !Ref 'PublicSubnetOne'
Export:
Name: !Sub ${EnvironmentName}:PublicSubnetOne
PublicSubnetTwo:
Description: Public subnet two
Value: !Ref 'PublicSubnetTwo'
Export:
Name: !Sub ${EnvironmentName}:PublicSubnetTwo
ContainerSecurityGroup:
Description: A security group used to allow Fargate containers to receive traffic
Value: !Ref 'ContainerSecurityGroup'
Export:
Name: !Sub ${EnvironmentName}:ContainerSecurityGroup

View File

@@ -1,221 +0,0 @@
AWSTemplateFormatVersion: 2010-09-09
Description: >-
AWS Fargate cluster that can span public and private subnets. Supports public
facing load balancers, private internal load balancers, and both internal and
external service discovery namespaces.
Parameters:
EnvironmentName:
Type: String
Default: development
Description: 'Your deployment environment: DEV, QA , PROD'
ServiceName:
Type: String
Default: example
Description: A name for the service
ImageUrl:
Type: String
Default: nginx
Description: >-
The url of a docker image that contains the application process that will
handle the traffic for this service
ContainerPort:
Type: Number
Default: 80
Description: What port number the application inside the docker container is binding to
ContainerCpu:
Type: Number
Default: 1024
Description: How much CPU to give the container. 1024 is 1 CPU
ContainerMemory:
Type: Number
Default: 2048
Description: How much memory in megabytes to give the container
BUILDGUID:
Type: String
Default: ''
Command:
Type: String
Default: 'ls'
EntryPoint:
Type: String
Default: '/bin/sh'
WorkingDirectory:
Type: String
Default: '/efsdata/'
Role:
Type: String
Default: ''
Description: >-
(Optional) An IAM role to give the service's containers if the code within
needs to access other AWS resources
EFSMountDirectory:
Type: String
Default: '/efsdata'
# template secrets p1 - input
Mappings:
SubnetConfig:
VPC:
CIDR: 10.0.0.0/16
PublicOne:
CIDR: 10.0.0.0/24
PublicTwo:
CIDR: 10.0.1.0/24
Conditions:
HasCustomRole: !Not
- !Equals
- Ref: Role
- ''
Resources:
LogGroup:
Type: 'AWS::Logs::LogGroup'
Properties:
LogGroupName: !Ref ServiceName
Metadata:
'AWS::CloudFormation::Designer':
id: aece53ae-b82d-4267-bc16-ed964b05db27
SubscriptionFilter:
Type: 'AWS::Logs::SubscriptionFilter'
Properties:
FilterPattern: ''
RoleArn:
'Fn::ImportValue': !Sub '${EnvironmentName}:CloudWatchIAMRole'
LogGroupName: !Ref ServiceName
DestinationArn:
'Fn::GetAtt':
- KinesisStream
- Arn
Metadata:
'AWS::CloudFormation::Designer':
id: 7f809e91-9e5d-4678-98c1-c5085956c480
DependsOn:
- LogGroup
- KinesisStream
KinesisStream:
Type: 'AWS::Kinesis::Stream'
Properties:
Name: !Ref ServiceName
ShardCount: 1
Metadata:
'AWS::CloudFormation::Designer':
id: c6f18447-b879-4696-8873-f981b2cedd2b
# template secrets p2 - secret
TaskDefinition:
Type: 'AWS::ECS::TaskDefinition'
Properties:
Family: !Ref ServiceName
Cpu: !Ref ContainerCpu
Memory: !Ref ContainerMemory
NetworkMode: awsvpc
Volumes:
- Name: efs-data
EFSVolumeConfiguration:
FilesystemId:
'Fn::ImportValue': !Sub '${EnvironmentName}:EfsFileStorageId'
TransitEncryption: ENABLED
RequiresCompatibilities:
- FARGATE
ExecutionRoleArn:
'Fn::ImportValue': !Sub '${EnvironmentName}:ECSTaskExecutionRole'
TaskRoleArn:
'Fn::If':
- HasCustomRole
- !Ref Role
- !Ref 'AWS::NoValue'
ContainerDefinitions:
- Name: !Ref ServiceName
Cpu: !Ref ContainerCpu
Memory: !Ref ContainerMemory
Image: !Ref ImageUrl
EntryPoint:
Fn::Split:
- ","
- !Ref EntryPoint
Command:
Fn::Split:
- ","
- !Ref Command
WorkingDirectory: !Ref WorkingDirectory
Environment:
- Name: ALLOW_EMPTY_PASSWORD
Value: 'yes'
# template - env vars
MountPoints:
- SourceVolume: efs-data
ContainerPath: !Ref EFSMountDirectory
ReadOnly: false
Secrets:
# template secrets p3 - container def
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref ServiceName
awslogs-region: !Ref 'AWS::Region'
awslogs-stream-prefix: !Ref ServiceName
Metadata:
'AWS::CloudFormation::Designer':
id: dabb0116-abe0-48a6-a8af-cf9111c879a5
DependsOn:
- LogGroup
Metadata:
'AWS::CloudFormation::Designer':
dabb0116-abe0-48a6-a8af-cf9111c879a5:
size:
width: 60
height: 60
position:
x: 270
'y': 90
z: 1
embeds: []
dependson:
- aece53ae-b82d-4267-bc16-ed964b05db27
c6f18447-b879-4696-8873-f981b2cedd2b:
size:
width: 60
height: 60
position:
x: 270
'y': 210
z: 1
embeds: []
7f809e91-9e5d-4678-98c1-c5085956c480:
size:
width: 60
height: 60
position:
x: 60
'y': 300
z: 1
embeds: []
dependson:
- aece53ae-b82d-4267-bc16-ed964b05db27
- c6f18447-b879-4696-8873-f981b2cedd2b
aece53ae-b82d-4267-bc16-ed964b05db27:
size:
width: 150
height: 150
position:
x: 60
'y': 90
z: 1
embeds: []
4d2da56c-3643-46b8-aaee-e46e19f95fcc:
source:
id: 7f809e91-9e5d-4678-98c1-c5085956c480
target:
id: aece53ae-b82d-4267-bc16-ed964b05db27
z: 11
14eb957b-f094-4653-93c4-77b2f851953c:
source:
id: 7f809e91-9e5d-4678-98c1-c5085956c480
target:
id: c6f18447-b879-4696-8873-f981b2cedd2b
z: 12
85c57444-e5bb-4230-bc85-e545cd4558f6:
source:
id: dabb0116-abe0-48a6-a8af-cf9111c879a5
target:
id: aece53ae-b82d-4267-bc16-ed964b05db27
z: 13

View File

@@ -9,8 +9,13 @@ namespace UnityBuilderAction.Input
public static void Apply(Dictionary<string, string> options)
{
EditorUserBuildSettings.buildAppBundle = options["customBuildPath"].EndsWith(".aab");
#if UNITY_2019_1_OR_NEWER
if (options.TryGetValue("androidKeystoreName", out string keystoreName) && !string.IsNullOrEmpty(keystoreName))
{
PlayerSettings.Android.useCustomKeystore = true;
PlayerSettings.Android.keystoreName = keystoreName;
}
#endif
if (options.TryGetValue("androidKeystorePass", out string keystorePass) && !string.IsNullOrEmpty(keystorePass))
PlayerSettings.Android.keystorePass = keystorePass;
if (options.TryGetValue("androidKeyaliasName", out string keyaliasName) && !string.IsNullOrEmpty(keyaliasName))
@@ -18,7 +23,19 @@ namespace UnityBuilderAction.Input
if (options.TryGetValue("androidKeyaliasPass", out string keyaliasPass) && !string.IsNullOrEmpty(keyaliasPass))
PlayerSettings.Android.keyaliasPass = keyaliasPass;
if (options.TryGetValue("androidTargetSdkVersion", out string androidTargetSdkVersion) && !string.IsNullOrEmpty(androidTargetSdkVersion))
PlayerSettings.Android.targetSdkVersion = (AndroidSdkVersions) Enum.Parse(typeof(AndroidSdkVersions), androidTargetSdkVersion);
{
var targetSdkVersion = AndroidSdkVersions.AndroidApiLevelAuto;
try
{
targetSdkVersion =
(AndroidSdkVersions) Enum.Parse(typeof(AndroidSdkVersions), androidTargetSdkVersion);
}
catch
{
UnityEngine.Debug.Log("Failed to parse androidTargetSdkVersion! Fallback to AndroidApiLevelAuto");
}
PlayerSettings.Android.targetSdkVersion = targetSdkVersion;
}
}
}
}

3557
dist/index.js generated vendored

File diff suppressed because it is too large Load Diff

2
dist/index.js.map generated vendored

File diff suppressed because one or more lines are too long

20
dist/licenses.txt generated vendored
View File

@@ -4194,25 +4194,13 @@ uuid
MIT
The MIT License (MIT)
Copyright (c) 2010-2016 Robert Kieffer and other contributors
Copyright (c) 2010-2020 Robert Kieffer and other contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
verror

View File

@@ -126,7 +126,6 @@ echo ""
# Reference: https://docs.unity3d.com/2019.3/Documentation/Manual/CommandLineArguments.html
/Applications/Unity/Hub/Editor/$UNITY_VERSION/Unity.app/Contents/MacOS/Unity \
-logfile /dev/stdout \
-quit \
-batchmode \
-nographics \
@@ -145,11 +144,15 @@ echo ""
-androidKeyaliasName "$ANDROID_KEYALIAS_NAME" \
-androidKeyaliasPass "$ANDROID_KEYALIAS_PASS" \
-androidTargetSdkVersion "$ANDROID_TARGET_SDK_VERSION" \
$CUSTOM_PARAMETERS
$CUSTOM_PARAMETERS \
> "$UNITY_PROJECT_PATH/out.log" 2>&1
# Catch exit code
BUILD_EXIT_CODE=$?
# Display logs
cat "$UNITY_PROJECT_PATH/out.log"
# Display results
if [ $BUILD_EXIT_CODE -eq 0 ]; then
echo "Build succeeded";

View File

@@ -9,6 +9,8 @@ pushd "$ACTIVATE_LICENSE_PATH"
-batchmode \
-nographics \
-quit \
-username "$UNITY_EMAIL" \
-password "$UNITY_PASSWORD" \
-returnlicense \
-projectPath "$ACTIVATE_LICENSE_PATH"

View File

@@ -10,7 +10,7 @@ else
git config --global --replace-all url."https://token:$GIT_PRIVATE_TOKEN@github.com/".insteadOf ssh://git@github.com/
git config --global --add url."https://token:$GIT_PRIVATE_TOKEN@github.com/".insteadOf git@github.com
git config --global url."https://token:$GIT_PRIVATE_TOKEN@github.com/".insteadOf "https://github.com/"
git config --global --add url."https://token:$GIT_PRIVATE_TOKEN@github.com/".insteadOf "https://github.com/"
git config --global url."https://ssh:$GIT_PRIVATE_TOKEN@github.com/".insteadOf "ssh://git@github.com/"
git config --global url."https://git:$GIT_PRIVATE_TOKEN@github.com/".insteadOf "git@github.com:"

View File

@@ -116,6 +116,12 @@ Write-Output ""
-customBuildTarget $Env:BUILD_TARGET `
-customBuildPath $Env:CUSTOM_BUILD_PATH `
-buildVersion $Env:VERSION `
-androidVersionCode $Env:ANDROID_VERSION_CODE `
-androidKeystoreName $Env:ANDROID_KEYSTORE_NAME `
-androidKeystorePass $Env:ANDROID_KEYSTORE_PASS `
-androidKeyaliasName $Env:ANDROID_KEYALIAS_NAME `
-androidKeyaliasPass $Env:ANDROID_KEYALIAS_PASS `
-androidTargetSdkVersion $Env:ANDROID_TARGET_SDK_VERSION `
$Env:CUSTOM_PARAMETERS `
-logfile | Out-Host

View File

@@ -6,13 +6,6 @@
color: true
extends: {}
pre-push:
parallel: true
commands:
packages-audit:
tags: security
run: yarn audit
pre-commit:
parallel: true
commands:

6956
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -7,7 +7,7 @@
"author": "Webber <webber@takken.io>",
"license": "MIT",
"scripts": {
"prepare": "lefthook install",
"prepare": "lefthook install && npx husky uninstall -y",
"build": "yarn && tsc && ncc build lib --source-map --license licenses.txt",
"lint": "prettier --check \"src/**/*.{js,ts}\" && eslint src/**/*.ts",
"format": "prettier --write \"src/**/*.{js,ts}\"",
@@ -25,7 +25,7 @@
"test-i-k8s": "cross-env cloudRunnerTests=true cloudRunnerCluster=k8s yarn test -i -t \"cloud runner\""
},
"dependencies": {
"@actions/core": "^1.6.0",
"@actions/core": "^1.10.0",
"@actions/exec": "^1.1.0",
"@actions/github": "^5.0.0",
"@kubernetes/client-node": "^0.16.3",

View File

@@ -1,5 +1,5 @@
import * as core from '@actions/core';
import { Action, BuildParameters, Cache, Docker, ImageTag, Output, CloudRunner } from './model';
import { Action, BuildParameters, Cache, CloudRunner, Docker, ImageTag, Output } from './model';
import { Cli } from './model/cli/cli';
import MacBuilder from './model/mac-builder';
import PlatformSetup from './model/platform-setup';
@@ -7,6 +7,7 @@ async function runMain() {
try {
if (Cli.InitCliMode()) {
await Cli.RunCli();
return;
}
Action.checkCompatibility();

9
src/integrity.test.ts Normal file
View File

@@ -0,0 +1,9 @@
import { stat } from 'fs/promises';
describe('Integrity tests', () => {
describe('package-lock.json', () => {
it('does not exist', async () => {
await expect(stat(`${process.cwd()}/package-lock.json`)).rejects.toThrowError();
});
});
});

View File

@@ -0,0 +1,112 @@
export const completelyValidSemanticVersions = [
'0.0.4',
'1.2.3',
'10.20.30',
'1.1.2-prerelease+meta',
'1.1.2+meta',
'1.1.2+meta-valid',
'1.0.0-alpha',
'1.0.0-beta',
'1.0.0-alpha.beta',
'1.0.0-alpha.beta.1',
'1.0.0-alpha.1',
'1.0.0-alpha0.valid',
'1.0.0-alpha.0valid',
'1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay',
'1.0.0-rc.1+build.1',
'2.0.0-rc.1+build.123',
'1.2.3-beta',
'10.2.3-DEV-SNAPSHOT',
'1.2.3-SNAPSHOT-123',
'1.0.0',
'2.0.0',
'1.1.7',
'2.0.0+build.1848',
'2.0.1-alpha.1227',
'1.0.0-alpha+beta',
'1.2.3----RC-SNAPSHOT.12.9.1--.12+788',
'1.2.3----R-S.12.9.1--.12+meta',
'1.2.3----RC-SNAPSHOT.12.9.1--.12',
'1.0.0+0.build.1-rc.10000aaa-kk-0.1',
'99999999999999999999999.999999999999999999.99999999999999999',
'1.0.0-0A.is.legal',
];
export const notCompletelyValidSemanticVersions = [
'1',
'1.2',
'1.2.3-0123',
'1.2.3-0123.0123',
'1.1.2+.123',
'+invalid',
'-invalid',
'-invalid+invalid',
'-invalid.01',
'alpha',
'alpha.beta',
'alpha.beta.1',
'alpha.1',
'alpha+beta',
'alpha_beta',
'alpha.',
'alpha..',
'beta',
'1.0.0-alpha_beta',
'-alpha.',
'1.0.0-alpha..',
'1.0.0-alpha..1',
'1.0.0-alpha...1',
'1.0.0-alpha....1',
'1.0.0-alpha.....1',
'1.0.0-alpha......1',
'1.0.0-alpha.......1',
'01.1.1',
'1.01.1',
'1.1.01',
'1.2',
'1.2.3.DEV',
'1.2-SNAPSHOT',
'1.2.31.2.3----RC-SNAPSHOT.12.09.1--..12+788',
'1.2-RC-SNAPSHOT',
'-1.0.3-gamma+b7718',
'+justmeta',
'9.8.7+meta+meta',
'9.8.7-whatever+meta+meta',
'99999999999999999999999.999999999999999999.99999999999999999----RC-SNAPSHOT.12.09.1--------------------------------..12',
];
const addVariantsPrependingV = (array: string[]) => array.map((tag) => [tag, `v${tag}`]).flat();
/**
* Array of versions that will be detected as version tags. Not all of these are
* "semantic versions", but can be used to generate one. Especially using the
* `versioning: Semantic` option.
*/
export const validVersionTagInputs = addVariantsPrependingV([
'0',
'1',
'0.1',
'1.0',
'1.1.0',
'1.2.3',
...completelyValidSemanticVersions,
]);
export const invalidVersionTagInputs = addVariantsPrependingV([
'+invalid',
'-invalid',
'-invalid+invalid',
'-invalid.01',
'alpha',
'alpha.beta',
'alpha.beta.1',
'alpha.1',
'alpha+beta',
'alpha_beta',
'alpha.',
'alpha..',
'beta',
'-alpha.',
'-1.0.3-gamma+b7718',
'+justmeta',
]);

View File

@@ -6,12 +6,14 @@ export default class AndroidVersioning {
if (!inputVersionCode) {
return AndroidVersioning.versionToVersionCode(version);
}
return inputVersionCode;
}
static versionToVersionCode(version) {
if (version === 'none') {
core.info(`Versioning strategy is set to ${version}, so android version code should not be applied.`);
return 0;
}
@@ -19,6 +21,7 @@ export default class AndroidVersioning {
if (!parsedVersion) {
core.warning(`Could not parse "${version}" to semver, defaulting android version code to 1`);
return 1;
}
@@ -32,11 +35,13 @@ export default class AndroidVersioning {
);
}
core.info(`Using android versionCode ${versionCode}`);
return versionCode;
}
static determineSdkManagerParameters(targetSdkVersion) {
const parsedVersion = Number.parseInt(targetSdkVersion.slice(-2), 10);
return Number.isNaN(parsedVersion) ? '' : `platforms;android-${parsedVersion}`;
}
}

View File

@@ -76,8 +76,8 @@ class BuildParameters {
// Todo - Don't use process.env directly, that's what the input model class is for.
// ---
let unitySerial = '';
if (!process.env.UNITY_SERIAL && Input.githubInputEnabled && Cli.options === undefined) {
//No serial was present so it is a personal license that we need to convert
if (!process.env.UNITY_SERIAL && Input.githubInputEnabled) {
// No serial was present, so it is a personal license that we need to convert
if (!process.env.UNITY_LICENSE) {
throw new Error(`Missing Unity License File and no Serial was found. If this
is a personal license, make sure to follow the activation
@@ -167,6 +167,7 @@ class BuildParameters {
throw new Error(`License File was corrupted, unable to locate serial`);
}
const endIndex = license.indexOf(endKey, startIndex);
// Slice off the first 4 characters as they are garbage values
return Buffer.from(license.slice(startIndex, endIndex), 'base64').toString('binary').slice(4);
}

View File

@@ -21,6 +21,7 @@ export class CliFunctionsRepository {
if (results === undefined || results.length === 0) {
throw new Error(`no CLI mode found for ${key}`);
}
return results;
}

View File

@@ -22,6 +22,7 @@ export class Cli {
if (Cli.options && alternativeKey && Cli.options[alternativeKey] !== undefined) {
return Cli.options[alternativeKey];
}
return;
}
@@ -49,6 +50,7 @@ export class Cli {
program.option('--artifactName <artifactName>', 'caching artifact name');
program.parse(process.argv);
Cli.options = program.opts();
return Cli.isCliMode;
}
@@ -61,6 +63,7 @@ export class Cli {
const results = CliFunctionsRepository.GetCliFunctions(Cli.options.mode);
CloudRunnerLogger.log(`Entrypoint: ${results.key}`);
Cli.options.versioning = 'None';
return await results.target[results.propertyKey]();
}
@@ -88,6 +91,7 @@ export class Cli {
public static async CLIBuild(): Promise<string> {
const buildParameter = await BuildParameters.create();
const baseImage = new ImageTag(buildParameter);
return await CloudRunner.run(buildParameter, baseImage.toString());
}
}

View File

@@ -1,3 +1,3 @@
export class CloudRunnerStatics {
public static readonly logPrefix = `Cloud-Runner-System`;
public static readonly logPrefix = `Cloud-Runner`;
}

View File

@@ -16,7 +16,7 @@ describe('Cloud Runner', () => {
const testSecretValue = 'testSecretValue';
if (Input.cloudRunnerTests) {
it('All build parameters sent to cloud runner as env vars', async () => {
// build parameters
// Build parameters
Cli.options = {
versioning: 'None',
projectPath: 'test-project',
@@ -32,13 +32,16 @@ describe('Cloud Runner', () => {
`,
};
Input.githubInputEnabled = false;
// setup parameters
// Setup parameters
const buildParameter = await BuildParameters.create();
Input.githubInputEnabled = true;
const baseImage = new ImageTag(buildParameter);
// run the job
// Run the job
const file = await CloudRunner.run(buildParameter, baseImage.toString());
// assert results
// Assert results
expect(file).toContain(JSON.stringify(buildParameter));
expect(file).toContain(`${Input.ToEnvVarFormat(testSecretName)}=${testSecretValue}`);
const environmentVariables = TaskParameterSerializer.readBuildEnvironmentVariables();
@@ -89,7 +92,7 @@ describe('Cloud Runner', () => {
}, 1000000);
}
it('Local cloud runner returns commands', async () => {
// build parameters
// Build parameters
Cli.options = {
versioning: 'None',
projectPath: 'test-project',
@@ -106,16 +109,18 @@ describe('Cloud Runner', () => {
`,
};
Input.githubInputEnabled = false;
// setup parameters
// Setup parameters
const buildParameter = await BuildParameters.create();
const baseImage = new ImageTag(buildParameter);
// run the job
// Run the job
await expect(CloudRunner.run(buildParameter, baseImage.toString())).resolves.not.toThrow();
Input.githubInputEnabled = true;
delete Cli.options;
}, 1000000);
it('Test cloud runner returns commands', async () => {
// build parameters
// Build parameters
Cli.options = {
versioning: 'None',
projectPath: 'test-project',
@@ -124,10 +129,12 @@ describe('Cloud Runner', () => {
targetPlatform: 'StandaloneLinux64',
};
Input.githubInputEnabled = false;
// setup parameters
// Setup parameters
const buildParameter = await BuildParameters.create();
const baseImage = new ImageTag(buildParameter);
// run the job
// Run the job
await expect(CloudRunner.run(buildParameter, baseImage.toString())).resolves.not.toThrow();
Input.githubInputEnabled = true;
delete Cli.options;

View File

@@ -80,6 +80,7 @@ class CloudRunner {
);
CloudRunnerLogger.log(`Cleanup complete`);
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();
return output;
} catch (error) {
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();

View File

@@ -1,8 +1,7 @@
import CloudRunnerLogger from '../../services/cloud-runner-logger';
import * as core from '@actions/core';
import * as SDK from 'aws-sdk';
import * as fs from 'fs';
import path from 'path';
import { BaseStackFormation } from './cloud-formations/base-stack-formation';
const crypto = require('crypto');
export class AWSBaseStack {
@@ -14,7 +13,7 @@ export class AWSBaseStack {
async setupBaseStack(CF: SDK.CloudFormation) {
const baseStackName = this.baseStackName;
const baseStack = fs.readFileSync(path.join(__dirname, 'cloud-formations', 'base-setup.yml'), 'utf8');
const baseStack = BaseStackFormation.formation;
// Cloud Formation Input
const describeStackInput: SDK.CloudFormation.DescribeStacksInput = {

View File

@@ -1,4 +1,4 @@
import * as fs from 'fs';
import { TaskDefinitionFormation } from './cloud-formations/task-definition-formation';
export class AWSCloudFormationTemplates {
public static getParameterTemplate(p1) {
@@ -29,10 +29,11 @@ export class AWSCloudFormationTemplates {
public static insertAtTemplate(template, insertionKey, insertion) {
const index = template.search(insertionKey) + insertionKey.length + '\n'.length;
template = [template.slice(0, index), insertion, template.slice(index)].join('');
return template;
}
public static readTaskCloudFormationTemplate(): string {
return fs.readFileSync(`${__dirname}/cloud-formations/task-def-formation.yml`, 'utf8');
return TaskDefinitionFormation.formation;
}
}

View File

@@ -4,6 +4,7 @@ import CloudRunnerSecret from '../../services/cloud-runner-secret';
import { AWSCloudFormationTemplates } from './aws-cloud-formation-templates';
import CloudRunnerLogger from '../../services/cloud-runner-logger';
import { AWSError } from './aws-error';
import CloudRunner from '../../cloud-runner';
export class AWSJobStack {
private baseStackName: string;
@@ -23,6 +24,20 @@ export class AWSJobStack {
): Promise<CloudRunnerAWSTaskDef> {
const taskDefStackName = `${this.baseStackName}-${buildGuid}`;
let taskDefCloudFormation = AWSCloudFormationTemplates.readTaskCloudFormationTemplate();
const cpu = CloudRunner.buildParameters.cloudRunnerCpu || '1024';
const memory = CloudRunner.buildParameters.cloudRunnerMemory || '3072';
taskDefCloudFormation = taskDefCloudFormation.replace(
`ContainerCpu:
Default: 1024`,
`ContainerCpu:
Default: ${Number.parseInt(cpu)}`,
);
taskDefCloudFormation = taskDefCloudFormation.replace(
`ContainerMemory:
Default: 2048`,
`ContainerMemory:
Default: ${Number.parseInt(memory)}`,
);
for (const secret of secrets) {
secret.ParameterKey = `${buildGuid.replace(/[^\dA-Za-z]/g, '')}${secret.ParameterKey.replace(
/[^\dA-Za-z]/g,
@@ -85,7 +100,9 @@ export class AWSJobStack {
},
...secretsMappedToCloudFormationParameters,
];
CloudRunnerLogger.log(
`Starting AWS job with memory: ${CloudRunner.buildParameters.cloudRunnerMemory} cpu: ${CloudRunner.buildParameters.cloudRunnerCpu}`,
);
let previousStackExists = true;
while (previousStackExists) {
previousStackExists = false;
@@ -101,25 +118,19 @@ export class AWSJobStack {
}
}
}
const createStackInput: SDK.CloudFormation.CreateStackInput = {
StackName: taskDefStackName,
TemplateBody: taskDefCloudFormation,
Capabilities: ['CAPABILITY_IAM'],
Parameters: parameters,
};
try {
await CF.createStack({
StackName: taskDefStackName,
TemplateBody: taskDefCloudFormation,
Capabilities: ['CAPABILITY_IAM'],
Parameters: parameters,
}).promise();
await CF.createStack(createStackInput).promise();
CloudRunnerLogger.log('Creating cloud runner job');
await CF.waitFor('stackCreateComplete', { StackName: taskDefStackName }).promise();
} catch (error) {
await AWSError.handleStackCreationFailure(
error,
CF,
taskDefStackName,
//taskDefCloudFormation,
//parameters,
//secrets,
);
await AWSError.handleStackCreationFailure(error, CF, taskDefStackName);
throw error;
}

View File

@@ -6,8 +6,8 @@ import * as zlib from 'zlib';
import CloudRunnerLogger from '../../services/cloud-runner-logger';
import { Input } from '../../..';
import CloudRunner from '../../cloud-runner';
import { CloudRunnerStatics } from '../../cloud-runner-statics';
import { CloudRunnerBuildCommandProcessor } from '../../services/cloud-runner-build-command-process';
import { FollowLogStreamService } from '../../services/follow-log-stream-service';
class AWSTaskRunner {
static async runTask(
@@ -58,13 +58,21 @@ class AWSTaskRunner {
CloudRunnerLogger.log(
`Cloud runner job status is running ${(await AWSTaskRunner.describeTasks(ECS, cluster, taskArn))?.lastStatus}`,
);
const output = await this.streamLogsUntilTaskStops(ECS, CF, taskDef, cluster, taskArn, streamName);
const { output, shouldCleanup } = await this.streamLogsUntilTaskStops(
ECS,
CF,
taskDef,
cluster,
taskArn,
streamName,
);
const taskData = await AWSTaskRunner.describeTasks(ECS, cluster, taskArn);
const exitCode = taskData.containers?.[0].exitCode;
const wasSuccessful = exitCode === 0 || (exitCode === undefined && taskData.lastStatus === 'RUNNING');
if (wasSuccessful) {
CloudRunnerLogger.log(`Cloud runner job has finished successfully`);
return output;
return { output, shouldCleanup };
} else {
if (taskData.stoppedReason === 'Essential container in task exited' && exitCode === 1) {
throw new Error('Container exited with code 1');
@@ -118,24 +126,27 @@ class AWSTaskRunner {
const stream = await AWSTaskRunner.getLogStream(kinesis, kinesisStreamName);
let iterator = await AWSTaskRunner.getLogIterator(kinesis, stream);
const logBaseUrl = `https://${Input.region}.console.aws.amazon.com/cloudwatch/home?region=${CF.config.region}#logsV2:log-groups/log-group/${taskDef.taskDefStackName}`;
CloudRunnerLogger.log(`You can also see the logs at AWS Cloud Watch: ${logBaseUrl}`);
const logBaseUrl = `https://${Input.region}.console.aws.amazon.com/cloudwatch/home?region=${Input.region}#logsV2:log-groups/log-group/${CloudRunner.buildParameters.awsBaseStackName}-${CloudRunner.buildParameters.buildGuid}`;
CloudRunnerLogger.log(`You view the log stream on AWS Cloud Watch: ${logBaseUrl}`);
let shouldReadLogs = true;
let shouldCleanup = true;
let timestamp: number = 0;
let output = '';
while (shouldReadLogs) {
await new Promise((resolve) => setTimeout(resolve, 1500));
const taskData = await AWSTaskRunner.describeTasks(ECS, clusterName, taskArn);
({ timestamp, shouldReadLogs } = AWSTaskRunner.checkStreamingShouldContinue(taskData, timestamp, shouldReadLogs));
({ iterator, shouldReadLogs, output } = await AWSTaskRunner.handleLogStreamIteration(
({ iterator, shouldReadLogs, output, shouldCleanup } = await AWSTaskRunner.handleLogStreamIteration(
kinesis,
iterator,
shouldReadLogs,
taskDef,
output,
shouldCleanup,
));
}
return output;
return { output, shouldCleanup };
}
private static async handleLogStreamIteration(
@@ -144,6 +155,7 @@ class AWSTaskRunner {
shouldReadLogs: boolean,
taskDef: CloudRunnerAWSTaskDef,
output: string,
shouldCleanup: boolean,
) {
const records = await kinesis
.getRecords({
@@ -151,8 +163,16 @@ class AWSTaskRunner {
})
.promise();
iterator = records.NextShardIterator || '';
({ shouldReadLogs, output } = AWSTaskRunner.logRecords(records, iterator, taskDef, shouldReadLogs, output));
return { iterator, shouldReadLogs, output };
({ shouldReadLogs, output, shouldCleanup } = AWSTaskRunner.logRecords(
records,
iterator,
taskDef,
shouldReadLogs,
output,
shouldCleanup,
));
return { iterator, shouldReadLogs, output, shouldCleanup };
}
private static checkStreamingShouldContinue(taskData: AWS.ECS.Task, timestamp: number, shouldReadLogs: boolean) {
@@ -170,6 +190,7 @@ class AWSTaskRunner {
}
CloudRunnerLogger.log(`## Status of job: ${taskData.lastStatus}`);
}
return { timestamp, shouldReadLogs };
}
@@ -179,6 +200,7 @@ class AWSTaskRunner {
taskDef: CloudRunnerAWSTaskDef,
shouldReadLogs: boolean,
output: string,
shouldCleanup: boolean,
) {
if (records.Records.length > 0 && iterator) {
for (let index = 0; index < records.Records.length; index++) {
@@ -187,28 +209,19 @@ class AWSTaskRunner {
);
if (json.messageType === 'DATA_MESSAGE') {
for (let logEventsIndex = 0; logEventsIndex < json.logEvents.length; logEventsIndex++) {
let message = json.logEvents[logEventsIndex].message;
if (json.logEvents[logEventsIndex].message.includes(`---${CloudRunner.buildParameters.logId}`)) {
CloudRunnerLogger.log('End of log transmission received');
shouldReadLogs = false;
} else if (message.includes('Rebuilding Library because the asset database could not be found!')) {
core.warning('LIBRARY NOT FOUND!');
} else if (message.includes('Build succeeded')) {
core.setOutput('build-result', 'success');
} else if (message.includes('Build fail')) {
core.setOutput('build-result', 'failed');
core.error('BUILD FAILED!');
}
message = `[${CloudRunnerStatics.logPrefix}] ${message}`;
if (CloudRunner.buildParameters.cloudRunnerIntegrationTests) {
output += message;
}
CloudRunnerLogger.log(message);
const message = json.logEvents[logEventsIndex].message;
({ shouldReadLogs, shouldCleanup, output } = FollowLogStreamService.handleIteration(
message,
shouldReadLogs,
shouldCleanup,
output,
));
}
}
}
}
return { shouldReadLogs, output };
return { shouldReadLogs, output, shouldCleanup };
}
private static async getLogStream(kinesis: AWS.Kinesis, kinesisStreamName: string) {

View File

@@ -1,7 +1,7 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Fargate cluster that can span public and private subnets. Supports
public facing load balancers, private internal load balancers, and
both internal and external service discovery namespaces.
export class BaseStackFormation {
public static readonly baseStackDecription = `Game-CI base stack`;
public static readonly formation: string = `AWSTemplateFormatVersion: '2010-09-09'
Description: ${BaseStackFormation.baseStackDecription}
Parameters:
EnvironmentName:
Type: String
@@ -335,57 +335,58 @@ Outputs:
Description: 'The connection endpoint for the database.'
Value: !Ref EfsFileStorage
Export:
Name: !Sub ${EnvironmentName}:EfsFileStorageId
Name: !Sub ${'${EnvironmentName}'}:EfsFileStorageId
ClusterName:
Description: The name of the ECS cluster
Value: !Ref 'ECSCluster'
Export:
Name: !Sub ${EnvironmentName}:ClusterName
Name: !Sub${' ${EnvironmentName}'}:ClusterName
AutoscalingRole:
Description: The ARN of the role used for autoscaling
Value: !GetAtt 'AutoscalingRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:AutoscalingRole
Name: !Sub ${'${EnvironmentName}'}:AutoscalingRole
ECSRole:
Description: The ARN of the ECS role
Value: !GetAtt 'ECSRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:ECSRole
Name: !Sub ${'${EnvironmentName}'}:ECSRole
ECSTaskExecutionRole:
Description: The ARN of the ECS role tsk execution role
Value: !GetAtt 'ECSTaskExecutionRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:ECSTaskExecutionRole
Name: !Sub ${'${EnvironmentName}'}:ECSTaskExecutionRole
DeleteCFNLambdaExecutionRole:
Description: Lambda execution role for cleaning up cloud formations
Value: !GetAtt 'DeleteCFNLambdaExecutionRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:DeleteCFNLambdaExecutionRole
Name: !Sub ${'${EnvironmentName}'}:DeleteCFNLambdaExecutionRole
CloudWatchIAMRole:
Description: The ARN of the CloudWatch role for subscription filter
Value: !GetAtt 'CloudWatchIAMRole.Arn'
Export:
Name: !Sub ${EnvironmentName}:CloudWatchIAMRole
Name: !Sub ${'${EnvironmentName}'}:CloudWatchIAMRole
VpcId:
Description: The ID of the VPC that this stack is deployed in
Value: !Ref 'VPC'
Export:
Name: !Sub ${EnvironmentName}:VpcId
Name: !Sub ${'${EnvironmentName}'}:VpcId
PublicSubnetOne:
Description: Public subnet one
Value: !Ref 'PublicSubnetOne'
Export:
Name: !Sub ${EnvironmentName}:PublicSubnetOne
Name: !Sub ${'${EnvironmentName}'}:PublicSubnetOne
PublicSubnetTwo:
Description: Public subnet two
Value: !Ref 'PublicSubnetTwo'
Export:
Name: !Sub ${EnvironmentName}:PublicSubnetTwo
Name: !Sub ${'${EnvironmentName}'}:PublicSubnetTwo
ContainerSecurityGroup:
Description: A security group used to allow Fargate containers to receive traffic
Value: !Ref 'ContainerSecurityGroup'
Export:
Name: !Sub ${EnvironmentName}:ContainerSecurityGroup
Name: !Sub ${'${EnvironmentName}'}:ContainerSecurityGroup
`;
}

View File

@@ -1,8 +1,7 @@
AWSTemplateFormatVersion: 2010-09-09
Description: >-
AWS Fargate cluster that can span public and private subnets. Supports public
facing load balancers, private internal load balancers, and both internal and
external service discovery namespaces.
export class TaskDefinitionFormation {
public static readonly description: string = `Game CI Cloud Runner Task Stack`;
public static readonly formation: string = `AWSTemplateFormatVersion: 2010-09-09
Description: ${TaskDefinitionFormation.description}
Parameters:
EnvironmentName:
Type: String
@@ -23,12 +22,12 @@ Parameters:
Default: 80
Description: What port number the application inside the docker container is binding to
ContainerCpu:
Type: Number
Default: 1024
Type: Number
Description: How much CPU to give the container. 1024 is 1 CPU
ContainerMemory:
Type: Number
Default: 2048
Type: Number
Description: How much memory in megabytes to give the container
BUILDGUID:
Type: String
@@ -78,7 +77,7 @@ Resources:
Properties:
FilterPattern: ''
RoleArn:
'Fn::ImportValue': !Sub '${EnvironmentName}:CloudWatchIAMRole'
'Fn::ImportValue': !Sub '${'${EnvironmentName}'}:CloudWatchIAMRole'
LogGroupName: !Ref ServiceName
DestinationArn:
'Fn::GetAtt':
@@ -98,9 +97,7 @@ Resources:
Metadata:
'AWS::CloudFormation::Designer':
id: c6f18447-b879-4696-8873-f981b2cedd2b
# template secrets p2 - secret
TaskDefinition:
Type: 'AWS::ECS::TaskDefinition'
Properties:
@@ -112,12 +109,12 @@ Resources:
- Name: efs-data
EFSVolumeConfiguration:
FilesystemId:
'Fn::ImportValue': !Sub '${EnvironmentName}:EfsFileStorageId'
'Fn::ImportValue': !Sub '${'${EnvironmentName}'}:EfsFileStorageId'
TransitEncryption: ENABLED
RequiresCompatibilities:
- FARGATE
ExecutionRoleArn:
'Fn::ImportValue': !Sub '${EnvironmentName}:ECSTaskExecutionRole'
'Fn::ImportValue': !Sub '${'${EnvironmentName}'}:ECSTaskExecutionRole'
TaskRoleArn:
'Fn::If':
- HasCustomRole
@@ -153,69 +150,7 @@ Resources:
awslogs-group: !Ref ServiceName
awslogs-region: !Ref 'AWS::Region'
awslogs-stream-prefix: !Ref ServiceName
Metadata:
'AWS::CloudFormation::Designer':
id: dabb0116-abe0-48a6-a8af-cf9111c879a5
DependsOn:
- LogGroup
Metadata:
'AWS::CloudFormation::Designer':
dabb0116-abe0-48a6-a8af-cf9111c879a5:
size:
width: 60
height: 60
position:
x: 270
'y': 90
z: 1
embeds: []
dependson:
- aece53ae-b82d-4267-bc16-ed964b05db27
c6f18447-b879-4696-8873-f981b2cedd2b:
size:
width: 60
height: 60
position:
x: 270
'y': 210
z: 1
embeds: []
7f809e91-9e5d-4678-98c1-c5085956c480:
size:
width: 60
height: 60
position:
x: 60
'y': 300
z: 1
embeds: []
dependson:
- aece53ae-b82d-4267-bc16-ed964b05db27
- c6f18447-b879-4696-8873-f981b2cedd2b
aece53ae-b82d-4267-bc16-ed964b05db27:
size:
width: 150
height: 150
position:
x: 60
'y': 90
z: 1
embeds: []
4d2da56c-3643-46b8-aaee-e46e19f95fcc:
source:
id: 7f809e91-9e5d-4678-98c1-c5085956c480
target:
id: aece53ae-b82d-4267-bc16-ed964b05db27
z: 11
14eb957b-f094-4653-93c4-77b2f851953c:
source:
id: 7f809e91-9e5d-4678-98c1-c5085956c480
target:
id: c6f18447-b879-4696-8873-f981b2cedd2b
z: 12
85c57444-e5bb-4230-bc85-e545cd4558f6:
source:
id: dabb0116-abe0-48a6-a8af-cf9111c879a5
target:
id: aece53ae-b82d-4267-bc16-ed964b05db27
z: 13
`;
}

View File

@@ -2,29 +2,169 @@ import AWS from 'aws-sdk';
import { CliFunction } from '../../../../cli/cli-functions-repository';
import Input from '../../../../input';
import CloudRunnerLogger from '../../../services/cloud-runner-logger';
import { BaseStackFormation } from '../cloud-formations/base-stack-formation';
export class AwsCliCommands {
@CliFunction(`aws-garbage-collect`, `garbage collect aws`)
@CliFunction(`aws-list-all`, `List all resources`)
static async awsListAll() {
await AwsCliCommands.awsListStacks(undefined, true);
await AwsCliCommands.awsListTasks();
await AwsCliCommands.awsListLogGroups(undefined, true);
}
@CliFunction(`aws-garbage-collect`, `garbage collect aws resources not in use !WIP!`)
static async garbageCollectAws() {
await AwsCliCommands.cleanup(false);
}
@CliFunction(`aws-garbage-collect-all`, `garbage collect aws resources regardless of whether they are in use`)
static async garbageCollectAwsAll() {
await AwsCliCommands.cleanup(true);
}
@CliFunction(
`aws-garbage-collect-all-1d-older`,
`garbage collect aws resources created more than 1d ago (ignore if they are in use)`,
)
static async garbageCollectAwsAllOlderThanOneDay() {
await AwsCliCommands.cleanup(true, true);
}
static isOlderThan1day(date: any) {
const ageDate = new Date(date.getTime() - Date.now());
return ageDate.getDay() > 0;
}
@CliFunction(`aws-list-stacks`, `List stacks`)
static async awsListStacks(perResultCallback: any = false, verbose: boolean = false) {
process.env.AWS_REGION = Input.region;
CloudRunnerLogger.log(`Cloud Formation stacks`);
const CF = new AWS.CloudFormation();
const stacks =
(await CF.listStacks().promise()).StackSummaries?.filter((_x) => _x.StackStatus !== 'DELETE_COMPLETE') || [];
(await CF.listStacks().promise()).StackSummaries?.filter(
(_x) => _x.StackStatus !== 'DELETE_COMPLETE', // &&
// _x.TemplateDescription === TaskDefinitionFormation.description.replace('\n', ''),
) || [];
CloudRunnerLogger.log(`Stacks ${stacks.length}`);
for (const element of stacks) {
CloudRunnerLogger.log(JSON.stringify(element, undefined, 4));
const ageDate = new Date(element.CreationTime.getTime() - Date.now());
if (verbose)
CloudRunnerLogger.log(
`Task Stack ${element.StackName} - Age D${ageDate.getDay()} H${ageDate.getHours()} M${ageDate.getMinutes()}`,
);
if (perResultCallback) await perResultCallback(element);
}
const baseStacks =
(await CF.listStacks().promise()).StackSummaries?.filter(
(_x) =>
_x.StackStatus !== 'DELETE_COMPLETE' && _x.TemplateDescription === BaseStackFormation.baseStackDecription,
) || [];
CloudRunnerLogger.log(`Base Stacks ${baseStacks.length}`);
for (const element of baseStacks) {
const ageDate = new Date(element.CreationTime.getTime() - Date.now());
if (verbose)
CloudRunnerLogger.log(
`Base Stack ${
element.StackName
} - Age D${ageDate.getHours()} H${ageDate.getHours()} M${ageDate.getMinutes()}`,
);
if (perResultCallback) await perResultCallback(element);
}
CloudRunnerLogger.log(`ECS Clusters`);
const ecs = new AWS.ECS();
const clusters = (await ecs.listClusters().promise()).clusterArns || [];
if (stacks === undefined) {
return;
}
}
@CliFunction(`aws-list-tasks`, `List tasks`)
static async awsListTasks(perResultCallback: any = false) {
process.env.AWS_REGION = Input.region;
const ecs = new AWS.ECS();
const clusters = (await ecs.listClusters().promise()).clusterArns || [];
CloudRunnerLogger.log(`Clusters ${clusters.length}`);
for (const element of clusters) {
const input: AWS.ECS.ListTasksRequest = {
cluster: element,
};
CloudRunnerLogger.log(JSON.stringify(await ecs.listTasks(input).promise(), undefined, 4));
const list = (await ecs.listTasks(input).promise()).taskArns || [];
if (list.length > 0) {
const describeInput: AWS.ECS.DescribeTasksRequest = { tasks: list, cluster: element };
const describeList = (await ecs.describeTasks(describeInput).promise()).tasks || [];
if (describeList === []) {
continue;
}
CloudRunnerLogger.log(`Tasks ${describeList.length}`);
for (const taskElement of describeList) {
if (taskElement === undefined) {
continue;
}
taskElement.overrides = {};
taskElement.attachments = [];
if (taskElement.createdAt === undefined) {
CloudRunnerLogger.log(`Skipping ${taskElement.taskDefinitionArn} no createdAt date`);
continue;
}
if (perResultCallback) await perResultCallback(taskElement, element);
}
}
}
}
@CliFunction(`aws-list-log-groups`, `List tasks`)
static async awsListLogGroups(perResultCallback: any = false, verbose: boolean = false) {
process.env.AWS_REGION = Input.region;
const ecs = new AWS.CloudWatchLogs();
let logStreamInput: AWS.CloudWatchLogs.DescribeLogGroupsRequest = {
/* logGroupNamePrefix: 'game-ci' */
};
let logGroupsDescribe = await ecs.describeLogGroups(logStreamInput).promise();
const logGroups = logGroupsDescribe.logGroups || [];
while (logGroupsDescribe.nextToken) {
logStreamInput = { /* logGroupNamePrefix: 'game-ci',*/ nextToken: logGroupsDescribe.nextToken };
logGroupsDescribe = await ecs.describeLogGroups(logStreamInput).promise();
logGroups.push(...(logGroupsDescribe?.logGroups || []));
}
CloudRunnerLogger.log(`Log Groups ${logGroups.length}`);
for (const element of logGroups) {
if (element.creationTime === undefined) {
CloudRunnerLogger.log(`Skipping ${element.logGroupName} no createdAt date`);
continue;
}
const ageDate = new Date(new Date(element.creationTime).getTime() - Date.now());
if (verbose)
CloudRunnerLogger.log(
`Log Group Name ${
element.logGroupName
} - Age D${ageDate.getDay()} H${ageDate.getHours()} M${ageDate.getMinutes()} - 1d old ${AwsCliCommands.isOlderThan1day(
new Date(element.creationTime),
)}`,
);
if (perResultCallback) await perResultCallback(element, element);
}
}
private static async cleanup(deleteResources = false, OneDayOlderOnly: boolean = false) {
process.env.AWS_REGION = Input.region;
const CF = new AWS.CloudFormation();
const ecs = new AWS.ECS();
const cwl = new AWS.CloudWatchLogs();
await AwsCliCommands.awsListStacks(async (element) => {
if (deleteResources && (!OneDayOlderOnly || AwsCliCommands.isOlderThan1day(element.CreationTime))) {
if (element.StackName === 'game-ci' || element.TemplateDescription === 'Game-CI base stack') {
CloudRunnerLogger.log(`Skipping ${element.StackName} ignore list`);
return;
}
CloudRunnerLogger.log(`Deleting ${element.logGroupName}`);
const deleteStackInput: AWS.CloudFormation.DeleteStackInput = { StackName: element.StackName };
await CF.deleteStack(deleteStackInput).promise();
}
});
await AwsCliCommands.awsListTasks(async (taskElement, element) => {
if (deleteResources && (!OneDayOlderOnly || AwsCliCommands.isOlderThan1day(taskElement.CreatedAt))) {
CloudRunnerLogger.log(`Stopping task ${taskElement.containers?.[0].name}`);
await ecs.stopTask({ task: taskElement.taskArn || '', cluster: element }).promise();
}
});
await AwsCliCommands.awsListLogGroups(async (element) => {
if (deleteResources && (!OneDayOlderOnly || AwsCliCommands.isOlderThan1day(new Date(element.createdAt)))) {
CloudRunnerLogger.log(`Deleting ${element.logGroupName}`);
await cwl.deleteLogGroup({ logGroupName: element.logGroupName || '' }).promise();
}
});
}
}

View File

@@ -66,20 +66,24 @@ class AWSBuildEnvironment implements ProviderInterface {
);
let postRunTaskTimeMs;
let output = '';
try {
const postSetupStacksTimeMs = Date.now();
CloudRunnerLogger.log(`Setup job time: ${Math.floor((postSetupStacksTimeMs - startTimeMs) / 1000)}s`);
output = await AWSTaskRunner.runTask(taskDef, ECS, CF, environment, buildGuid, commands);
const { output, shouldCleanup } = await AWSTaskRunner.runTask(taskDef, ECS, CF, environment, buildGuid, commands);
postRunTaskTimeMs = Date.now();
CloudRunnerLogger.log(`Run job time: ${Math.floor((postRunTaskTimeMs - postSetupStacksTimeMs) / 1000)}s`);
} finally {
await this.cleanupResources(CF, taskDef);
if (shouldCleanup) {
await this.cleanupResources(CF, taskDef);
}
const postCleanupTimeMs = Date.now();
if (postRunTaskTimeMs !== undefined)
CloudRunnerLogger.log(`Cleanup job time: ${Math.floor((postCleanupTimeMs - postRunTaskTimeMs) / 1000)}s`);
return output;
} catch (error) {
await this.cleanupResources(CF, taskDef);
throw error;
}
return output;
}
async cleanupResources(CF: SDK.CloudFormation, taskDef: CloudRunnerAWSTaskDef) {

View File

@@ -77,7 +77,7 @@ class Kubernetes implements ProviderInterface {
secrets: CloudRunnerSecret[],
): Promise<string> {
try {
// setup
// Setup
this.buildGuid = buildGuid;
this.secretName = `build-credentials-${buildGuid}`;
this.jobName = `unity-builder-job-${buildGuid}`;
@@ -98,7 +98,7 @@ class Kubernetes implements ProviderInterface {
k8s,
);
//run
// Run
const jobResult = await this.kubeClientBatch.createNamespacedJob(this.namespace, jobSpec);
CloudRunnerLogger.log(`Creating build job ${JSON.stringify(jobResult.body.metadata, undefined, 4)}`);
@@ -119,7 +119,6 @@ class Kubernetes implements ProviderInterface {
this.podName,
'main',
this.namespace,
CloudRunnerLogger.log,
);
break;
} catch (error: any) {
@@ -131,6 +130,7 @@ class Kubernetes implements ProviderInterface {
}
}
await this.cleanupTaskResources();
return output;
} catch (error) {
CloudRunnerLogger.log('Running job failed');
@@ -163,6 +163,7 @@ class Kubernetes implements ProviderInterface {
async () => {
const jobBody = (await this.kubeClientBatch.readNamespacedJob(this.jobName, this.namespace)).body;
const podBody = (await this.kubeClient.readNamespacedPod(this.podName, this.namespace)).body;
return (jobBody === null || jobBody.status?.active === 0) && podBody === null;
},
{
@@ -195,6 +196,7 @@ class Kubernetes implements ProviderInterface {
if (pod === undefined) {
throw new Error("pod with job-name label doesn't exist");
}
return pod;
}
}

View File

@@ -108,8 +108,8 @@ class KubernetesJobSpecFactory {
workingDir: `${workingDirectory}`,
resources: {
requests: {
memory: buildParameters.cloudRunnerMemory,
cpu: buildParameters.cloudRunnerCpu,
memory: buildParameters.cloudRunnerMemory || '750M',
cpu: buildParameters.cloudRunnerCpu || '1',
},
},
env: [
@@ -117,6 +117,7 @@ class KubernetesJobSpecFactory {
const environmentVariable = new V1EnvVar();
environmentVariable.name = x.name;
environmentVariable.value = x.value;
return environmentVariable;
}),
...secrets.map((x) => {
@@ -127,6 +128,7 @@ class KubernetesJobSpecFactory {
const environmentVariable = new V1EnvVar();
environmentVariable.name = x.EnvironmentVariable;
environmentVariable.valueFrom = secret;
return environmentVariable;
}),
],
@@ -155,6 +157,7 @@ class KubernetesJobSpecFactory {
},
},
};
return job;
}
}

View File

@@ -21,6 +21,7 @@ class KubernetesSecret {
for (const buildSecret of secrets) {
secret.data[buildSecret.ParameterKey] = base64.encode(buildSecret.ParameterValue);
}
return kubeClient.createNamespacedSecret(namespace, secret);
}
}

View File

@@ -10,6 +10,7 @@ class KubernetesServiceAccount {
name: serviceAccountName,
};
serviceAccount.automountServiceAccountToken = false;
return kubeClient.createNamespacedServiceAccount(namespace, serviceAccount);
}
}

View File

@@ -4,6 +4,7 @@ import * as k8s from '@kubernetes/client-node';
import BuildParameters from '../../../build-parameters';
import CloudRunnerLogger from '../../services/cloud-runner-logger';
import YAML from 'yaml';
import { IncomingMessage } from 'http';
class KubernetesStorage {
public static async createPersistentVolumeClaim(
@@ -15,6 +16,7 @@ class KubernetesStorage {
if (buildParameters.kubeVolume) {
CloudRunnerLogger.log(buildParameters.kubeVolume);
pvcName = buildParameters.kubeVolume;
return;
}
const pvcList = (await kubeClient.listNamespacedPersistentVolumeClaim(namespace)).body.items.map(
@@ -27,6 +29,7 @@ class KubernetesStorage {
if (!buildParameters.isCliMode) {
core.setOutput('volume', pvcName);
}
return;
}
CloudRunnerLogger.log(`Creating PVC ${pvcName} (does not exist)`);
@@ -96,11 +99,12 @@ class KubernetesStorage {
YAML.parse(process.env.K8s_STORAGE_PVC_SPEC);
}
const result = await kubeClient.createNamespacedPersistentVolumeClaim(namespace, pvc);
return result;
}
private static async handleResult(
result: { response: import('http').IncomingMessage; body: k8s.V1PersistentVolumeClaim },
result: { response: IncomingMessage; body: k8s.V1PersistentVolumeClaim },
kubeClient: k8s.CoreV1Api,
namespace: string,
pvcName: string,

View File

@@ -4,7 +4,7 @@ import CloudRunnerLogger from '../../services/cloud-runner-logger';
import * as core from '@actions/core';
import { CloudRunnerStatics } from '../../cloud-runner-statics';
import waitUntil from 'async-wait-until';
import CloudRunner from '../../cloud-runner';
import { FollowLogStreamService } from '../../services/follow-log-stream-service';
class KubernetesTaskRunner {
static async runTask(
@@ -14,20 +14,23 @@ class KubernetesTaskRunner {
podName: string,
containerName: string,
namespace: string,
logCallback: any,
) {
CloudRunnerLogger.log(`Streaming logs from pod: ${podName} container: ${containerName} namespace: ${namespace}`);
const stream = new Writable();
let output = '';
let didStreamAnyLogs: boolean = false;
let shouldReadLogs = true;
let shouldCleanup = true;
stream._write = (chunk, encoding, next) => {
didStreamAnyLogs = true;
let message = chunk.toString().trimRight(`\n`);
message = `[${CloudRunnerStatics.logPrefix}] ${message}`;
if (CloudRunner.buildParameters.cloudRunnerIntegrationTests) {
output += message;
}
logCallback(message);
({ shouldReadLogs, shouldCleanup, output } = FollowLogStreamService.handleIteration(
message,
shouldReadLogs,
shouldCleanup,
output,
));
next();
};
const logOptions = {
@@ -73,6 +76,7 @@ class KubernetesTaskRunner {
throw error;
}
CloudRunnerLogger.log('end of log stream');
return output;
}
@@ -90,6 +94,7 @@ class KubernetesTaskRunner {
}`,
);
if (success || phase !== 'Pending') return true;
return false;
},
{
@@ -97,6 +102,7 @@ class KubernetesTaskRunner {
intervalBetweenAttempts: 15000,
},
);
return success;
}
}

View File

@@ -42,6 +42,7 @@ class LocalDockerCloudRunner implements ProviderInterface {
): Promise<string> {
CloudRunnerLogger.log(buildGuid);
CloudRunnerLogger.log(commands);
return CloudRunnerSystem.Run(commands, false, false);
}
}

View File

@@ -42,6 +42,7 @@ class LocalCloudRunner implements ProviderInterface {
CloudRunnerLogger.log(image);
CloudRunnerLogger.log(buildGuid);
CloudRunnerLogger.log(commands);
return await CloudRunnerSystem.Run(commands);
}
}

View File

@@ -41,6 +41,7 @@ class TestCloudRunner implements ProviderInterface {
CloudRunnerLogger.log(image);
CloudRunnerLogger.log(buildGuid);
CloudRunnerLogger.log(commands);
return await new Promise((result) => {
result(commands);
});

View File

@@ -26,19 +26,19 @@ describe('Cloud Runner Caching', () => {
const buildParameter = await BuildParameters.create();
CloudRunner.buildParameters = buildParameter;
// create test folder
// Create test folder
const testFolder = path.resolve(__dirname, Cli.options.cacheKey);
fs.mkdirSync(testFolder);
// crate cache folder
// Create cache folder
const cacheFolder = path.resolve(__dirname, `cache-${Cli.options.cacheKey}`);
fs.mkdirSync(cacheFolder);
// add test has file to test folders
// Add test file to test folders
fs.writeFileSync(path.resolve(testFolder, 'test.txt'), Cli.options.cacheKey);
await Caching.PushToCache(cacheFolder, testFolder, `${Cli.options.cacheKey}`);
// delete test folder
// Delete test folder
fs.rmdirSync(testFolder, { recursive: true });
await Caching.PullFromCache(
cacheFolder.replace(/\\/g, `/`),
@@ -49,7 +49,7 @@ describe('Cloud Runner Caching', () => {
await CloudRunnerSystem.Run(`tree ${testFolder}`);
await CloudRunnerSystem.Run(`tree ${cacheFolder}`);
// compare validity to original hash
// Compare validity to original hash
expect(fs.readFileSync(path.resolve(testFolder, 'test.txt'), { encoding: 'utf8' }).toString()).toContain(
Cli.options.cacheKey,
);

View File

@@ -65,21 +65,22 @@ export class Caching {
[path.resolve(sourceFolder, '..'), cacheFolder, cacheArtifactName],
1,
);
return format.replace(/{(\d+)}/g, function (match, number) {
return typeof arguments_[number] != 'undefined' ? arguments_[number] : match;
});
};
await CloudRunnerSystem.Run(`zip -q -r ${cacheArtifactName}.zip ${path.basename(sourceFolder)}`);
assert(await fileExists(`${cacheArtifactName}.zip`), 'cache zip exists');
await CloudRunnerSystem.Run(`tar -cf ${cacheArtifactName}.tar ${path.basename(sourceFolder)}`);
assert(await fileExists(`${cacheArtifactName}.tar`), 'cache archive exists');
assert(await fileExists(path.basename(sourceFolder)), 'source folder exists');
if (CloudRunner.buildParameters.cachePushOverrideCommand) {
await CloudRunnerSystem.Run(formatFunction(CloudRunner.buildParameters.cachePushOverrideCommand));
}
await CloudRunnerSystem.Run(`mv ${cacheArtifactName}.zip ${cacheFolder}`);
RemoteClientLogger.log(`moved ${cacheArtifactName}.zip to ${cacheFolder}`);
await CloudRunnerSystem.Run(`mv ${cacheArtifactName}.tar ${cacheFolder}`);
RemoteClientLogger.log(`moved cache entry ${cacheArtifactName} to ${cacheFolder}`);
assert(
await fileExists(`${path.join(cacheFolder, cacheArtifactName)}.zip`),
'cache zip exists inside cache folder',
await fileExists(`${path.join(cacheFolder, cacheArtifactName)}.tar`),
'cache archive exists inside cache folder',
);
} catch (error) {
process.chdir(`${startPath}`);
@@ -100,14 +101,14 @@ export class Caching {
await fs.promises.mkdir(destinationFolder);
}
const latestInBranch = await (await CloudRunnerSystem.Run(`ls -t "${cacheFolder}" | grep .zip$ | head -1`))
const latestInBranch = await (await CloudRunnerSystem.Run(`ls -t "${cacheFolder}" | grep .tar$ | head -1`))
.replace(/\n/g, ``)
.replace('.zip', '');
.replace('.tar', '');
process.chdir(cacheFolder);
const cacheSelection =
cacheArtifactName !== `` && (await fileExists(`${cacheArtifactName}.zip`)) ? cacheArtifactName : latestInBranch;
cacheArtifactName !== `` && (await fileExists(`${cacheArtifactName}.tar`)) ? cacheArtifactName : latestInBranch;
await CloudRunnerLogger.log(`cache key ${cacheArtifactName} selection ${cacheSelection}`);
// eslint-disable-next-line func-style
@@ -116,6 +117,7 @@ export class Caching {
[path.resolve(destinationFolder, '..'), cacheFolder, cacheArtifactName],
1,
);
return format.replace(/{(\d+)}/g, function (match, number) {
return typeof arguments_[number] != 'undefined' ? arguments_[number] : match;
});
@@ -125,12 +127,12 @@ export class Caching {
await CloudRunnerSystem.Run(formatFunction(CloudRunner.buildParameters.cachePullOverrideCommand));
}
if (await fileExists(`${cacheSelection}.zip`)) {
if (await fileExists(`${cacheSelection}.tar`)) {
const resultsFolder = `results${CloudRunner.buildParameters.buildGuid}`;
await CloudRunnerSystem.Run(`mkdir -p ${resultsFolder}`);
RemoteClientLogger.log(`cache item exists ${cacheFolder}/${cacheSelection}.zip`);
RemoteClientLogger.log(`cache item exists ${cacheFolder}/${cacheSelection}.tar`);
const fullResultsFolder = path.join(cacheFolder, resultsFolder);
await CloudRunnerSystem.Run(`unzip -q ${cacheSelection}.zip -d ${path.basename(resultsFolder)}`);
await CloudRunnerSystem.Run(`tar -xf ${cacheSelection}.tar -C ${fullResultsFolder}`);
RemoteClientLogger.log(`cache item extracted to ${fullResultsFolder}`);
assert(await fileExists(fullResultsFolder), `cache extraction results folder exists`);
const destinationParentFolder = path.resolve(destinationFolder, '..');
@@ -141,7 +143,6 @@ export class Caching {
await CloudRunnerSystem.Run(
`mv "${path.join(fullResultsFolder, path.basename(destinationFolder))}" "${destinationParentFolder}"`,
);
await CloudRunnerSystem.Run(`du -sh ${path.join(destinationParentFolder, path.basename(destinationFolder))}`);
const contents = await fs.promises.readdir(
path.join(destinationParentFolder, path.basename(destinationFolder)),
);
@@ -151,7 +152,7 @@ export class Caching {
} else {
RemoteClientLogger.logWarning(`cache item ${cacheArtifactName} doesn't exist ${destinationFolder}`);
if (cacheSelection !== ``) {
RemoteClientLogger.logWarning(`cache item ${cacheArtifactName}.zip doesn't exist ${destinationFolder}`);
RemoteClientLogger.logWarning(`cache item ${cacheArtifactName}.tar doesn't exist ${destinationFolder}`);
throw new Error(`Failed to get cache item, but cache hit was found: ${cacheSelection}`);
}
}

View File

@@ -45,8 +45,10 @@ export class RemoteClient {
}
private static async sizeOfFolder(message: string, folder: string) {
CloudRunnerLogger.log(`Size of ${message}`);
await CloudRunnerSystem.Run(`du -sh ${folder}`);
if (CloudRunner.buildParameters.cloudRunnerIntegrationTests) {
CloudRunnerLogger.log(`Size of ${message}`);
await CloudRunnerSystem.Run(`du -sh ${folder}`);
}
}
private static async cloneRepoWithoutLFSFiles() {

View File

@@ -30,6 +30,7 @@ export class CloudRunnerBuildCommandProcessor {
throw error;
}
}
return output.filter((x) => x.step !== undefined && x.hook !== undefined && x.hook.length > 0);
}
}

View File

@@ -4,7 +4,7 @@ import { CloudRunner } from '../..';
export class CloudRunnerFolders {
public static readonly repositoryFolder = 'repo';
// only the following paths that do not start a path.join with another "Full" suffixed property need to start with an absolute /
// Only the following paths that do not start a path.join with another "Full" suffixed property need to start with an absolute /
public static get uniqueCloudRunnerJobFolderAbsolute(): string {
return path.join(`/`, CloudRunnerFolders.buildVolumeFolder, CloudRunner.buildParameters.buildGuid);

View File

@@ -4,6 +4,7 @@ import CloudRunnerConstants from './cloud-runner-constants';
class CloudRunnerNamespace {
static generateGuid(runNumber: string | number, platform: string) {
const nanoid = customAlphabet(CloudRunnerConstants.alphabet, 4);
return `${runNumber}-${platform.toLowerCase().replace('standalone', '')}-${nanoid()}`;
}
}

View File

@@ -5,6 +5,7 @@ const formatFunction = (value, arguments_) => {
for (const element of arguments_) {
value = value.replace(`{${element.key}}`, element.value);
}
return value;
};
@@ -22,6 +23,7 @@ class CloudRunnerQueryOverride {
) {
return CloudRunnerQueryOverride.queryOverrides[alternativeKey];
}
return;
}
@@ -31,6 +33,7 @@ class CloudRunnerQueryOverride {
const doesInclude =
Input.readInputFromOverrideList().split(',').includes(query) ||
Input.readInputFromOverrideList().split(',').includes(Input.ToEnvVarFormat(query));
return doesInclude ? true : false;
} else {
return true;

View File

@@ -8,6 +8,7 @@ export class CloudRunnerSystem {
RemoteClientLogger.log(element);
}
}
return await new Promise<string>((promise, throwError) => {
let output = '';
const child = exec(command, (error, stdout, stderr) => {

View File

@@ -10,6 +10,7 @@ class DependencyOverrideService {
return false;
}
}
return true;
}
public static async TryStartDependencies() {

View File

@@ -0,0 +1,34 @@
import CloudRunnerLogger from './cloud-runner-logger';
import * as core from '@actions/core';
import CloudRunner from '../cloud-runner';
import { CloudRunnerStatics } from '../cloud-runner-statics';
export class FollowLogStreamService {
public static handleIteration(message, shouldReadLogs, shouldCleanup, output) {
if (message.includes(`---${CloudRunner.buildParameters.logId}`)) {
CloudRunnerLogger.log('End of log transmission received');
shouldReadLogs = false;
} else if (message.includes('Rebuilding Library because the asset database could not be found!')) {
core.warning('LIBRARY NOT FOUND!');
core.setOutput('library-found', 'false');
} else if (message.includes('Build succeeded')) {
core.setOutput('build-result', 'success');
} else if (message.includes('Build fail')) {
core.setOutput('build-result', 'failed');
core.setFailed('unity build failed');
core.error('BUILD FAILED!');
} else if (CloudRunner.buildParameters.cloudRunnerIntegrationTests && message.includes(': Listening for Jobs')) {
core.setOutput('cloud runner stop watching', 'true');
shouldReadLogs = false;
shouldCleanup = false;
core.warning('cloud runner stop watching');
}
message = `[${CloudRunnerStatics.logPrefix}] ${message}`;
if (CloudRunner.buildParameters.cloudRunnerIntegrationTests) {
output += message;
}
CloudRunnerLogger.log(message);
return { shouldReadLogs, shouldCleanup, output };
}
}

View File

@@ -22,6 +22,7 @@ export class LfsHashing {
.replace(' .lfs-assets-guid', '')
.replace(/\n/g, ``),
};
return lfsHashes;
} catch (error) {
throw error;
@@ -34,6 +35,7 @@ export class LfsHashing {
.replace(/\n/g, '')
.split(` `)[0];
process.chdir(startPath);
return result;
}

View File

@@ -40,8 +40,10 @@ export class TaskParameterSerializer {
array = array.map((x) => {
x.name = Input.ToEnvVarFormat(x.name);
x.value = `${x.value}`;
return x;
});
return array;
}
@@ -54,6 +56,7 @@ export class TaskParameterSerializer {
});
}
array.push({ name: 'buildParameters', value: JSON.stringify(CloudRunner.buildParameters) });
return array;
}
@@ -67,6 +70,7 @@ export class TaskParameterSerializer {
});
}
}
return array;
}
@@ -86,6 +90,7 @@ export class TaskParameterSerializer {
};
}),
);
return array;
}
private static getValue(key) {
@@ -104,6 +109,7 @@ export class TaskParameterSerializer {
ParameterValue: value,
});
}
return array;
}
}

View File

@@ -69,8 +69,9 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
(x) => x.step.includes(`build`),
);
const builderPath = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist', `index.js`).replace(/\\/g, `/`);
return `apt-get update > /dev/null
apt-get install -y zip tree npm git-lfs jq unzip git > /dev/null
apt-get install -y tar tree npm git-lfs jq git > /dev/null
npm install -g n > /dev/null
n stable > /dev/null
${setupHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
@@ -98,6 +99,7 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
const linuxCacheFolder = CloudRunnerFolders.cacheFolderFull.replace(/\\/g, `/`);
const distFolder = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist');
const ubuntuPlatformsFolder = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist', 'platforms', 'ubuntu');
return `echo "game ci cloud runner init"
mkdir -p ${`${CloudRunnerFolders.projectBuildFolderAbsolute}/build`.replace(/\\/g, `/`)}
cd ${CloudRunnerFolders.projectPathAbsolute}

View File

@@ -25,6 +25,7 @@ export class CustomWorkflow {
EnvironmentVariable: Input.ToEnvVarFormat(x.name),
ParameterValue: x.value,
};
return secret;
});
output += await CloudRunner.Provider.runTask(
@@ -37,6 +38,7 @@ export class CustomWorkflow {
[...CloudRunner.defaultSecrets, ...stepSecrets],
);
}
return output;
} catch (error) {
throw error;

View File

@@ -18,6 +18,7 @@ export class WorkflowCompositionRoot implements WorkflowInterface {
if (CloudRunner.buildParameters.customJob !== '') {
return await CustomWorkflow.runCustomJob(CloudRunner.buildParameters.customJob);
}
return await new BuildAutomationWorkflow().run(
new CloudRunnerStepState(baseImage, CloudRunner.cloudRunnerEnvironmentVariables, CloudRunner.defaultSecrets),
);

View File

@@ -46,6 +46,7 @@ class Docker {
static getWindowsCommand(image: any, parameters: any): string {
const { workspace, actionFolder, unitySerial, gitPrivateToken } = parameters;
return `docker run \
--workdir /github/workspace \
--rm \

View File

@@ -21,6 +21,7 @@ class ImageEnvironmentFactory {
string += `--env ${p.name}="${p.value}" `;
}
return string;
}
public static getEnvironmentVariables(parameters: BuildParameters) {
@@ -45,6 +46,7 @@ class ImageEnvironmentFactory {
{ name: 'ANDROID_KEYSTORE_PASS', value: parameters.androidKeystorePass },
{ name: 'ANDROID_KEYALIAS_NAME', value: parameters.androidKeyaliasName },
{ name: 'ANDROID_KEYALIAS_PASS', value: parameters.androidKeyaliasPass },
{ name: 'ANDROID_TARGET_SDK_VERSION', value: parameters.androidTargetSdkVersion },
{ name: 'ANDROID_SDK_MANAGER_PARAMETERS', value: parameters.androidSdkManagerParameters },
{ name: 'CUSTOM_PARAMETERS', value: parameters.customParameters },
{ name: 'CHOWN_FILES_TO', value: parameters.chownFilesTo },
@@ -65,6 +67,7 @@ class ImageEnvironmentFactory {
{ name: 'RUNNER_WORKSPACE', value: process.env.RUNNER_WORKSPACE },
];
if (parameters.sshAgent) environmentVariables.push({ name: 'SSH_AUTH_SOCK', value: '/ssh-agent' });
return environmentVariables;
}
}

View File

@@ -35,7 +35,7 @@ class ImageTag {
this.imagePlatformPrefix = ImageTag.getImagePlatformPrefixes(
isCloudRunnerLocal ? process.platform : cloudRunnerBuilderPlatform,
);
this.imageRollingVersion = 1; // will automatically roll to the latest non-breaking version.
this.imageRollingVersion = 1; // Will automatically roll to the latest non-breaking version.
}
static get versionPattern() {
@@ -75,6 +75,7 @@ class ImageTag {
ImageTag.targetPlatformSuffixes;
const [major, minor] = version.split('.').map((digit) => Number(digit));
// @see: https://docs.unity3d.com/ScriptReference/BuildTarget.html
switch (platform) {
case Platform.types.StandaloneOSX:
@@ -91,12 +92,14 @@ class ImageTag {
If you are trying to build for windows-mono, please use a Linux based OS.`);
}
}
return windows;
case Platform.types.StandaloneLinux64: {
// Unity versions before 2019.3 do not support il2cpp
if (major >= 2020 || (major === 2019 && minor >= 3)) {
return linuxIl2cpp;
}
return linux;
}
case Platform.types.iOS:
@@ -109,6 +112,7 @@ class ImageTag {
if (process.platform !== 'win32') {
throw new Error(`WSAPlayer can only be built on a windows base OS`);
}
return wsaPlayer;
case Platform.types.PS4:
return windows;
@@ -118,9 +122,11 @@ class ImageTag {
if (process.platform !== 'win32') {
throw new Error(`tvOS can only be built on a windows base OS`);
}
return tvos;
case Platform.types.Switch:
return windows;
// Unsupported
case Platform.types.Lumin:
return windows;

View File

@@ -1,7 +1,12 @@
import { CloudRunnerSystem } from '../cloud-runner/services/cloud-runner-system';
import Input from '../input';
export class GenericInputReader {
public static async Run(command) {
if (Input.cloudRunnerCluster === 'local') {
return '';
}
return await CloudRunnerSystem.Run(command, false, true);
}
}

View File

@@ -2,18 +2,27 @@ import { assert } from 'console';
import fs from 'fs';
import { CloudRunnerSystem } from '../cloud-runner/services/cloud-runner-system';
import CloudRunnerLogger from '../cloud-runner/services/cloud-runner-logger';
import Input from '../input';
export class GitRepoReader {
public static async GetRemote() {
if (Input.cloudRunnerCluster === 'local') {
return '';
}
assert(fs.existsSync(`.git`));
const value = (await CloudRunnerSystem.Run(`git remote -v`, false, true)).replace(/ /g, ``);
CloudRunnerLogger.log(`value ${value}`);
assert(value.includes('github.com'));
return value.split('github.com/')[1].split('.git')[0];
}
public static async GetBranch() {
if (Input.cloudRunnerCluster === 'local') {
return '';
}
assert(fs.existsSync(`.git`));
return (await CloudRunnerSystem.Run(`git branch --show-current`, false, true))
.split('\n')[0]
.replace(/ /g, ``)

View File

@@ -1,19 +1,25 @@
import { CloudRunnerSystem } from '../cloud-runner/services/cloud-runner-system';
import * as core from '@actions/core';
import Input from '../input';
export class GithubCliReader {
static async GetGitHubAuthToken() {
if (Input.cloudRunnerCluster === 'local') {
return '';
}
try {
const authStatus = await CloudRunnerSystem.Run(`gh auth status`, true, true);
if (authStatus.includes('You are not logged') || authStatus === '') {
return '';
}
return (await CloudRunnerSystem.Run(`gh auth status -t`, false, true))
.split(`Token: `)[1]
.replace(/ /g, '')
.replace(/\n/g, '');
} catch (error: any) {
core.info(error || 'Failed to get github auth token from gh cli');
return '';
}
}

View File

@@ -1,8 +1,13 @@
import path from 'path';
import fs from 'fs';
import YAML from 'yaml';
import Input from '../input';
export function ReadLicense() {
if (Input.cloudRunnerCluster === 'local') {
return '';
}
const pipelineFile = path.join(__dirname, `.github`, `workflows`, `cloud-runner-k8s-pipeline.yml`);
return fs.existsSync(pipelineFile) ? YAML.parse(fs.readFileSync(pipelineFile, 'utf8')).env.UNITY_LICENSE : '';
}

View File

@@ -25,7 +25,7 @@ class Input {
}
const alternativeQuery = Input.ToEnvVarFormat(query);
// query input sources
// Query input sources
if (Cli.query(query, alternativeQuery)) {
return Cli.query(query, alternativeQuery);
}
@@ -69,6 +69,7 @@ class Input {
if (Input.cloudRunnerCluster !== 'local') {
return 'linux';
}
return;
}
@@ -117,7 +118,7 @@ class Input {
}
static get buildMethod() {
return Input.getInput('buildMethod') || ''; // processed in docker file
return Input.getInput('buildMethod') || ''; // Processed in docker file
}
static get customParameters() {
@@ -228,15 +229,16 @@ class Input {
if (Cli.isCliMode) {
return Input.getInput('cloudRunnerCluster') || 'aws';
}
return Input.getInput('cloudRunnerCluster') || 'local';
}
static get cloudRunnerCpu() {
return Input.getInput('cloudRunnerCpu') || '1.0';
return Input.getInput('cloudRunnerCpu');
}
static get cloudRunnerMemory() {
return Input.getInput('cloudRunnerMemory') || '750M';
return Input.getInput('cloudRunnerMemory');
}
static get kubeConfig() {
@@ -275,6 +277,7 @@ class Input {
if (input.toUpperCase() === input) {
return input;
}
return input
.replace(/([A-Z])/g, ' $1')
.trim()

View File

@@ -1,5 +1,5 @@
import { BuildParameters } from '.';
import { SetupWindows, SetupMac } from './platform-setup/';
import { SetupMac, SetupWindows } from './platform-setup/';
import ValidateWindows from './platform-validation/validate-windows';
class PlatformSetup {
@@ -12,7 +12,8 @@ class PlatformSetup {
case 'darwin':
await SetupMac.setup(buildParameters, actionFolder);
break;
//Add other baseOS's here
// Add other baseOS's here
}
}
}

View File

@@ -13,8 +13,9 @@ class SetupWindows {
if (!fs.existsSync('c:/regkeys')) {
fs.mkdirSync('c:/regkeys');
}
// These all need the Windows 10 SDK
switch (targetPlatform) {
//These all need the Windows 10 SDK
case 'StandaloneWindows':
case 'StandaloneWindows64':
case 'WSAPlayer':

View File

@@ -32,7 +32,7 @@ class ValidateWindows {
}
private static checkForWin10SDK() {
//Check for Windows 10 SDK on runner
// Check for Windows 10 SDK on runner
const windows10SDKPathExists = fs.existsSync('C:/Program Files (x86)/Windows Kits');
if (!windows10SDKPathExists) {
throw new Error(`Windows 10 SDK not found in default location. Make sure
@@ -42,7 +42,7 @@ class ValidateWindows {
}
private static checkForVisualStudio() {
//Note: When upgrading to Server 2022, we will need to move to just "program files" since VS will be 64-bit
// Note: When upgrading to Server 2022, we will need to move to just "program files" since VS will be 64-bit
const visualStudioInstallPathExists = fs.existsSync('C:/Program Files (x86)/Microsoft Visual Studio');
const visualStudioDataPathExists = fs.existsSync('C:/ProgramData/Microsoft/VisualStudio');

View File

@@ -17,12 +17,14 @@ class Platform {
XboxOne: 'XboxOne',
tvOS: 'tvOS',
Switch: 'Switch',
// Unsupported
Lumin: 'Lumin',
BJM: 'BJM',
Stadia: 'Stadia',
Facebook: 'Facebook',
NoTarget: 'NoTarget',
// Test specific
Test: 'Test',
};

View File

@@ -34,6 +34,7 @@ describe('System', () => {
it('outputs info', async () => {
execSpy.mockImplementationOnce(async (input, _, options) => {
options?.listeners?.stdout?.(Buffer.from(input, 'utf8'));
return 0;
});

View File

@@ -10,6 +10,7 @@ export default class UnityVersioning {
if (unityVersion === 'auto') {
return UnityVersioning.read(projectPath);
}
return unityVersion;
}
@@ -18,6 +19,7 @@ export default class UnityVersioning {
if (!fs.existsSync(filePath)) {
throw new Error(`Project settings file not found at "${filePath}". Have you correctly set the projectPath?`);
}
return UnityVersioning.parse(fs.readFileSync(filePath, 'utf8'));
}
@@ -26,6 +28,7 @@ export default class UnityVersioning {
if (!matches || matches.length === 0) {
throw new Error(`Failed to parse version from "${projectVersionTxt}".`);
}
return matches[0];
}
}

View File

@@ -2,6 +2,7 @@ import * as core from '@actions/core';
import NotImplementedException from './error/not-implemented-exception';
import System from './system';
import Versioning from './versioning';
import { validVersionTagInputs, invalidVersionTagInputs } from './__data__/versions';
afterEach(() => {
jest.restoreAllMocks();
@@ -34,6 +35,26 @@ describe('Versioning', () => {
});
});
describe('grepCompatibleInputVersionRegex', () => {
// eslint-disable-next-line unicorn/consistent-function-scoping
const matchInputUsingGrep = async (input) => {
const output = await System.run('sh', undefined, {
input: Buffer.from(`echo '${input}' | grep -E '${Versioning.grepCompatibleInputVersionRegex}'`),
silent: true,
});
return output.trim();
};
it.concurrent.each(validVersionTagInputs)(`accepts valid tag input '%s'`, async (input) => {
expect(await matchInputUsingGrep(input)).toStrictEqual(input);
});
it.concurrent.each(invalidVersionTagInputs)(`rejects non-version tag input '%s'`, async (input) => {
await expect(async () => matchInputUsingGrep(input)).rejects.toThrowError(/^Failed to run/);
});
});
describe('branch', () => {
it('returns headRef when set', () => {
const headReference = jest.spyOn(Versioning, 'headRef', 'get').mockReturnValue('feature-branch-1');
@@ -110,6 +131,7 @@ describe('Versioning', () => {
expect(logDiffSpy).toHaveBeenCalledTimes(1);
expect(gitSpy).toHaveBeenCalledTimes(1);
// Todo - this no longer works since typescript
// const issuedCommand = System.run.mock.calls[0][2].input.toString();
// expect(issuedCommand.indexOf('diff')).toBeGreaterThan(-1);
@@ -137,6 +159,7 @@ describe('Versioning', () => {
test.each(['v0', 'v0.1', 'v0.1.2', 'v0.1-2', 'v0.1-2-g'])('does not like %s', (description) => {
expect(Versioning.descriptionRegex1.test(description)).toBeFalsy();
// Also, never expect without the v to work for any of these cases.
expect(Versioning.descriptionRegex1.test(description?.slice(1))).toBeFalsy();
});

View File

@@ -17,6 +17,10 @@ export default class Versioning {
return { None: 'None', Semantic: 'Semantic', Tag: 'Tag', Custom: 'Custom' };
}
static get grepCompatibleInputVersionRegex() {
return '^v?([0-9]+\\.)*[0-9]+.*';
}
/**
* Get the branch name of the (related) branch
*/
@@ -130,6 +134,7 @@ export default class Versioning {
if (!(await this.hasAnyVersionTags())) {
const version = `0.0.${await this.getTotalNumberOfCommits()}`;
core.info(`Generated version ${version} (no version tags found).`);
return version;
}
@@ -148,6 +153,7 @@ export default class Versioning {
const version = `0.0.${await this.getTotalNumberOfCommits()}`;
core.info(`Generated version ${version} (semantic version couldn't be determined).`);
return version;
}
@@ -203,6 +209,7 @@ export default class Versioning {
core.warning(
`Failed to parse git describe output or version can not be determined through: "${description}".`,
);
return false;
}
}
@@ -269,18 +276,20 @@ export default class Versioning {
}
/**
* Whether or not the repository has any version tags yet.
* Whether the current tree has any version tags yet.
*
* Note: Currently this is run in all OSes, so the syntax must be cross-platform.
*/
static async hasAnyVersionTags() {
const numberOfCommitsAsString = await System.run('sh', undefined, {
input: Buffer.from('git tag --list --merged HEAD | grep v[0-9]* | wc -l'),
const numberOfTagsAsString = await System.run('sh', undefined, {
input: Buffer.from(`git tag --list --merged HEAD | grep -E '${this.grepCompatibleInputVersionRegex}' | wc -l`),
cwd: this.projectPath,
silent: false,
});
const numberOfCommits = Number.parseInt(numberOfCommitsAsString, 10);
const numberOfTags = Number.parseInt(numberOfTagsAsString, 10);
return numberOfCommits !== 0;
return numberOfTags !== 0;
}
/**

View File

@@ -9,5 +9,5 @@
"noImplicitAny": false /* Re-enable after fixing compatibility */ /* Raise error on expressions and declarations with an implied 'any' type. */,
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
},
"exclude": ["node_modules", "**/*.test.ts"]
"exclude": ["node_modules", "dist"]
}

View File

@@ -2,12 +2,13 @@
# yarn lockfile v1
"@actions/core@^1.6.0":
version "1.6.0"
resolved "https://registry.yarnpkg.com/@actions/core/-/core-1.6.0.tgz#0568e47039bfb6a9170393a73f3b7eb3b22462cb"
integrity sha512-NB1UAZomZlCV/LmJqkLhNTqtKfFXJZAUPcfl/zqG7EfsQdeUJtaWO98SGbuQ3pydJ3fHl2CvI/51OKYlCYYcaw==
"@actions/core@^1.10.0":
version "1.10.0"
resolved "https://registry.yarnpkg.com/@actions/core/-/core-1.10.0.tgz#44551c3c71163949a2f06e94d9ca2157a0cfac4f"
integrity sha512-2aZDDa3zrrZbP5ZYg159sNoLRb61nQ7awl5pSvIq5Qpj81vwDzdMRKzkWJGJuwVvWpvZKx7vspJALyvaaIQyug==
dependencies:
"@actions/http-client" "^1.0.11"
"@actions/http-client" "^2.0.1"
uuid "^8.3.2"
"@actions/exec@^1.1.0":
version "1.1.0"
@@ -33,6 +34,13 @@
dependencies:
tunnel "0.0.6"
"@actions/http-client@^2.0.1":
version "2.0.1"
resolved "https://registry.yarnpkg.com/@actions/http-client/-/http-client-2.0.1.tgz#873f4ca98fe32f6839462a6f046332677322f99c"
integrity sha512-PIXiMVtz6VvyaRsGY268qvj57hXQEpsYogYOu2nrQhlf+XCGmZstmuZBbAybUl1nQGnvS1k1eEsQ69ZoD7xlSw==
dependencies:
tunnel "^0.0.6"
"@actions/io@^1.0.1":
version "1.1.0"
resolved "https://registry.npmjs.org/@actions/io/-/io-1.1.0.tgz"
@@ -3656,9 +3664,9 @@ jmespath@0.16.0:
integrity sha512-9FzQjJ7MATs1tSpnco1K6ayiYE3figslrXA72G2HQ/n76RzvYlofyi5QM+iX4YRs/pu3yzxlVQSST23+dMDknw==
jose@^1.27.1:
version "1.28.1"
resolved "https://registry.npmjs.org/jose/-/jose-1.28.1.tgz"
integrity sha512-6JK28rFu5ENp/yxMwM+iN7YeaInnY9B9Bggjkz5fuwLiJhbVrl2O4SJr65bdNBPl9y27fdC3Mymh+FVCvozLIg==
version "1.28.2"
resolved "https://registry.yarnpkg.com/jose/-/jose-1.28.2.tgz#97f4aa608d0020ae5c1051a2a33247b957401e5a"
integrity sha512-wWy51U2MXxYi3g8zk2lsQ8M6O1lartpkxuq1TYexzPKYLgHLZkCjklaATP36I5BUoWjF2sInB9U1Qf18fBZxNA==
dependencies:
"@panva/asn1.js" "^1.0.0"
@@ -5244,7 +5252,7 @@ tunnel-agent@^0.6.0:
dependencies:
safe-buffer "^5.0.1"
tunnel@0.0.6:
tunnel@0.0.6, tunnel@^0.0.6:
version "0.0.6"
resolved "https://registry.npmjs.org/tunnel/-/tunnel-0.0.6.tgz"
integrity sha512-1h/Lnq9yajKY2PEbBadPXj3VxsDDu844OnaAo52UVmIzIvwwtBPIuNvkjuzBlTWpfJyUbG3ez0KSBibQkj4ojg==