mirror of
https://github.com/game-ci/unity-builder.git
synced 2026-02-05 08:59:09 +08:00
* Update cloud-runner-aws-pipeline.yml * Update cloud-runner-k8s-pipeline.yml * yarn build * yarn build * correct branch ref * correct branch ref passed to target repo * Create k8s-tests.yml * Delete k8s-tests.yml * correct branch ref passed to target repo * correct branch ref passed to target repo * Always describe AWS tasks for now, because unstable error handling * Remove unused tree commands * Use lfs guid sum * Simple override cache push * Simple override cache push and pull override to allow pure cloud storage driven caching * Removal of early branch (breaks lfs caching) * Remove unused tree commands * Update action.yml * Update action.yml * Support cache and input override commands as input + full support custom hooks * Increase k8s timeout * replace filename being appended for unknclear reason * cache key should not contain whitespaces * Always try and deploy rook for k8s * Apply k8s files for rook * Update action.yml * Apply k8s files for rook * Apply k8s files for rook * cache test and action description for kuber storage class * Correct test and implement dependency health check and start * GCP-secret run, cache key * lfs smudge set explicit and undo explicit * Run using external secret provider to speed up input * Update cloud-runner-aws-pipeline.yml * Add nodejs as build step dependency * Add nodejs as build step dependency * Cloud Runner Tests must be specified to capture logs from cloud runner for tests * Cloud Runner Tests must be specified to capture logs from cloud runner for tests * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs * better defaults for new inputs * better defaults * merge latest * force build update * use npm n to update node in unity builder * use npm n to update node in unity builder * use npm n to update node in unity builder * correct new line * quiet zipping * quiet zipping * default secrets for unity username and password * default secrets for unity username and password * ls active directory before lfs install * Get cloud runner secrets from * Get cloud runner secrets from * Cleanup setup of default secrets * Various fixes * Cleanup setup of default secrets * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * Various fixes * AWS secrets manager support * less caching logs * default k8s storage class to pd-standard * more readable build commands * Capture aws exit code 1 reliably * Always replace /head from branch * k8s default storage class to standard-rwo * cleanup * further cleanup input * further cleanup input * further cleanup input * further cleanup input * further cleanup input * folder sizes to inspect caching * dir command for local cloud runner test * k8s wait for pending because pvc will not create earlier * prefer k8s standard storage * handle empty string as cloud runner cluster input * local-system is now used for cloud runner test implementation AND correctly unset test CLI input * local-system is now used for cloud runner test implementation AND correctly unset test CLI input * fix unterminated quote * fix unterminated quote * do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name * remove head and heads from branch prefix * fix reversed caching direction of cache-push * fixes * fixes * fixes * cachePull cli * fixes * fixes * fixes * fixes * fixes * order cache test to be first * order cache test to be first * fixes * populate cache key instead of using branch * cleanup cli * garbage-collect-aws cli can iterate over aws resources and cli scans all ts files * import cli methods * import cli files explicitly * import cli files explicitly * import cli files explicitly * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * import cli methods * log parameters in cloud runner parameter test * log parameters in cloud runner parameter test * log parameters in cloud runner parameter test * Cloud runner param test before caching because we have a fast local cache test now * Using custom build path relative to repo root rather than project root * aws-garbage-collect at end of pipeline * aws-garbage-collect do not actually delete anything for now - just list * remove some legacy du commands * Update cloud-runner-aws-pipeline.yml * log contents after cache pull and fix some scenarios with duplicate secrets * log contents after cache pull and fix some scenarios with duplicate secrets * log contents after cache pull and fix some scenarios with duplicate secrets * PR comments * Replace guid with uuid package * use fileExists lambda instead of stat to check file exists in caching * build failed results in core error message * Delete sample.txt * cloud-runner-system prefix changed to cloud-runner * Update cloud-runner-aws-pipeline.yml * remove du from caching, should run manually if interested in size, adds too much runtime to job to include by default * github ephemeral pipeline support * github ephemeral pipeline support * Merge remote-tracking branch 'origin/main' into cloud-runner-develop # Conflicts: # dist/index.js.map # src/model/cloud-runner/providers/aws/aws-task-runner.ts # src/model/cloud-runner/providers/aws/index.ts * garbage collection * garbage collection * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * self hosted runner pipeline * ephemeral runner pipeline * ephemeral runner pipeline * ephemeral runner pipeline * download runner each time * download runner each time * download runner each time * garbage collect all older than 1d as part of cleanup * download runner each time * number container cpu and memory for aws * per provider container defaults * per provider container defaults * per provider container defaults * per provider container defaults * Skip printing size unless cloudRunnerIntegrationTests is true * transition zip usage in cache to uncompressed tar for speed * transition zip usage in cache to uncompressed tar for speed * transition zip usage in cache to uncompressed tar for speed * transition zip usage in cache to uncompressed tar for speed * per provider container defaults * per provider container defaults * per provider container defaults * per provider container defaults * per provider container defaults * per provider container defaults * per provider container defaults * per provider container defaults * baked in cloud formation template * baked in cloud formation template * baked in cloud formation template * baked in cloud formation template * baked in cloud formation template * baked in cloud formation template * baked in cloud formation template * baked in cloud formation template * better aws commands * better aws commands * parse number for cloud formation template * remove container resource defaults from actions yaml * remove container resource defaults from actions yaml * skip all input readers when cloud runner is local * prefer fs/promises * actually set aws cloud runner step as failure if unity build fails * default to 3gb of ram - webgl fails on 2
124 lines
5.8 KiB
TypeScript
124 lines
5.8 KiB
TypeScript
import CloudRunnerLogger from '../services/cloud-runner-logger';
|
|
import { CloudRunnerFolders } from '../services/cloud-runner-folders';
|
|
import { CloudRunnerStepState } from '../cloud-runner-step-state';
|
|
import { CustomWorkflow } from './custom-workflow';
|
|
import { WorkflowInterface } from './workflow-interface';
|
|
import * as core from '@actions/core';
|
|
import { CloudRunnerBuildCommandProcessor } from '../services/cloud-runner-build-command-process';
|
|
import path from 'path';
|
|
import CloudRunner from '../cloud-runner';
|
|
|
|
export class BuildAutomationWorkflow implements WorkflowInterface {
|
|
async run(cloudRunnerStepState: CloudRunnerStepState) {
|
|
try {
|
|
return await BuildAutomationWorkflow.standardBuildAutomation(cloudRunnerStepState.image);
|
|
} catch (error) {
|
|
throw error;
|
|
}
|
|
}
|
|
|
|
private static async standardBuildAutomation(baseImage: any) {
|
|
try {
|
|
CloudRunnerLogger.log(`Cloud Runner is running standard build automation`);
|
|
|
|
if (!CloudRunner.buildParameters.isCliMode) core.startGroup('pre build steps');
|
|
let output = '';
|
|
if (CloudRunner.buildParameters.preBuildSteps !== '') {
|
|
output += await CustomWorkflow.runCustomJob(CloudRunner.buildParameters.preBuildSteps);
|
|
}
|
|
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();
|
|
CloudRunnerLogger.logWithTime('Configurable pre build step(s) time');
|
|
|
|
if (!CloudRunner.buildParameters.isCliMode) core.startGroup('build');
|
|
CloudRunnerLogger.log(baseImage.toString());
|
|
CloudRunnerLogger.logLine(` `);
|
|
CloudRunnerLogger.logLine('Starting build automation job');
|
|
|
|
output += await CloudRunner.Provider.runTask(
|
|
CloudRunner.buildParameters.buildGuid,
|
|
baseImage.toString(),
|
|
BuildAutomationWorkflow.BuildWorkflow,
|
|
`/${CloudRunnerFolders.buildVolumeFolder}`,
|
|
`/${CloudRunnerFolders.buildVolumeFolder}/`,
|
|
CloudRunner.cloudRunnerEnvironmentVariables,
|
|
CloudRunner.defaultSecrets,
|
|
);
|
|
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();
|
|
CloudRunnerLogger.logWithTime('Build time');
|
|
|
|
if (!CloudRunner.buildParameters.isCliMode) core.startGroup('post build steps');
|
|
if (CloudRunner.buildParameters.postBuildSteps !== '') {
|
|
output += await CustomWorkflow.runCustomJob(CloudRunner.buildParameters.postBuildSteps);
|
|
}
|
|
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();
|
|
CloudRunnerLogger.logWithTime('Configurable post build step(s) time');
|
|
|
|
CloudRunnerLogger.log(`Cloud Runner finished running standard build automation`);
|
|
|
|
return output;
|
|
} catch (error) {
|
|
throw error;
|
|
}
|
|
}
|
|
|
|
private static get BuildWorkflow() {
|
|
const setupHooks = CloudRunnerBuildCommandProcessor.getHooks(CloudRunner.buildParameters.customJobHooks).filter(
|
|
(x) => x.step.includes(`setup`),
|
|
);
|
|
const buildHooks = CloudRunnerBuildCommandProcessor.getHooks(CloudRunner.buildParameters.customJobHooks).filter(
|
|
(x) => x.step.includes(`build`),
|
|
);
|
|
const builderPath = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist', `index.js`).replace(/\\/g, `/`);
|
|
|
|
return `apt-get update > /dev/null
|
|
apt-get install -y tar tree npm git-lfs jq git > /dev/null
|
|
npm install -g n > /dev/null
|
|
n stable > /dev/null
|
|
${setupHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
|
|
export GITHUB_WORKSPACE="${CloudRunnerFolders.repoPathAbsolute.replace(/\\/g, `/`)}"
|
|
${BuildAutomationWorkflow.setupCommands(builderPath)}
|
|
${setupHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}
|
|
${buildHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
|
|
${BuildAutomationWorkflow.BuildCommands(builderPath, CloudRunner.buildParameters.buildGuid)}
|
|
${buildHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}`;
|
|
}
|
|
|
|
private static setupCommands(builderPath) {
|
|
return `export GIT_DISCOVERY_ACROSS_FILESYSTEM=1
|
|
echo "game ci cloud runner clone"
|
|
mkdir -p ${CloudRunnerFolders.builderPathAbsolute.replace(/\\/g, `/`)}
|
|
git clone -q -b ${CloudRunner.buildParameters.cloudRunnerBranch} ${
|
|
CloudRunnerFolders.unityBuilderRepoUrl
|
|
} "${CloudRunnerFolders.builderPathAbsolute.replace(/\\/g, `/`)}"
|
|
chmod +x ${builderPath}
|
|
echo "game ci cloud runner bootstrap"
|
|
node ${builderPath} -m remote-cli`;
|
|
}
|
|
|
|
private static BuildCommands(builderPath, guid) {
|
|
const linuxCacheFolder = CloudRunnerFolders.cacheFolderFull.replace(/\\/g, `/`);
|
|
const distFolder = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist');
|
|
const ubuntuPlatformsFolder = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist', 'platforms', 'ubuntu');
|
|
|
|
return `echo "game ci cloud runner init"
|
|
mkdir -p ${`${CloudRunnerFolders.projectBuildFolderAbsolute}/build`.replace(/\\/g, `/`)}
|
|
cd ${CloudRunnerFolders.projectPathAbsolute}
|
|
cp -r "${path.join(distFolder, 'default-build-script').replace(/\\/g, `/`)}" "/UnityBuilderAction"
|
|
cp -r "${path.join(ubuntuPlatformsFolder, 'entrypoint.sh').replace(/\\/g, `/`)}" "/entrypoint.sh"
|
|
cp -r "${path.join(ubuntuPlatformsFolder, 'steps').replace(/\\/g, `/`)}" "/steps"
|
|
chmod -R +x "/entrypoint.sh"
|
|
chmod -R +x "/steps"
|
|
echo "game ci cloud runner start"
|
|
/entrypoint.sh
|
|
echo "game ci cloud runner push library to cache"
|
|
chmod +x ${builderPath}
|
|
node ${builderPath} -m cache-push --cachePushFrom ${
|
|
CloudRunnerFolders.libraryFolderAbsolute
|
|
} --artifactName lib-${guid} --cachePushTo ${linuxCacheFolder}/Library
|
|
echo "game ci cloud runner push build to cache"
|
|
node ${builderPath} -m cache-push --cachePushFrom ${
|
|
CloudRunnerFolders.projectBuildFolderAbsolute
|
|
} --artifactName build-${guid} --cachePushTo ${`${linuxCacheFolder}/build`.replace(/\\/g, `/`)}`;
|
|
}
|
|
}
|