Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml
* Update cloud-runner-k8s-pipeline.yml
* yarn build
* yarn build
* correct branch ref
* correct branch ref passed to target repo
* Create k8s-tests.yml
* Delete k8s-tests.yml
* correct branch ref passed to target repo
* correct branch ref passed to target repo
* Always describe AWS tasks for now, because unstable error handling
* Remove unused tree commands
* Use lfs guid sum
* Simple override cache push
* Simple override cache push and pull override to allow pure cloud storage driven caching
* Removal of early branch (breaks lfs caching)
* Remove unused tree commands
* Update action.yml
* Update action.yml
* Support cache and input override commands as input + full support custom hooks
* Increase k8s timeout
* replace filename being appended for unknclear reason
* cache key should not contain whitespaces
* Always try and deploy rook for k8s
* Apply k8s files for rook
* Update action.yml
* Apply k8s files for rook
* Apply k8s files for rook
* cache test and action description for kuber storage class
* Correct test and implement dependency health check and start
* GCP-secret run, cache key
* lfs smudge set explicit and undo explicit
* Run using external secret provider to speed up input
* Update cloud-runner-aws-pipeline.yml
* Add nodejs as build step dependency
* Add nodejs as build step dependency
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* better defaults for new inputs
* better defaults
* merge latest
* force build update
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* correct new line
* quiet zipping
* quiet zipping
* default secrets for unity username and password
* default secrets for unity username and password
* ls active directory before lfs install
* Get cloud runner secrets from
* Get cloud runner secrets from
* Cleanup setup of default secrets
* Various fixes
* Cleanup setup of default secrets
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* AWS secrets manager support
* less caching logs
* default k8s storage class to pd-standard
* more readable build commands
* Capture aws exit code 1 reliably
* Always replace /head from branch
* k8s default storage class to standard-rwo
* cleanup
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* folder sizes to inspect caching
* dir command for local cloud runner test
* k8s wait for pending because pvc will not create earlier
* prefer k8s standard storage
* handle empty string as cloud runner cluster input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* fix unterminated quote
* fix unterminated quote
* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name
* remove head and heads from branch prefix
* fix reversed caching direction of cache-push
* fixes
* fixes
* fixes
* cachePull cli
* fixes
* fixes
* fixes
* fixes
* fixes
* order cache test to be first
* order cache test to be first
* fixes
* populate cache key instead of using branch
* cleanup cli
* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files
* import cli methods
* import cli files explicitly
* import cli files explicitly
* import cli files explicitly
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* Cloud runner param test before caching because we have a fast local cache test now
* Using custom build path relative to repo root rather than project root
* aws-garbage-collect at end of pipeline
* aws-garbage-collect do not actually delete anything for now - just list
* remove some legacy du commands
* Update cloud-runner-aws-pipeline.yml
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* PR comments
* Replace guid with uuid package
* use fileExists lambda instead of stat to check file exists in caching
* build failed results in core error message
* Delete sample.txt
2022-04-10 23:00:37 +00:00
|
|
|
import BuildParameters from '../../../build-parameters';
|
|
|
|
|
import CloudRunnerEnvironmentVariable from '../../services/cloud-runner-environment-variable';
|
|
|
|
|
import CloudRunnerLogger from '../../services/cloud-runner-logger';
|
|
|
|
|
import { ProviderInterface } from '../provider-interface';
|
|
|
|
|
import CloudRunnerSecret from '../../services/cloud-runner-secret';
|
2022-09-17 08:05:43 +00:00
|
|
|
import Docker from '../../../docker';
|
2022-10-18 22:00:28 +00:00
|
|
|
import { Action } from '../../..';
|
2022-09-21 21:00:42 +00:00
|
|
|
import { writeFileSync } from 'fs';
|
2022-10-08 03:23:24 +00:00
|
|
|
import CloudRunner from '../../cloud-runner';
|
2022-10-18 23:35:10 +00:00
|
|
|
import { ProviderResource } from '../provider-resource';
|
|
|
|
|
import { ProviderWorkflow } from '../provider-workflow';
|
2022-02-01 02:31:20 +00:00
|
|
|
|
Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml
* Update cloud-runner-k8s-pipeline.yml
* yarn build
* yarn build
* correct branch ref
* correct branch ref passed to target repo
* Create k8s-tests.yml
* Delete k8s-tests.yml
* correct branch ref passed to target repo
* correct branch ref passed to target repo
* Always describe AWS tasks for now, because unstable error handling
* Remove unused tree commands
* Use lfs guid sum
* Simple override cache push
* Simple override cache push and pull override to allow pure cloud storage driven caching
* Removal of early branch (breaks lfs caching)
* Remove unused tree commands
* Update action.yml
* Update action.yml
* Support cache and input override commands as input + full support custom hooks
* Increase k8s timeout
* replace filename being appended for unknclear reason
* cache key should not contain whitespaces
* Always try and deploy rook for k8s
* Apply k8s files for rook
* Update action.yml
* Apply k8s files for rook
* Apply k8s files for rook
* cache test and action description for kuber storage class
* Correct test and implement dependency health check and start
* GCP-secret run, cache key
* lfs smudge set explicit and undo explicit
* Run using external secret provider to speed up input
* Update cloud-runner-aws-pipeline.yml
* Add nodejs as build step dependency
* Add nodejs as build step dependency
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* better defaults for new inputs
* better defaults
* merge latest
* force build update
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* correct new line
* quiet zipping
* quiet zipping
* default secrets for unity username and password
* default secrets for unity username and password
* ls active directory before lfs install
* Get cloud runner secrets from
* Get cloud runner secrets from
* Cleanup setup of default secrets
* Various fixes
* Cleanup setup of default secrets
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* AWS secrets manager support
* less caching logs
* default k8s storage class to pd-standard
* more readable build commands
* Capture aws exit code 1 reliably
* Always replace /head from branch
* k8s default storage class to standard-rwo
* cleanup
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* folder sizes to inspect caching
* dir command for local cloud runner test
* k8s wait for pending because pvc will not create earlier
* prefer k8s standard storage
* handle empty string as cloud runner cluster input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* fix unterminated quote
* fix unterminated quote
* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name
* remove head and heads from branch prefix
* fix reversed caching direction of cache-push
* fixes
* fixes
* fixes
* cachePull cli
* fixes
* fixes
* fixes
* fixes
* fixes
* order cache test to be first
* order cache test to be first
* fixes
* populate cache key instead of using branch
* cleanup cli
* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files
* import cli methods
* import cli files explicitly
* import cli files explicitly
* import cli files explicitly
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* Cloud runner param test before caching because we have a fast local cache test now
* Using custom build path relative to repo root rather than project root
* aws-garbage-collect at end of pipeline
* aws-garbage-collect do not actually delete anything for now - just list
* remove some legacy du commands
* Update cloud-runner-aws-pipeline.yml
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* PR comments
* Replace guid with uuid package
* use fileExists lambda instead of stat to check file exists in caching
* build failed results in core error message
* Delete sample.txt
2022-04-10 23:00:37 +00:00
|
|
|
class LocalDockerCloudRunner implements ProviderInterface {
|
2022-09-20 22:14:08 +00:00
|
|
|
public buildParameters: BuildParameters | undefined;
|
|
|
|
|
|
2022-10-18 23:35:10 +00:00
|
|
|
listResources(): Promise<ProviderResource[]> {
|
2022-10-18 22:39:20 +00:00
|
|
|
throw new Error('Method not implemented.');
|
|
|
|
|
}
|
2022-10-18 23:35:10 +00:00
|
|
|
inspectResources(): Promise<ProviderResource> {
|
2022-10-18 22:39:20 +00:00
|
|
|
throw new Error('Method not implemented.');
|
|
|
|
|
}
|
2022-10-18 23:35:10 +00:00
|
|
|
listWorkflow(): Promise<ProviderWorkflow[]> {
|
2022-09-16 18:48:40 +00:00
|
|
|
throw new Error('Method not implemented.');
|
|
|
|
|
}
|
2022-10-18 23:35:10 +00:00
|
|
|
inspectWorkflow(): Promise<ProviderWorkflow> {
|
2022-09-16 18:48:40 +00:00
|
|
|
throw new Error('Method not implemented.');
|
|
|
|
|
}
|
2022-10-18 23:35:10 +00:00
|
|
|
watchWorkflow(): Promise<string> {
|
2022-09-16 18:48:40 +00:00
|
|
|
throw new Error('Method not implemented.');
|
|
|
|
|
}
|
|
|
|
|
garbageCollect(
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
filter: string,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
previewOnly: boolean,
|
2022-10-18 22:00:28 +00:00
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
olderThan: Number,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
fullCache: boolean,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
baseDependencies: boolean,
|
2022-09-16 18:48:40 +00:00
|
|
|
): Promise<string> {
|
|
|
|
|
throw new Error('Method not implemented.');
|
|
|
|
|
}
|
2022-10-18 22:10:08 +00:00
|
|
|
cleanupWorkflow(
|
2022-02-01 02:31:20 +00:00
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
buildGuid: string,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
buildParameters: BuildParameters,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
branchName: string,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
|
Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml
* Update cloud-runner-k8s-pipeline.yml
* yarn build
* yarn build
* correct branch ref
* correct branch ref passed to target repo
* Create k8s-tests.yml
* Delete k8s-tests.yml
* correct branch ref passed to target repo
* correct branch ref passed to target repo
* Always describe AWS tasks for now, because unstable error handling
* Remove unused tree commands
* Use lfs guid sum
* Simple override cache push
* Simple override cache push and pull override to allow pure cloud storage driven caching
* Removal of early branch (breaks lfs caching)
* Remove unused tree commands
* Update action.yml
* Update action.yml
* Support cache and input override commands as input + full support custom hooks
* Increase k8s timeout
* replace filename being appended for unknclear reason
* cache key should not contain whitespaces
* Always try and deploy rook for k8s
* Apply k8s files for rook
* Update action.yml
* Apply k8s files for rook
* Apply k8s files for rook
* cache test and action description for kuber storage class
* Correct test and implement dependency health check and start
* GCP-secret run, cache key
* lfs smudge set explicit and undo explicit
* Run using external secret provider to speed up input
* Update cloud-runner-aws-pipeline.yml
* Add nodejs as build step dependency
* Add nodejs as build step dependency
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* better defaults for new inputs
* better defaults
* merge latest
* force build update
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* correct new line
* quiet zipping
* quiet zipping
* default secrets for unity username and password
* default secrets for unity username and password
* ls active directory before lfs install
* Get cloud runner secrets from
* Get cloud runner secrets from
* Cleanup setup of default secrets
* Various fixes
* Cleanup setup of default secrets
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* AWS secrets manager support
* less caching logs
* default k8s storage class to pd-standard
* more readable build commands
* Capture aws exit code 1 reliably
* Always replace /head from branch
* k8s default storage class to standard-rwo
* cleanup
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* folder sizes to inspect caching
* dir command for local cloud runner test
* k8s wait for pending because pvc will not create earlier
* prefer k8s standard storage
* handle empty string as cloud runner cluster input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* fix unterminated quote
* fix unterminated quote
* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name
* remove head and heads from branch prefix
* fix reversed caching direction of cache-push
* fixes
* fixes
* fixes
* cachePull cli
* fixes
* fixes
* fixes
* fixes
* fixes
* order cache test to be first
* order cache test to be first
* fixes
* populate cache key instead of using branch
* cleanup cli
* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files
* import cli methods
* import cli files explicitly
* import cli files explicitly
* import cli files explicitly
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* Cloud runner param test before caching because we have a fast local cache test now
* Using custom build path relative to repo root rather than project root
* aws-garbage-collect at end of pipeline
* aws-garbage-collect do not actually delete anything for now - just list
* remove some legacy du commands
* Update cloud-runner-aws-pipeline.yml
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* PR comments
* Replace guid with uuid package
* use fileExists lambda instead of stat to check file exists in caching
* build failed results in core error message
* Delete sample.txt
2022-04-10 23:00:37 +00:00
|
|
|
) {}
|
2022-10-18 22:10:08 +00:00
|
|
|
setupWorkflow(
|
2022-02-01 02:31:20 +00:00
|
|
|
buildGuid: string,
|
|
|
|
|
buildParameters: BuildParameters,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
branchName: string,
|
|
|
|
|
// eslint-disable-next-line no-unused-vars
|
|
|
|
|
defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
|
2022-09-20 22:14:08 +00:00
|
|
|
) {
|
|
|
|
|
this.buildParameters = buildParameters;
|
|
|
|
|
}
|
|
|
|
|
|
2022-10-18 22:10:08 +00:00
|
|
|
public async runTaskInWorkflow(
|
2022-02-01 02:31:20 +00:00
|
|
|
buildGuid: string,
|
|
|
|
|
image: string,
|
2022-09-17 07:43:09 +00:00
|
|
|
commands: string,
|
2022-02-01 02:31:20 +00:00
|
|
|
mountdir: string,
|
|
|
|
|
workingdir: string,
|
|
|
|
|
environment: CloudRunnerEnvironmentVariable[],
|
|
|
|
|
secrets: CloudRunnerSecret[],
|
Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml
* Update cloud-runner-k8s-pipeline.yml
* yarn build
* yarn build
* correct branch ref
* correct branch ref passed to target repo
* Create k8s-tests.yml
* Delete k8s-tests.yml
* correct branch ref passed to target repo
* correct branch ref passed to target repo
* Always describe AWS tasks for now, because unstable error handling
* Remove unused tree commands
* Use lfs guid sum
* Simple override cache push
* Simple override cache push and pull override to allow pure cloud storage driven caching
* Removal of early branch (breaks lfs caching)
* Remove unused tree commands
* Update action.yml
* Update action.yml
* Support cache and input override commands as input + full support custom hooks
* Increase k8s timeout
* replace filename being appended for unknclear reason
* cache key should not contain whitespaces
* Always try and deploy rook for k8s
* Apply k8s files for rook
* Update action.yml
* Apply k8s files for rook
* Apply k8s files for rook
* cache test and action description for kuber storage class
* Correct test and implement dependency health check and start
* GCP-secret run, cache key
* lfs smudge set explicit and undo explicit
* Run using external secret provider to speed up input
* Update cloud-runner-aws-pipeline.yml
* Add nodejs as build step dependency
* Add nodejs as build step dependency
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* better defaults for new inputs
* better defaults
* merge latest
* force build update
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* correct new line
* quiet zipping
* quiet zipping
* default secrets for unity username and password
* default secrets for unity username and password
* ls active directory before lfs install
* Get cloud runner secrets from
* Get cloud runner secrets from
* Cleanup setup of default secrets
* Various fixes
* Cleanup setup of default secrets
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* AWS secrets manager support
* less caching logs
* default k8s storage class to pd-standard
* more readable build commands
* Capture aws exit code 1 reliably
* Always replace /head from branch
* k8s default storage class to standard-rwo
* cleanup
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* folder sizes to inspect caching
* dir command for local cloud runner test
* k8s wait for pending because pvc will not create earlier
* prefer k8s standard storage
* handle empty string as cloud runner cluster input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* fix unterminated quote
* fix unterminated quote
* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name
* remove head and heads from branch prefix
* fix reversed caching direction of cache-push
* fixes
* fixes
* fixes
* cachePull cli
* fixes
* fixes
* fixes
* fixes
* fixes
* order cache test to be first
* order cache test to be first
* fixes
* populate cache key instead of using branch
* cleanup cli
* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files
* import cli methods
* import cli files explicitly
* import cli files explicitly
* import cli files explicitly
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* Cloud runner param test before caching because we have a fast local cache test now
* Using custom build path relative to repo root rather than project root
* aws-garbage-collect at end of pipeline
* aws-garbage-collect do not actually delete anything for now - just list
* remove some legacy du commands
* Update cloud-runner-aws-pipeline.yml
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* PR comments
* Replace guid with uuid package
* use fileExists lambda instead of stat to check file exists in caching
* build failed results in core error message
* Delete sample.txt
2022-04-10 23:00:37 +00:00
|
|
|
): Promise<string> {
|
|
|
|
|
CloudRunnerLogger.log(buildGuid);
|
|
|
|
|
CloudRunnerLogger.log(commands);
|
2022-04-11 22:43:41 +00:00
|
|
|
|
2022-09-17 08:05:43 +00:00
|
|
|
const { workspace, actionFolder } = Action;
|
2022-09-21 15:40:22 +00:00
|
|
|
const content: any[] = [];
|
2022-09-21 15:33:12 +00:00
|
|
|
for (const x of secrets) {
|
2022-09-21 15:40:22 +00:00
|
|
|
content.push({ name: x.EnvironmentVariable, value: x.ParameterValue });
|
2022-09-21 15:33:12 +00:00
|
|
|
}
|
|
|
|
|
for (const x of environment) {
|
2022-09-21 15:40:22 +00:00
|
|
|
content.push({ name: x.name, value: x.value });
|
2022-09-21 15:33:12 +00:00
|
|
|
}
|
2022-09-27 22:59:01 +00:00
|
|
|
|
|
|
|
|
// if (this.buildParameters?.cloudRunnerIntegrationTests) {
|
|
|
|
|
// core.info(JSON.stringify(content, undefined, 4));
|
|
|
|
|
// core.info(JSON.stringify(secrets, undefined, 4));
|
|
|
|
|
// core.info(JSON.stringify(environment, undefined, 4));
|
|
|
|
|
// }
|
2022-09-21 15:40:22 +00:00
|
|
|
|
2022-09-21 12:54:17 +00:00
|
|
|
// eslint-disable-next-line unicorn/no-for-loop
|
|
|
|
|
for (let index = 0; index < content.length; index++) {
|
|
|
|
|
if (content[index] === undefined) {
|
|
|
|
|
delete content[index];
|
|
|
|
|
}
|
|
|
|
|
}
|
2022-09-21 15:33:12 +00:00
|
|
|
let myOutput = '';
|
2022-10-06 21:05:42 +00:00
|
|
|
const sharedFolder = `/data/`;
|
2022-09-21 01:56:48 +00:00
|
|
|
|
|
|
|
|
// core.info(JSON.stringify({ workspace, actionFolder, ...this.buildParameters, ...content }, undefined, 4));
|
2022-09-22 22:38:25 +00:00
|
|
|
const entrypointFilePath = `start.sh`;
|
2022-10-08 03:23:24 +00:00
|
|
|
const fileContents = `#!/bin/bash
|
|
|
|
|
set -e
|
2022-09-28 00:48:09 +00:00
|
|
|
|
2022-10-08 03:23:24 +00:00
|
|
|
mkdir -p /github/workspace/cloud-runner-cache
|
|
|
|
|
mkdir -p /data/cache
|
|
|
|
|
cp -a /github/workspace/cloud-runner-cache/. ${sharedFolder}
|
|
|
|
|
${commands}
|
|
|
|
|
cp -a ${sharedFolder}. /github/workspace/cloud-runner-cache/
|
2022-10-18 19:03:33 +00:00
|
|
|
`;
|
2022-10-08 03:23:24 +00:00
|
|
|
writeFileSync(`${workspace}/${entrypointFilePath}`, fileContents, {
|
|
|
|
|
flag: 'w',
|
|
|
|
|
});
|
|
|
|
|
|
2022-10-13 17:54:25 +00:00
|
|
|
if (CloudRunner.buildParameters.cloudRunnerDebug) {
|
2022-10-08 03:42:30 +00:00
|
|
|
CloudRunnerLogger.log(`Running local-docker: \n ${fileContents}`);
|
2022-10-08 03:23:24 +00:00
|
|
|
}
|
2022-09-21 21:00:42 +00:00
|
|
|
|
2022-09-27 22:05:40 +00:00
|
|
|
await Docker.run(
|
|
|
|
|
image,
|
|
|
|
|
{ workspace, actionFolder, ...this.buildParameters },
|
|
|
|
|
false,
|
2022-10-13 23:36:55 +00:00
|
|
|
`chmod +x /github/workspace/${entrypointFilePath} && /github/workspace/${entrypointFilePath}`,
|
2022-09-27 22:05:40 +00:00
|
|
|
content,
|
|
|
|
|
{
|
|
|
|
|
listeners: {
|
|
|
|
|
stdout: (data: Buffer) => {
|
|
|
|
|
myOutput += data.toString();
|
|
|
|
|
},
|
|
|
|
|
stderr: (data: Buffer) => {
|
|
|
|
|
myOutput += `[LOCAL-DOCKER-ERROR]${data.toString()}`;
|
|
|
|
|
},
|
2022-09-19 00:21:56 +00:00
|
|
|
},
|
2022-09-19 00:13:27 +00:00
|
|
|
},
|
2022-10-13 20:54:37 +00:00
|
|
|
true,
|
2022-09-27 22:05:40 +00:00
|
|
|
);
|
2022-09-17 08:05:43 +00:00
|
|
|
|
2022-09-19 00:13:27 +00:00
|
|
|
return myOutput;
|
Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml
* Update cloud-runner-k8s-pipeline.yml
* yarn build
* yarn build
* correct branch ref
* correct branch ref passed to target repo
* Create k8s-tests.yml
* Delete k8s-tests.yml
* correct branch ref passed to target repo
* correct branch ref passed to target repo
* Always describe AWS tasks for now, because unstable error handling
* Remove unused tree commands
* Use lfs guid sum
* Simple override cache push
* Simple override cache push and pull override to allow pure cloud storage driven caching
* Removal of early branch (breaks lfs caching)
* Remove unused tree commands
* Update action.yml
* Update action.yml
* Support cache and input override commands as input + full support custom hooks
* Increase k8s timeout
* replace filename being appended for unknclear reason
* cache key should not contain whitespaces
* Always try and deploy rook for k8s
* Apply k8s files for rook
* Update action.yml
* Apply k8s files for rook
* Apply k8s files for rook
* cache test and action description for kuber storage class
* Correct test and implement dependency health check and start
* GCP-secret run, cache key
* lfs smudge set explicit and undo explicit
* Run using external secret provider to speed up input
* Update cloud-runner-aws-pipeline.yml
* Add nodejs as build step dependency
* Add nodejs as build step dependency
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* better defaults for new inputs
* better defaults
* merge latest
* force build update
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* correct new line
* quiet zipping
* quiet zipping
* default secrets for unity username and password
* default secrets for unity username and password
* ls active directory before lfs install
* Get cloud runner secrets from
* Get cloud runner secrets from
* Cleanup setup of default secrets
* Various fixes
* Cleanup setup of default secrets
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* AWS secrets manager support
* less caching logs
* default k8s storage class to pd-standard
* more readable build commands
* Capture aws exit code 1 reliably
* Always replace /head from branch
* k8s default storage class to standard-rwo
* cleanup
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* folder sizes to inspect caching
* dir command for local cloud runner test
* k8s wait for pending because pvc will not create earlier
* prefer k8s standard storage
* handle empty string as cloud runner cluster input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* fix unterminated quote
* fix unterminated quote
* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name
* remove head and heads from branch prefix
* fix reversed caching direction of cache-push
* fixes
* fixes
* fixes
* cachePull cli
* fixes
* fixes
* fixes
* fixes
* fixes
* order cache test to be first
* order cache test to be first
* fixes
* populate cache key instead of using branch
* cleanup cli
* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files
* import cli methods
* import cli files explicitly
* import cli files explicitly
* import cli files explicitly
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* Cloud runner param test before caching because we have a fast local cache test now
* Using custom build path relative to repo root rather than project root
* aws-garbage-collect at end of pipeline
* aws-garbage-collect do not actually delete anything for now - just list
* remove some legacy du commands
* Update cloud-runner-aws-pipeline.yml
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* PR comments
* Replace guid with uuid package
* use fileExists lambda instead of stat to check file exists in caching
* build failed results in core error message
* Delete sample.txt
2022-04-10 23:00:37 +00:00
|
|
|
}
|
2022-02-01 02:31:20 +00:00
|
|
|
}
|
Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml
* Update cloud-runner-k8s-pipeline.yml
* yarn build
* yarn build
* correct branch ref
* correct branch ref passed to target repo
* Create k8s-tests.yml
* Delete k8s-tests.yml
* correct branch ref passed to target repo
* correct branch ref passed to target repo
* Always describe AWS tasks for now, because unstable error handling
* Remove unused tree commands
* Use lfs guid sum
* Simple override cache push
* Simple override cache push and pull override to allow pure cloud storage driven caching
* Removal of early branch (breaks lfs caching)
* Remove unused tree commands
* Update action.yml
* Update action.yml
* Support cache and input override commands as input + full support custom hooks
* Increase k8s timeout
* replace filename being appended for unknclear reason
* cache key should not contain whitespaces
* Always try and deploy rook for k8s
* Apply k8s files for rook
* Update action.yml
* Apply k8s files for rook
* Apply k8s files for rook
* cache test and action description for kuber storage class
* Correct test and implement dependency health check and start
* GCP-secret run, cache key
* lfs smudge set explicit and undo explicit
* Run using external secret provider to speed up input
* Update cloud-runner-aws-pipeline.yml
* Add nodejs as build step dependency
* Add nodejs as build step dependency
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Cloud Runner Tests must be specified to capture logs from cloud runner for tests
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs
* better defaults for new inputs
* better defaults
* merge latest
* force build update
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* use npm n to update node in unity builder
* correct new line
* quiet zipping
* quiet zipping
* default secrets for unity username and password
* default secrets for unity username and password
* ls active directory before lfs install
* Get cloud runner secrets from
* Get cloud runner secrets from
* Cleanup setup of default secrets
* Various fixes
* Cleanup setup of default secrets
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* Various fixes
* AWS secrets manager support
* less caching logs
* default k8s storage class to pd-standard
* more readable build commands
* Capture aws exit code 1 reliably
* Always replace /head from branch
* k8s default storage class to standard-rwo
* cleanup
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* further cleanup input
* folder sizes to inspect caching
* dir command for local cloud runner test
* k8s wait for pending because pvc will not create earlier
* prefer k8s standard storage
* handle empty string as cloud runner cluster input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* local-system is now used for cloud runner test implementation AND correctly unset test CLI input
* fix unterminated quote
* fix unterminated quote
* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name
* remove head and heads from branch prefix
* fix reversed caching direction of cache-push
* fixes
* fixes
* fixes
* cachePull cli
* fixes
* fixes
* fixes
* fixes
* fixes
* order cache test to be first
* order cache test to be first
* fixes
* populate cache key instead of using branch
* cleanup cli
* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files
* import cli methods
* import cli files explicitly
* import cli files explicitly
* import cli files explicitly
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* import cli methods
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* log parameters in cloud runner parameter test
* Cloud runner param test before caching because we have a fast local cache test now
* Using custom build path relative to repo root rather than project root
* aws-garbage-collect at end of pipeline
* aws-garbage-collect do not actually delete anything for now - just list
* remove some legacy du commands
* Update cloud-runner-aws-pipeline.yml
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* log contents after cache pull and fix some scenarios with duplicate secrets
* PR comments
* Replace guid with uuid package
* use fileExists lambda instead of stat to check file exists in caching
* build failed results in core error message
* Delete sample.txt
2022-04-10 23:00:37 +00:00
|
|
|
export default LocalDockerCloudRunner;
|