Commit Graph

119 Commits (605fca859fd74a1af4a55bf468db141925ed7bc0)

Author SHA1 Message Date
Frostebite b4cab93433 cleanup 2022-10-16 05:23:05 +01:00
Frostebite a320cf893c cleanup 2022-10-14 20:04:14 +01:00
Frostebite d7a17bb106 AWS upload step implemented in cloud runner pipeline 2022-10-13 19:53:57 +01:00
Frostebite 17e652cb49 Cloud Runner pipeline will not use IL2CPP 2022-10-02 15:56:35 +01:00
Frostebite e6d7768bbe For now we won't run a full k8s build, doesn't exit at end of workflow 2022-09-30 23:29:04 +01:00
Frostebite be914c1b7a Better locking 2022-09-29 22:30:28 +01:00
Frostebite 9999dcf4be Merge branch 'main' into cloud-runner-develop
# Conflicts:
#	dist/default-build-script/Assets/Editor/UnityBuilderAction/Input/AndroidSettings.cs
#	dist/index.js.map
#	lefthook.yml
2022-09-28 03:32:32 +01:00
Frostebite 6fe0209198 Copy cloud runner cache into github workspace directory 2022-09-28 02:56:13 +01:00
Frostebite 17bbc5fadf Copy cloud runner cache into github workspace directory 2022-09-28 02:41:06 +01:00
Frostebite b1c3214d79 Do not serialize custom job parameter 2022-09-28 00:13:17 +01:00
Frostebite d64a7f5929 Do not serialize custom job parameter 2022-09-27 23:48:40 +01:00
Frostebite d4d10abfa4 Do not serialize custom job parameter 2022-09-27 23:45:15 +01:00
Frostebite 9e5ebff23a get output from local-docker 2022-09-19 01:51:42 +01:00
Frostebite 857c9c1ca6 skip included contents 2022-09-19 00:35:12 +01:00
Frostebite e5c5c9058c tree cache push directory 2022-09-19 00:13:44 +01:00
Frostebite 7dea2c8655 --runInBand 2022-09-18 05:49:02 +01:00
Frostebite b10f1fe828 Remove additional early caching test step in ci pipeline 2022-09-17 08:31:21 +01:00
Frostebite ec78ebd247 Remove additional early caching test step in ci pipeline 2022-09-17 07:24:16 +01:00
Frostebite e425856b61 Remove additional early caching test step in ci pipeline 2022-09-17 07:23:32 +01:00
Frostebite 8fa71d7e3e Remove additional early caching test step in ci pipeline 2022-09-17 07:22:43 +01:00
Frostebite 059968df4c Remove additional early caching test step in ci pipeline 2022-09-17 07:18:59 +01:00
Frostebite a68366708a Remove additional early caching test step in ci pipeline 2022-09-17 07:18:15 +01:00
Frostebite 2992fb6115 Remove additional early caching test step in ci pipeline 2022-09-17 07:03:50 +01:00
Frostebite f226bef5af
Cloud runner preview (#419) 2022-09-16 19:48:40 +01:00
Webber Takken cdee7d1d9a
Update FUNDING.yml (#431) 2022-09-06 16:54:29 +02:00
Webber Takken fb633660a4
chore: add game-ci github sponsoring option (#429) 2022-08-29 23:49:54 +02:00
Webber Takken 5b8bf1c8b7
chore: update action-cats (#421) 2022-08-03 00:56:52 +02:00
Frostebite 17faf202fb
Update cloud-runner-pipeline.yml 2022-06-02 22:32:33 +01:00
Frostebite a4c0e7626d
Update cloud-runner-pipeline.yml 2022-05-19 01:58:33 +01:00
Frostebite f77696efae
Cloud runner develop v0.1 (#395)
* Correct aws logs link

* Correct aws logs link

* better aws cli commands and better cleanup for aws

* better aws cli commands and better cleanup for aws

* improved garbage collection cli options

* Only allow ephemeral runners when using cloud runner integration tests flag to avoid unexpected hangup

* Only allow ephemeral runners when using cloud runner integration tests flag to avoid unexpected hangup

* fix issue #393

* Extract follow log stream service

* consolidate into one pipeline file

* consolidate into one pipeline file
2022-05-05 00:25:17 +01:00
Frostebite f9492052d8 consolidate into one pipeline file 2022-05-02 09:21:07 +01:00
Frostebite ef19bdf03a consolidate into one pipeline file 2022-05-02 09:19:34 +01:00
Frostebite 76fc76bbcd Only allow ephemeral runners when using cloud runner integration tests flag to avoid unexpected hangup 2022-05-01 19:50:29 +01:00
Frostebite 3bd297d1c8 better aws cli commands and better cleanup for aws 2022-05-01 08:24:55 +01:00
Frostebite 8abce48a48
Cloud runner v0.2 - continued quality of life improvements (#387)
* Update cloud-runner-aws-pipeline.yml

* Update cloud-runner-k8s-pipeline.yml

* yarn build

* yarn build

* correct branch ref

* correct branch ref passed to target repo

* Create k8s-tests.yml

* Delete k8s-tests.yml

* correct branch ref passed to target repo

* correct branch ref passed to target repo

* Always describe AWS tasks for now, because unstable error handling

* Remove unused tree commands

* Use lfs guid sum

* Simple override cache push

* Simple override cache push and pull override to allow pure cloud storage driven caching

* Removal of early branch (breaks lfs caching)

* Remove unused tree commands

* Update action.yml

* Update action.yml

* Support cache and input override commands as input + full support custom hooks

* Increase k8s timeout

* replace filename being appended for unknclear reason

* cache key should not contain whitespaces

* Always try and deploy rook for k8s

* Apply k8s files for rook

* Update action.yml

* Apply k8s files for rook

* Apply k8s files for rook

* cache test and action description for kuber storage class

* Correct test and implement dependency health check and start

* GCP-secret run, cache key

* lfs smudge set explicit and undo explicit

* Run using external secret provider to speed up input

* Update cloud-runner-aws-pipeline.yml

* Add nodejs as build step dependency

* Add nodejs as build step dependency

* Cloud Runner Tests must be specified to capture logs from cloud runner for tests

* Cloud Runner Tests must be specified to capture logs from cloud runner for tests

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* better defaults for new inputs

* better defaults

* merge latest

* force build update

* use npm n to update node in unity builder

* use npm n to update node in unity builder

* use npm n to update node in unity builder

* correct new line

* quiet zipping

* quiet zipping

* default secrets for unity username and password

* default secrets for unity username and password

* ls active directory before lfs install

* Get cloud runner secrets from

* Get cloud runner secrets from

* Cleanup setup of default secrets

* Various fixes

* Cleanup setup of default secrets

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* AWS secrets manager support

* less caching logs

* default k8s storage class to pd-standard

* more readable build commands

* Capture aws exit code 1 reliably

* Always replace /head from branch

* k8s default storage class to standard-rwo

* cleanup

* further cleanup input

* further cleanup input

* further cleanup input

* further cleanup input

* further cleanup input

* folder sizes to inspect caching

* dir command for local cloud runner test

* k8s wait for pending because pvc will not create earlier

* prefer k8s standard storage

* handle empty string as cloud runner cluster input

* local-system is now used for cloud runner test implementation AND correctly unset test CLI input

* local-system is now used for cloud runner test implementation AND correctly unset test CLI input

* fix unterminated quote

* fix unterminated quote

* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name

* remove head and heads from branch prefix

* fix reversed caching direction of cache-push

* fixes

* fixes

* fixes

* cachePull cli

* fixes

* fixes

* fixes

* fixes

* fixes

* order cache test to be first

* order cache test to be first

* fixes

* populate cache key instead of using branch

* cleanup cli

* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files

* import cli methods

* import cli files explicitly

* import cli files explicitly

* import cli files explicitly

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* log parameters in cloud runner parameter test

* log parameters in cloud runner parameter test

* log parameters in cloud runner parameter test

* Cloud runner param test before caching because we have a fast local cache test now

* Using custom build path relative to repo root rather than project root

* aws-garbage-collect at end of pipeline

* aws-garbage-collect do not actually delete anything for now - just list

* remove some legacy du commands

* Update cloud-runner-aws-pipeline.yml

* log contents after cache pull and fix some scenarios with duplicate secrets

* log contents after cache pull and fix some scenarios with duplicate secrets

* log contents after cache pull and fix some scenarios with duplicate secrets

* PR comments

* Replace guid with uuid package

* use fileExists lambda instead of stat to check file exists in caching

* build failed results in core error message

* Delete sample.txt

* cloud-runner-system prefix changed to cloud-runner

* Update cloud-runner-aws-pipeline.yml

* remove du from caching, should run manually if interested in size, adds too much runtime to job to include by default

* github ephemeral pipeline support

* github ephemeral pipeline support

* Merge remote-tracking branch 'origin/main' into cloud-runner-develop

# Conflicts:
#	dist/index.js.map
#	src/model/cloud-runner/providers/aws/aws-task-runner.ts
#	src/model/cloud-runner/providers/aws/index.ts

* garbage collection

* garbage collection

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* self hosted runner pipeline

* ephemeral runner pipeline

* ephemeral runner pipeline

* ephemeral runner pipeline

* download runner each time

* download runner each time

* download runner each time

* garbage collect all older than 1d as part of cleanup

* download runner each time

* number container cpu and memory for aws

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* Skip printing size unless cloudRunnerIntegrationTests is true

* transition zip usage in cache to uncompressed tar for speed

* transition zip usage in cache to uncompressed tar for speed

* transition zip usage in cache to uncompressed tar for speed

* transition zip usage in cache to uncompressed tar for speed

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* per provider container defaults

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* baked in cloud formation template

* better aws commands

* better aws commands

* parse number for cloud formation template

* remove container resource defaults from actions yaml

* remove container resource defaults from actions yaml

* skip all input readers when cloud runner is local

* prefer fs/promises

* actually set aws cloud runner step as failure if unity build fails

* default to 3gb of ram - webgl fails on 2
2022-04-22 00:47:45 +01:00
Webber Takken 5ae03dfef6
Streamline code styles (#384)
* feat: streamline code styles

* feat: spacing for comments and return statements

* chore: enforce camelcase

* fix: remove npm lock file

* fix: add integrity test

* fix: remove logfile

* chore: update node in test workflow
2022-04-12 00:43:41 +02:00
Frostebite a61c02481f
Cloud Runner v0 - Reliable and trimmed down cloud runner (#353)
* Update cloud-runner-aws-pipeline.yml

* Update cloud-runner-k8s-pipeline.yml

* yarn build

* yarn build

* correct branch ref

* correct branch ref passed to target repo

* Create k8s-tests.yml

* Delete k8s-tests.yml

* correct branch ref passed to target repo

* correct branch ref passed to target repo

* Always describe AWS tasks for now, because unstable error handling

* Remove unused tree commands

* Use lfs guid sum

* Simple override cache push

* Simple override cache push and pull override to allow pure cloud storage driven caching

* Removal of early branch (breaks lfs caching)

* Remove unused tree commands

* Update action.yml

* Update action.yml

* Support cache and input override commands as input + full support custom hooks

* Increase k8s timeout

* replace filename being appended for unknclear reason

* cache key should not contain whitespaces

* Always try and deploy rook for k8s

* Apply k8s files for rook

* Update action.yml

* Apply k8s files for rook

* Apply k8s files for rook

* cache test and action description for kuber storage class

* Correct test and implement dependency health check and start

* GCP-secret run, cache key

* lfs smudge set explicit and undo explicit

* Run using external secret provider to speed up input

* Update cloud-runner-aws-pipeline.yml

* Add nodejs as build step dependency

* Add nodejs as build step dependency

* Cloud Runner Tests must be specified to capture logs from cloud runner for tests

* Cloud Runner Tests must be specified to capture logs from cloud runner for tests

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* Refactor and cleanup - no async input, combined setup/build, removed github logs for cli runs

* better defaults for new inputs

* better defaults

* merge latest

* force build update

* use npm n to update node in unity builder

* use npm n to update node in unity builder

* use npm n to update node in unity builder

* correct new line

* quiet zipping

* quiet zipping

* default secrets for unity username and password

* default secrets for unity username and password

* ls active directory before lfs install

* Get cloud runner secrets from

* Get cloud runner secrets from

* Cleanup setup of default secrets

* Various fixes

* Cleanup setup of default secrets

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* Various fixes

* AWS secrets manager support

* less caching logs

* default k8s storage class to pd-standard

* more readable build commands

* Capture aws exit code 1 reliably

* Always replace /head from branch

* k8s default storage class to standard-rwo

* cleanup

* further cleanup input

* further cleanup input

* further cleanup input

* further cleanup input

* further cleanup input

* folder sizes to inspect caching

* dir command for local cloud runner test

* k8s wait for pending because pvc will not create earlier

* prefer k8s standard storage

* handle empty string as cloud runner cluster input

* local-system is now used for cloud runner test implementation AND correctly unset test CLI input

* local-system is now used for cloud runner test implementation AND correctly unset test CLI input

* fix unterminated quote

* fix unterminated quote

* do not share build parameters in tests - in cloud runner this will cause conflicts with resouces of the same name

* remove head and heads from branch prefix

* fix reversed caching direction of cache-push

* fixes

* fixes

* fixes

* cachePull cli

* fixes

* fixes

* fixes

* fixes

* fixes

* order cache test to be first

* order cache test to be first

* fixes

* populate cache key instead of using branch

* cleanup cli

* garbage-collect-aws cli can iterate over aws resources and cli scans all ts files

* import cli methods

* import cli files explicitly

* import cli files explicitly

* import cli files explicitly

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* import cli methods

* log parameters in cloud runner parameter test

* log parameters in cloud runner parameter test

* log parameters in cloud runner parameter test

* Cloud runner param test before caching because we have a fast local cache test now

* Using custom build path relative to repo root rather than project root

* aws-garbage-collect at end of pipeline

* aws-garbage-collect do not actually delete anything for now - just list

* remove some legacy du commands

* Update cloud-runner-aws-pipeline.yml

* log contents after cache pull and fix some scenarios with duplicate secrets

* log contents after cache pull and fix some scenarios with duplicate secrets

* log contents after cache pull and fix some scenarios with duplicate secrets

* PR comments

* Replace guid with uuid package

* use fileExists lambda instead of stat to check file exists in caching

* build failed results in core error message

* Delete sample.txt
2022-04-11 00:00:37 +01:00
David Finol ca9d445c3c
Revert windows and mac builds on PR (#367) 2022-03-31 05:31:57 -05:00
David Finol 89c1b38c3b Test windows and mac in PRs 2022-03-31 04:45:42 -05:00
Frostebite efa2eddee9
Update cloud-runner-aws-pipeline.yml 2022-03-15 23:09:38 +00:00
Frostebite d975f3b80f
Minor tweaks and corrections for cloud runner pipeline and branch parameter (#351)
* Update cloud-runner-aws-pipeline.yml

* Update cloud-runner-k8s-pipeline.yml

* yarn build

* yarn build

* correct branch ref

* correct branch ref passed to target repo

* Create k8s-tests.yml

* Delete k8s-tests.yml

* correct branch ref passed to target repo

* correct branch ref passed to target repo
2022-03-04 19:42:24 +00:00
AndrewKahr a6ebcb5b75
Fix minor typo in mac build test workflow (#333) 2022-02-02 12:38:35 +01:00
AndrewKahr c7907c7e78
Initial Support for MacOS IL2CPP Builds (#326)
* Add intial framework for macos builds. Test install editor

* Fix unity hub path space

* Single quote space in path

* Escape space character

* More backslashes

* Move to bash scripts for setup

* Add path to command args

* Different command to run shell script

* Use full path to scripts

* Unpack changeset value and fix missing escape characters

* Print changeset

* More debug

* Remove debug

* Fix script paths

* Printenv debug

* Write environment variables to file to read in bash script

* Debug file write

* More debug

* Fix missing await

* Move back to process.env

* Fix path typo

* Add missing flags

* Make directory for license activation

* Add missing sudo

* Give permissions to license folder

* Fix path issues

* Add build tests

* Try ts setup again

* Try quoting path

* Further migrate mac scripts to align with linux scripts

* print pwd

* Fix changeset and remove unneeded env vars

* Ignore return code

* fix missing current directory

* Fix project path

* pwd

* Remove project path

* Revert to cwd being the workspace folder and pass action folder as an env variable.

* Add blank project to use for activation

* Add blank project path

* Fix build tests

* Don't rebuild library on windows

* Fix project path windows

* Fix platform specific workspace env variable

* Fix incorrect variable name

* Update .github/workflows/mac-build-tests.yml

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update .github/workflows/mac-build-tests.yml

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update dist/BlankProject/Packages/packages-lock.json

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/platform-setup/setup-mac.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/platform-setup/setup-mac.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Fix formatting

Co-authored-by: Webber Takken <webber.nl@gmail.com>
2022-02-02 10:15:37 +01:00
Frostebite 03ae77dc7c
Cloud Runner v2 (#310) 2022-02-01 02:31:20 +00:00
AndrewKahr 3d0eb0805b
Initial Support for Windows Based Builds (#305)
* Implemented logic for windows based docker builds. Moved dockerfiles and scripts to platform specific folders.

* Add missing newline character

* Add build-tests for windows and a unity project configured to output il2cpp

* Add additional build targets (uwp and tvOS)
Adjustments to build scripts to not require win10 sdk when not needed (tvOS)
Platform-based prereq setup
Setup image tags for the new platforms with errors if building on the wrong base os
Rename test-project-il2cpp to test-project-windows to be used for all windows based project building (IL2CPP backend selected instead of mono)
Fix tests to be platform based

* Update dist/platforms/windows/steps/return_license.ps1

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/docker.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/docker.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/docker.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Fix outdated repository and homepage links in dockerfiles

* Fix comment style and rename validateWindowsPrereqs to validateWindowsPlatformRequirements

* Remove redundant comment

* Remove windows unity test project, add ProjectSettings for the il2cpp backend, and add logic to replace the projectsettings file with the il2cpp one on windows test builds.

* Fix action.test.ts to accept windows as a base platform

* Fix camelcase for wsaPlayer

* Switch from add to copy in windows dockerfile

* Change slash direction

* Switch ADD to COPY to conform with best practices, change ls to dir on windows dockerfile

* Improve error message for unset UNITY_EMAIL and UNITY_PASSWORD

* Further improve missing email and password error. Remove temppaths being mounted to docker image

* Add debug statement. TODO: Remove these

* Add more debug

* Explicitly pass in unity email to docker run

* Remove debug and fix environment variables for activation/deactivation scripts

* Prevent Unity serial from leaking to console

* Debug folder listings

* More debug print dirs

* fix debug print path

* fix reg export command

* Remove debug directory listings and try setSecret to mask serial

* Update src/model/action.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/docker.ts

Co-authored-by: Webber Takken <webber.nl@gmail.com>

* Update src/model/image-tag.ts

Co-authored-by: David Finol <davidmfinol@gmail.com>

* Update .github/workflows/build-tests.yml

Co-authored-by: David Finol <davidmfinol@gmail.com>

* Move platform validation and setup out of docker and into its own layer, remove branching on docker run command

* Fix test failure due to missing license

* Fix camelCase and duplicate variables

* Fix lint issues and make paths more understandable

* Fix typo in build-tests.yml

* Fix move command in build-tests.yml

* Different method to force move file

* Fix missing quote and backslash

* Pass unity email and password to builder action for windows build tests

* Push serial to windows test builds

* Make windows build tests only run on push to main

Co-authored-by: Webber Takken <webber.nl@gmail.com>
Co-authored-by: David Finol <davidmfinol@gmail.com>
2022-01-25 22:18:15 +01:00
David Finol 15a02a4b1b
Remove deprecated param from versioning (#315) 2022-01-02 19:02:45 -06:00
Webber Takken 70081d231f
Indicate how to support us (#277) 2021-07-12 19:28:11 +02:00
Webber Takken 8c1a159dd0
add default tag for feature request (#272) 2021-06-05 23:59:56 +02:00
Frostebite c96b8cf443
Remote builder refactoring (#264)
Remote builder refactoring
2021-05-23 05:08:40 +01:00
Webber Takken 947c8b8e20
Fix typo in issue template (#260) 2021-05-13 14:03:00 +02:00