unity-test-runner/src/model/docker.ts

216 lines
8.3 KiB
TypeScript
Raw Normal View History

feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
import { existsSync, mkdirSync, readFileSync, rmSync } from 'fs';
import LicensingServerSetup from './licensing-server-setup';
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
import type { RunnerContext } from './action';
import { exec } from '@actions/exec';
import path from 'path';
2020-01-29 21:22:26 +00:00
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
/**
* Build a path for a docker --cidfile parameter. Docker will store the the created container.
* This path is stable for the whole execution of the action, so it can be executed with the same parameters
* multiple times and get the same result.
*/
const containerIdFilePath = parameters => {
const { runnerTemporaryPath, githubAction } = parameters;
return path.join(runnerTemporaryPath, `container_${githubAction}`);
};
const Docker = {
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
/**
* Remove a possible leftover container created by `Docker.run`.
*/
async ensureContainerRemoval(parameters: RunnerContext) {
const cidfile = containerIdFilePath(parameters);
if (!existsSync(cidfile)) {
return;
}
const container = readFileSync(cidfile, 'ascii').trim();
await exec(`docker`, ['rm', '--force', '--volumes', container], { silent: true });
rmSync(cidfile);
},
async run(image, parameters, silent = false) {
let runCommand = '';
if (parameters.unityLicensingServer !== '') {
LicensingServerSetup.Setup(parameters.unityLicensingServer, parameters.actionFolder);
}
switch (process.platform) {
case 'linux':
runCommand = this.getLinuxCommand(image, parameters);
break;
case 'win32':
runCommand = this.getWindowsCommand(image, parameters);
break;
default:
throw new Error(`Operation system, ${process.platform}, is not supported yet.`);
}
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
await exec(runCommand, undefined, { silent });
},
getLinuxCommand(image, parameters): string {
2020-01-30 23:57:08 +00:00
const {
actionFolder,
editorVersion,
2020-01-30 23:57:08 +00:00
workspace,
projectPath,
2022-01-01 20:38:47 +00:00
customParameters,
2020-01-30 23:57:08 +00:00
testMode,
coverageOptions,
2020-01-30 23:57:08 +00:00
artifactsPath,
2020-04-01 20:24:13 +00:00
useHostNetwork,
sshAgent,
gitPrivateToken,
2022-01-01 20:38:47 +00:00
githubToken,
runnerTemporaryPath,
chownFilesTo,
unityLicensingServer,
2020-01-30 23:57:08 +00:00
} = parameters;
2020-01-29 21:22:26 +00:00
const githubHome = path.join(runnerTemporaryPath, '_github_home');
if (!existsSync(githubHome)) mkdirSync(githubHome);
const githubWorkflow = path.join(runnerTemporaryPath, '_github_workflow');
if (!existsSync(githubWorkflow)) mkdirSync(githubWorkflow);
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
const cidfile = containerIdFilePath(parameters);
const testPlatforms = (
testMode === 'all' ? ['playmode', 'editmode', 'COMBINE_RESULTS'] : [testMode]
).join(';');
return `docker run \
--workdir /github/workspace \
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
--cidfile "${cidfile}" \
--rm \
--env UNITY_LICENSE \
--env UNITY_LICENSE_FILE \
--env UNITY_EMAIL \
--env UNITY_PASSWORD \
--env UNITY_SERIAL \
--env UNITY_LICENSING_SERVER="${unityLicensingServer}" \
--env UNITY_VERSION="${editorVersion}" \
--env PROJECT_PATH="${projectPath}" \
--env CUSTOM_PARAMETERS="${customParameters}" \
--env TEST_PLATFORMS="${testPlatforms}" \
--env COVERAGE_OPTIONS="${coverageOptions}" \
--env COVERAGE_RESULTS_PATH="CodeCoverage" \
--env ARTIFACTS_PATH="${artifactsPath}" \
--env GITHUB_REF \
--env GITHUB_SHA \
--env GITHUB_REPOSITORY \
--env GITHUB_ACTOR \
--env GITHUB_WORKFLOW \
--env GITHUB_HEAD_REF \
--env GITHUB_BASE_REF \
--env GITHUB_EVENT_NAME \
--env GITHUB_WORKSPACE="/github/workspace" \
--env GITHUB_ACTION \
--env GITHUB_EVENT_PATH \
--env RUNNER_OS \
--env RUNNER_TOOL_CACHE \
--env RUNNER_TEMP \
--env RUNNER_WORKSPACE \
--env GIT_PRIVATE_TOKEN="${gitPrivateToken}" \
--env CHOWN_FILES_TO="${chownFilesTo}" \
${sshAgent ? '--env SSH_AUTH_SOCK=/ssh-agent' : ''} \
--volume "${githubHome}:/root:z" \
--volume "${githubWorkflow}:/github/workflow:z" \
--volume "${workspace}:/github/workspace:z" \
--volume "${actionFolder}/test-standalone-scripts:/UnityStandaloneScripts:z" \
--volume "${actionFolder}/steps:/steps:z" \
--volume "${actionFolder}/entrypoint.sh:/entrypoint.sh:z" \
--volume "${actionFolder}/unity-config:/usr/share/unity3d/config/:z" \
${sshAgent ? `--volume ${sshAgent}:/ssh-agent` : ''} \
${
sshAgent ? `--volume /home/runner/.ssh/known_hosts:/root/.ssh/known_hosts:ro` : ''
} \
${useHostNetwork ? '--net=host' : ''} \
${githubToken ? '--env USE_EXIT_CODE=false' : '--env USE_EXIT_CODE=true'} \
${image} \
/bin/bash -c /entrypoint.sh`;
},
getWindowsCommand(image, parameters): string {
const {
actionFolder,
editorVersion,
workspace,
projectPath,
customParameters,
testMode,
coverageOptions,
artifactsPath,
useHostNetwork,
sshAgent,
gitPrivateToken,
githubToken,
runnerTemporaryPath,
chownFilesTo,
unityLicensingServer,
} = parameters;
const githubHome = path.join(runnerTemporaryPath, '_github_home');
if (!existsSync(githubHome)) mkdirSync(githubHome);
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
const cidfile = containerIdFilePath(parameters);
const githubWorkflow = path.join(runnerTemporaryPath, '_github_workflow');
if (!existsSync(githubWorkflow)) mkdirSync(githubWorkflow);
const testPlatforms = (
testMode === 'all' ? ['playmode', 'editmode', 'COMBINE_RESULTS'] : [testMode]
).join(';');
2020-01-29 21:22:26 +00:00
return `docker run \
--workdir /github/workspace \
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
--cidfile "${cidfile}" \
--rm \
--env UNITY_LICENSE \
--env UNITY_LICENSE_FILE \
--env UNITY_EMAIL \
--env UNITY_PASSWORD \
--env UNITY_SERIAL \
--env UNITY_LICENSING_SERVER="${unityLicensingServer}" \
--env UNITY_VERSION="${editorVersion}" \
--env PROJECT_PATH="${projectPath}" \
--env CUSTOM_PARAMETERS="${customParameters}" \
--env TEST_PLATFORMS="${testPlatforms}" \
--env COVERAGE_OPTIONS="${coverageOptions}" \
--env COVERAGE_RESULTS_PATH="CodeCoverage" \
--env ARTIFACTS_PATH="${artifactsPath}" \
--env GITHUB_REF \
--env GITHUB_SHA \
--env GITHUB_REPOSITORY \
--env GITHUB_ACTOR \
--env GITHUB_WORKFLOW \
--env GITHUB_HEAD_REF \
--env GITHUB_BASE_REF \
--env GITHUB_EVENT_NAME \
--env GITHUB_WORKSPACE="/github/workspace" \
--env GITHUB_ACTION \
--env GITHUB_EVENT_PATH \
--env RUNNER_OS \
--env RUNNER_TOOL_CACHE \
--env RUNNER_TEMP \
--env RUNNER_WORKSPACE \
--env GIT_PRIVATE_TOKEN="${gitPrivateToken}" \
--env CHOWN_FILES_TO="${chownFilesTo}" \
${sshAgent ? '--env SSH_AUTH_SOCK=c:/ssh-agent' : ''} \
--volume "${actionFolder}/test-standalone-scripts":"c:/UnityStandaloneScripts" \
--volume "${githubHome}":"c:/root" \
--volume "${githubWorkflow}":"c:/github/workflow" \
--volume "${workspace}":"c:/github/workspace" \
--volume "${actionFolder}/steps":"c:/steps" \
--volume "${actionFolder}":"c:/dist" \
${sshAgent ? `--volume ${sshAgent}:c:/ssh-agent` : ''} \
${
sshAgent
? `--volume c:/Users/Administrator/.ssh/known_hosts:c:/root/.ssh/known_hosts`
: ''
} \
${useHostNetwork ? '--net=host' : ''} \
${githubToken ? '--env USE_EXIT_CODE=false' : '--env USE_EXIT_CODE=true'} \
${image} \
powershell c:/dist/entrypoint.ps1`;
},
};
2020-01-29 21:22:26 +00:00
export default Docker;