unity-test-runner/src/model/docker.ts

166 lines
6.3 KiB
TypeScript
Raw Normal View History

2023-12-10 11:50:15 +00:00
import ImageEnvironmentFactory from './image-environment-factory';
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
import { existsSync, mkdirSync, readFileSync, rmSync } from 'fs';
import LicensingServerSetup from './licensing-server-setup';
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
import type { RunnerContext } from './action';
import { exec } from '@actions/exec';
import path from 'path';
2020-01-29 21:22:26 +00:00
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
/**
* Build a path for a docker --cidfile parameter. Docker will store the the created container.
* This path is stable for the whole execution of the action, so it can be executed with the same parameters
* multiple times and get the same result.
*/
const containerIdFilePath = parameters => {
const { runnerTemporaryPath, githubAction } = parameters;
return path.join(runnerTemporaryPath, `container_${githubAction}`);
};
const Docker = {
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
/**
* Remove a possible leftover container created by `Docker.run`.
*/
async ensureContainerRemoval(parameters: RunnerContext) {
const cidfile = containerIdFilePath(parameters);
if (!existsSync(cidfile)) {
return;
}
const container = readFileSync(cidfile, 'ascii').trim();
await exec(`docker`, ['rm', '--force', '--volumes', container], { silent: true });
rmSync(cidfile);
},
async run(image, parameters, silent = false) {
let runCommand = '';
if (parameters.unityLicensingServer !== '') {
LicensingServerSetup.Setup(parameters.unityLicensingServer, parameters.actionFolder);
}
switch (process.platform) {
case 'linux':
runCommand = this.getLinuxCommand(image, parameters);
break;
case 'win32':
runCommand = this.getWindowsCommand(image, parameters);
break;
default:
throw new Error(`Operation system, ${process.platform}, is not supported yet.`);
}
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
await exec(runCommand, undefined, { silent });
},
getLinuxCommand(image, parameters): string {
2020-01-30 23:57:08 +00:00
const {
actionFolder,
2020-01-30 23:57:08 +00:00
workspace,
testMode,
2020-04-01 20:24:13 +00:00
useHostNetwork,
sshAgent,
sshPublicKeysDirectoryPath,
2022-01-01 20:38:47 +00:00
githubToken,
runnerTemporaryPath,
dockerCpuLimit,
dockerMemoryLimit,
2020-01-30 23:57:08 +00:00
} = parameters;
2020-01-29 21:22:26 +00:00
const githubHome = path.join(runnerTemporaryPath, '_github_home');
if (!existsSync(githubHome)) mkdirSync(githubHome);
const githubWorkflow = path.join(runnerTemporaryPath, '_github_workflow');
if (!existsSync(githubWorkflow)) mkdirSync(githubWorkflow);
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
const cidfile = containerIdFilePath(parameters);
const testPlatforms = (
testMode === 'all' ? ['playmode', 'editmode', 'COMBINE_RESULTS'] : [testMode]
).join(';');
return `docker run \
2023-12-10 11:50:15 +00:00
--workdir /github/workspace \
--cidfile "${cidfile}" \
--rm \
${ImageEnvironmentFactory.getEnvVarString(parameters)} \
--env GIT_CONFIG_EXTENSIONS \
--env TEST_PLATFORMS="${testPlatforms}" \
2023-12-11 07:39:42 +00:00
--env GITHUB_WORKSPACE="/github/workspace" \
2023-12-10 11:50:15 +00:00
${sshAgent ? '--env SSH_AUTH_SOCK=/ssh-agent' : ''} \
--volume "${githubHome}:/root:z" \
--volume "${githubWorkflow}:/github/workflow:z" \
--volume "${workspace}:/github/workspace:z" \
--volume "${actionFolder}/test-standalone-scripts:/UnityStandaloneScripts:z" \
2023-12-11 00:33:51 +00:00
--volume "${actionFolder}/platforms/ubuntu:/steps:z" \
2023-12-10 11:50:15 +00:00
--volume "${actionFolder}/unity-config:/usr/share/unity3d/config/:z" \
--volume "${actionFolder}/BlankProject":"/BlankProject:z" \
--cpus=${dockerCpuLimit} \
--memory=${dockerMemoryLimit} \
${sshAgent ? `--volume ${sshAgent}:/ssh-agent` : ''} \
${
sshAgent && !sshPublicKeysDirectoryPath
? `--volume /home/runner/.ssh/known_hosts:/root/.ssh/known_hosts:ro`
: ''
} \
${
sshPublicKeysDirectoryPath
? `--volume ${sshPublicKeysDirectoryPath}:/root/.ssh:ro`
: ''
} \
${useHostNetwork ? '--net=host' : ''} \
${githubToken ? '--env USE_EXIT_CODE=false' : '--env USE_EXIT_CODE=true'} \
${image} \
2023-12-11 00:45:45 +00:00
/bin/bash -c "/steps/entrypoint.sh`;
},
getWindowsCommand(image, parameters): string {
const {
actionFolder,
workspace,
testMode,
useHostNetwork,
sshAgent,
githubToken,
runnerTemporaryPath,
dockerCpuLimit,
dockerMemoryLimit,
dockerIsolationMode,
} = parameters;
const githubHome = path.join(runnerTemporaryPath, '_github_home');
if (!existsSync(githubHome)) mkdirSync(githubHome);
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
const cidfile = containerIdFilePath(parameters);
const githubWorkflow = path.join(runnerTemporaryPath, '_github_workflow');
if (!existsSync(githubWorkflow)) mkdirSync(githubWorkflow);
const testPlatforms = (
testMode === 'all' ? ['playmode', 'editmode', 'COMBINE_RESULTS'] : [testMode]
).join(';');
2020-01-29 21:22:26 +00:00
return `docker run \
--workdir c:/github/workspace \
feat: ensure cleanup of docker containers (#198) Cancelled or timeouted workflow would keep the docker container running. Closes game-ci/unity-test-runner#197 This has two parts: Part one. The entrypoints. `runs.post`: GitHub Action metadata allow running something after the action (regardless of a failure, crash, timeout, ...). https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost However, it needs to be a `.js` file and it can't be configured to pass any arguments. There already was `index.js` used as the main entrypoint. The build process of this file uses typescript compiler and ncc to pack all dependencies into one .js file. And ncc has no way of generating multiple files in one go, so the only solution would be to run ncc twice and generate two independent files. That would be quite unfortunate, wasting time and storage. So I rather came up with a new entrypoint that symlinked from two locations. And this new entrypoint understands how it was executed, so it can run the correct behaviour. This makes it easy to add `runs.pre` if needed. This new entrypoint is in `index.ts`. The original `index.ts` is now in `main.ts`. Part two. The signals. I've tried: * try/catch/finally around the `await Docker.run`. Catch and finally are not executed when process receives SIGINT. See the discussion in: https://github.com/nodejs/node/discussions/29480 * New AbortController and AbortSignal. Great concept, but the action.exec does not support it. So it can't be aborted. * Doing cleanup on `process.on('exit')`. Unfortunately you can't really do async stuff from there, so can't really run the docker rm command to delete the container. * Using `process.on('SIGINT')`. For some reason that wasn't really executing for me. I'd not put my hand in fire for this, but I assume because it was in the signal handler it does something special, or would heed to be scheduled for later with `setTimeout(0)`. Evaluating all these I came to a conclusion that it is fragile and just relying on a `runs.post` is much better and safer. `
2022-11-03 18:14:51 +00:00
--cidfile "${cidfile}" \
--rm \
2023-12-10 11:50:15 +00:00
${ImageEnvironmentFactory.getEnvVarString(parameters)} \
--env TEST_PLATFORMS="${testPlatforms}" \
--env GITHUB_WORKSPACE="c:/github/workspace" \
${sshAgent ? '--env SSH_AUTH_SOCK=c:/ssh-agent' : ''} \
--volume "${actionFolder}/test-standalone-scripts":"c:/UnityStandaloneScripts" \
--volume "${githubHome}":"c:/root" \
--volume "${githubWorkflow}":"c:/github/workflow" \
--volume "${workspace}":"c:/github/workspace" \
--volume "${actionFolder}/platforms/windows":"c:/steps" \
--volume "${actionFolder}/BlankProject":"c:/BlankProject" \
${sshAgent ? `--volume ${sshAgent}:c:/ssh-agent` : ''} \
${
sshAgent
? `--volume c:/Users/Administrator/.ssh/known_hosts:c:/root/.ssh/known_hosts`
: ''
} \
--cpus=${dockerCpuLimit} \
--memory=${dockerMemoryLimit} \
--isolation=${dockerIsolationMode} \
${useHostNetwork ? '--net=host' : ''} \
${githubToken ? '--env USE_EXIT_CODE=false' : '--env USE_EXIT_CODE=true'} \
${image} \
powershell c:/steps/entrypoint.ps1`;
},
};
2020-01-29 21:22:26 +00:00
export default Docker;