Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding system tests #399

Open
wants to merge 24 commits into
base: dev_main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: using mpm to install MATLAB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that we're using GitHub Actions. Does this mean we can remove the Azure Pipelines yaml?

on: push
jobs:
my-job:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could maybe give this job a more descriptive name in case we want to add more in the future.

strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
name: Run System tests
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@v2

- name: Setup Java 11
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'

Comment on lines +10 to +19
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider updating to actions/checkout@v4 and actions/setup-java@v4

Also consider adding the cache: maven option to the setup-java step as shown here. It could improve build times by quite a bit.

- name: Setup MATLAB
id: setup-matlab
uses: matlab-actions/setup-matlab@v2

- name: Running tests
run: mvn verify
env:
MATLAB_ROOT: ${{ steps.setup-matlab.outputs.matlabroot }}
MLM_LICENSE_TOKEN: ${{ secrets.MLM_LICENSE_TOKEN}}
65 changes: 65 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,35 @@
<version>11.0.24</version>
<scope>test</scope>
</dependency>

<!-- Dependencies for system tests -->
<!-- https://mvnrepository.com/artifact/com.jcabi/jcabi-xml -->
<dependency>
<groupId>com.jcabi</groupId>
<artifactId>jcabi-xml</artifactId>
<version>0.23.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.jenkinsci.plugins/pipeline-model-definition -->
<dependency>
<groupId>org.jenkinsci.plugins</groupId>
<artifactId>pipeline-model-definition</artifactId>
</dependency>
<!-- Need unzip to add the files in pipeline workspace -->
<!-- https://mvnrepository.com/artifact/org.jenkins-ci.plugins/pipeline-utility-steps -->
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>pipeline-utility-steps</artifactId>
<version>2.7.1</version>
<scope>test</scope>
</dependency>
<!-- TODO: see if this is necessary Using for verifyCodeCoverage -->
<!-- https://mvnrepository.com/artifact/org.jenkins-ci.plugins/cobertura -->
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>cobertura</artifactId>
<version>1.16</version>
</dependency>
Comment on lines +129 to +155
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the pipeline-utility-steps dependency uses the test scope (<scope>test</scope>), can all of them use test scope?


</dependencies>
<build>
<plugins>
Expand Down Expand Up @@ -215,6 +244,42 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<includes>
<include>**/*Test.java</include>
<include>**/*Tester.java</include>
</includes>
<excludes>
<exclude>**/*IT.java</exclude>>
</excludes>
</configuration>
</plugin>

</plugins>
<pluginManagement>
<plugins>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,136 +1,136 @@
package unit.com.mathworks.ci.actions;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never noticed this before but we have separate test packages. In the Bamboo plugin we use the same package. Should the package be updated or are we going to keep them in separate packages?

/**
* Copyright 2024, The MathWorks Inc.
*/
import java.io.File;
import java.io.IOException;
import java.io.PrintStream;
import java.nio.file.Files;
import org.junit.Test;
import org.junit.Before;
import org.junit.runner.RunWith;
import static org.junit.Assert.*;
import org.mockito.InOrder;
import org.mockito.Mock;
import org.mockito.junit.MockitoJUnitRunner;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.*;
import hudson.FilePath;
import hudson.model.Run;
import hudson.model.TaskListener;
import com.mathworks.ci.MatlabBuilderConstants;
import com.mathworks.ci.BuildArtifactAction;
import com.mathworks.ci.BuildConsoleAnnotator;
import com.mathworks.ci.MatlabExecutionException;
import com.mathworks.ci.actions.RunMatlabCommandAction;
import com.mathworks.ci.utilities.MatlabCommandRunner;
import com.mathworks.ci.parameters.CommandActionParameters;
@RunWith(MockitoJUnitRunner.Silent.class)
public class MatlabActionTest {
@Mock
CommandActionParameters params;
@Mock
BuildConsoleAnnotator annotator;
@Mock
MatlabCommandRunner runner;
@Mock
PrintStream out;
@Mock
TaskListener listener;
@Mock
Run build;
@Mock
FilePath tempFolder;
private boolean setup = false;
private RunMatlabCommandAction action;
// Not using @BeforeClass to avoid static fields.
@Before
public void init() {
if (!setup) {
setup = true;
action = new RunMatlabCommandAction(runner, annotator, params);
when(runner.getTempFolder()).thenReturn(tempFolder);
when(tempFolder.getRemote()).thenReturn("/path/less/traveled");
when(params.getTaskListener()).thenReturn(listener);
when(listener.getLogger()).thenReturn(out);
when(params.getBuild()).thenReturn(build);
}
}
@Test
public void shouldCopyPluginsToTempDirectory() throws IOException, InterruptedException, MatlabExecutionException {
action.run();
InOrder inOrder = inOrder(runner);
inOrder.verify(runner)
.copyFileToTempFolder(MatlabBuilderConstants.DEFAULT_PLUGIN, MatlabBuilderConstants.DEFAULT_PLUGIN);
inOrder.verify(runner)
.copyFileToTempFolder(MatlabBuilderConstants.BUILD_REPORT_PLUGIN,
MatlabBuilderConstants.BUILD_REPORT_PLUGIN);
inOrder.verify(runner)
.copyFileToTempFolder(MatlabBuilderConstants.TASK_RUN_PROGRESS_PLUGIN,
MatlabBuilderConstants.TASK_RUN_PROGRESS_PLUGIN);
}
@Test
public void shouldOverrideDefaultBuildtoolPlugin()
throws IOException, InterruptedException, MatlabExecutionException {
action.run();
verify(runner).addEnvironmentVariable(
"MW_MATLAB_BUILDTOOL_DEFAULT_PLUGINS_FCN_OVERRIDE",
"ciplugins.jenkins.getDefaultPlugins");
}
@Test
public void shouldCopyBuildResultsToRootAndAddAction()
throws IOException, InterruptedException, MatlabExecutionException {
File tmp = Files.createTempDirectory("temp").toFile();
tmp.deleteOnExit();
File dest = Files.createTempDirectory("dest").toFile();
dest.deleteOnExit();
File json = new File(tmp, "buildArtifact.json");
json.createNewFile();
doReturn(new FilePath(tmp)).when(runner).getTempFolder();
doReturn(dest).when(build).getRootDir();
action.run();
// Should have deleted original file
assertFalse(json.exists());
// Should have copied file to root dir
assertTrue(new File(dest, "buildArtifact" + action.getActionID() + ".json").exists());
}
@Test
public void shouldNotAddActionIfNoBuildResult() throws IOException, InterruptedException, MatlabExecutionException {
action.run();
verify(build, never()).addAction(any(BuildArtifactAction.class));
}
@Test
public void shouldRemoveTempFolder() throws IOException, InterruptedException, MatlabExecutionException {
action.run();
verify(runner).removeTempFolder();
}
}
package unit.com.mathworks.ci.actions;

/**
* Copyright 2024, The MathWorks Inc.
*/

import java.io.File;
import java.io.IOException;
import java.io.PrintStream;
import java.nio.file.Files;

import org.junit.Test;
import org.junit.Before;
import org.junit.runner.RunWith;
import static org.junit.Assert.*;

import org.mockito.InOrder;
import org.mockito.Mock;
import org.mockito.junit.MockitoJUnitRunner;

import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.*;

import hudson.FilePath;
import hudson.model.Run;
import hudson.model.TaskListener;

import com.mathworks.ci.MatlabBuilderConstants;
import com.mathworks.ci.BuildArtifactAction;
import com.mathworks.ci.BuildConsoleAnnotator;
import com.mathworks.ci.MatlabExecutionException;
import com.mathworks.ci.actions.RunMatlabCommandAction;
import com.mathworks.ci.utilities.MatlabCommandRunner;
import com.mathworks.ci.parameters.CommandActionParameters;

@RunWith(MockitoJUnitRunner.Silent.class)
public class MatlabActionTest {
@Mock
CommandActionParameters params;
@Mock
BuildConsoleAnnotator annotator;
@Mock
MatlabCommandRunner runner;
@Mock
PrintStream out;
@Mock
TaskListener listener;
@Mock
Run build;

@Mock
FilePath tempFolder;

private boolean setup = false;
private RunMatlabCommandAction action;

// Not using @BeforeClass to avoid static fields.
@Before
public void init() {
if (!setup) {
setup = true;
action = new RunMatlabCommandAction(runner, annotator, params);

when(runner.getTempFolder()).thenReturn(tempFolder);
when(tempFolder.getRemote()).thenReturn("/path/less/traveled");

when(params.getTaskListener()).thenReturn(listener);
when(listener.getLogger()).thenReturn(out);

when(params.getBuild()).thenReturn(build);
}
}

@Test
public void shouldCopyPluginsToTempDirectory() throws IOException, InterruptedException, MatlabExecutionException {
action.run();

InOrder inOrder = inOrder(runner);

inOrder.verify(runner)
.copyFileToTempFolder(MatlabBuilderConstants.DEFAULT_PLUGIN, MatlabBuilderConstants.DEFAULT_PLUGIN);
inOrder.verify(runner)
.copyFileToTempFolder(MatlabBuilderConstants.BUILD_REPORT_PLUGIN,
MatlabBuilderConstants.BUILD_REPORT_PLUGIN);
inOrder.verify(runner)
.copyFileToTempFolder(MatlabBuilderConstants.TASK_RUN_PROGRESS_PLUGIN,
MatlabBuilderConstants.TASK_RUN_PROGRESS_PLUGIN);
}

@Test
public void shouldOverrideDefaultBuildtoolPlugin()
throws IOException, InterruptedException, MatlabExecutionException {
action.run();

verify(runner).addEnvironmentVariable(
"MW_MATLAB_BUILDTOOL_DEFAULT_PLUGINS_FCN_OVERRIDE",
"ciplugins.jenkins.getDefaultPlugins");
}

@Test
public void shouldCopyBuildResultsToRootAndAddAction()
throws IOException, InterruptedException, MatlabExecutionException {
File tmp = Files.createTempDirectory("temp").toFile();
tmp.deleteOnExit();

File dest = Files.createTempDirectory("dest").toFile();
dest.deleteOnExit();

File json = new File(tmp, "buildArtifact.json");
json.createNewFile();

doReturn(new FilePath(tmp)).when(runner).getTempFolder();
doReturn(dest).when(build).getRootDir();

action.run();

// Should have deleted original file
assertFalse(json.exists());
// Should have copied file to root dir
assertTrue(new File(dest, "buildArtifact" + action.getActionID() + ".json").exists());
}

@Test
public void shouldNotAddActionIfNoBuildResult() throws IOException, InterruptedException, MatlabExecutionException {
action.run();

verify(build, never()).addAction(any(BuildArtifactAction.class));
}

@Test
public void shouldRemoveTempFolder() throws IOException, InterruptedException, MatlabExecutionException {
action.run();

verify(runner).removeTempFolder();
}
}
Loading
Loading