Jenkins shared library unit tests - missing dependencies and assert singleton values

[BarelElbaz]

Describe your use-case which is not covered by existing documentation.

I'm trying to write unit test to my Jenkins shared library using jenkins-pipeline-unit but for no success.

My project/directory structure is:

src/
└── org
    └── company
        ├── JobData.groovy
        ├── Constants.groovy
        ├── Logger.groovy
        └── Utils.groovy
vars
└── automationTest.groovy
└── fileIsEmpty.groovy
test
└── AutomationTestTest.groovy

I've initialized gradle and created some build.gradle file with many dependencies and tried to be accurate and use the plugins versions from my jenkins instance.

What I'm trying to do is to write test for automationTest.groovy variable.

here is some relevant code:

//automationTest.groovy 
import org.company.Constants
import org.company.Utils
import org.company.JobData
import org.company.Logger
import java.nio.file.NoSuchFileException

def beafBAtest(String setupName, String suiteFile){
    stage("BA_Test_Beaf"){
        env.FAILED_STAGE = env.STAGE_NAME
        Logger.colorPrint(this, " Processing Beaf BA Test ".center(61, '='))
        String logLevel = "debug"
        sh label: "Pytest Execution (BA)", script: """#!/bin/bash
            #Create and activate virtualenv
            #python3.6 -m venv venv
            virtualenv --python=python3 venv
            source venv/bin/activate
        """
        logFiles = sh(returnStdout: true, script: "ls -1 pytests/log_*.ba || true", label: "BA log parsing").split()
        for(logFile in logFiles){
            if(fileIsEmpty(logFile))
                continue
            def logContent = readFile file: "$logFile"
            try{
                List testsSummariesRows = Utils.getTestLogRows(logContent.readLines(), 1)
                if(testsSummariesRows.isEmpty()){
                    Logger.printError(this, "Error")
                }
                testsSummariesRows.each{ testsSummaryRow ->
                    def currSummary = readJSON(text: testsSummaryRow)
                    currSummary.each{ testCaseName, testCaseInfo ->

                        JobData.instance.BAtests[testCaseName] = (JobData.instance.BAtests[testCaseName] ?: []) +
                                                                    [
                                                                        "link" : testCaseInfo["TestReportRunUrl"] ?: "n/a",
                                                                        "result" : testCaseInfo["Verdict"] ?: "n/a"
                                                                    ]
                    }
                }
            }catch(e){
                Logger.printError(this, "Error2")
            }
        }
        sh(label: "BA test BEAF result", script: "exit \$(cat pytests/exit_status)")
    }
}

I need to test the value of the singleton JobData within the test file but have no idea how to import it

//Utils.groovy
package org.company
import groovy.json.JsonOutput
import org.yaml.snakeyaml.DumperOptions
import org.yaml.snakeyaml.Yaml
import org.apache.http.client.utils.URIBuilder;
import org.company.Constants
import java.text.SimpleDateFormat;
import java.util.Date;
import hudson.model.User
import hudson.tasks.Mailer
import jenkins.model.Jenkins
import org.jenkinsci.plugins.pipeline.modeldefinition.Utils as jUtils
import org.company.Logger
import org.company.JobData

class Utils{
    // A lot of static classes which uses the imports (like jenkins.model.Jenkins)
    static def yamlToString(Map yaml){
     //implementation here
    }
    static List getTestLogRows(List loglines, int lineOffset = 0){
    }
    //... (more methods)

}

//JobData.groovy
package org.company
import org.company.Constants
import org.company.Utils
@Singleton
public class JobData{
    //singleton for storing job data
    public String releaseBranch
    public def BAtests = [:]
    ...
Everytime I run the test and getting undefined classes I struggled with it a lot and the current exceptions is:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during conversion: java.lang.NoClassDefFoundError: Unable to load class jenkins.model.Jenkins due to missing dependency antlr/ANTLRException
This is the dependencies block of my build.gradle:

dependencies {
    implementation "org.codehaus.groovy:groovy-all:2.4.12"
    implementation 'org.codehaus.groovy:groovy-jaxb:3.0.17'

    implementation 'javax.servlet:javax.servlet-api:4.0.1@jar'
    implementation 'org.jenkins-ci.main:remoting:3.40@jar'
    implementation 'org.springframework.security:spring-security-core:5.5.2'
    implementation 'com.thoughtworks.xstream:xstream:1.4.16'
    implementation 'org.antlr:antlr-runtime:3.5.2@jar'
    implementation 'antlr:antlr:3.0ea8'
    implementation 'org.kohsuke.stapler:stapler:1777.v7c6fe6d54a_0c@jar'
    // testImplementation 'org.spockframework:spock-core:2.0-groovy-3.0'
    testImplementation "com.lesfurets:jenkins-pipeline-unit:1.9"
    testImplementation "junit:junit:4.12"
    //Fetch 3rd imports for src/utils class
    testImplementation 'org.yaml:snakeyaml:1.28'
    testImplementation 'org.apache.httpcomponents:httpclient:4.5.13@jar'
    testImplementation 'org.jenkins-ci.main:jenkins-core:2.303.2@jar'

    testImplementation 'org.jenkinsci.plugins:pipeline-model-api:1.9.2@jar'
    testImplementation 'org.jenkinsci.plugins:pipeline-model-definition:1.9.2@jar'
    testImplementation 'org.jenkinsci.plugins:pipeline-model-extensions:1.9.2@jar'
    implementation 'org.jenkinsci.plugins:pipeline-model-declarative-agent:1.1@jar'


    testImplementation 'org.connectbot.jbcrypt:jbcrypt:1.0.0@jar'
    testImplementation 'org.jenkins-ci.plugins:mailer:408.vd726a_1130320@jar'
}

And this is the test that I'm trying to write:

//test/AutomationTestTest.groovy
//Should I?
//package org.company
//import org.company.JobData

import org.junit.*
import com.lesfurets.jenkins.unit.*
import static groovy.test.GroovyAssert.*

//Imports for loading the library itself 
import static com.lesfurets.jenkins.unit.global.lib.LibraryConfiguration.library
import static com.lesfurets.jenkins.unit.global.lib.ProjectSource.projectSource


class AutomationTestTest extends BasePipelineTest {
    Object script

    def automationTest
    def resourcesTestDir

    @Before
    void setUp(){
        super.setUp()
        //This fails due to unsatisfied dependencies in Utils.groovy
        Object library = library()
            .name('jenkins-shared-lib')
            .defaultVersion('<notNeeded>')
            .allowOverride(true)
            .implicit(true)
            .targetPath('<notNeeded>')
            .retriever(projectSource())
            .build()
        
        helper.registerSharedLibrary(library)

        script = loadScript('test/resources/EmptyPipeline.groovy')
        assertNotNull(script)
        
        resourcesTestDir = new File('test/resources/AutomationTest')
        automationTest = loadScript("vars/automationTest.groovy")

    }

    @Test
    void testBeafBAtest(){
        helper.addShMock(~'''(?m)#!/bin/bash
            #Create and activate virtualenv
            #python3.6 -m venv venv
            .*
        ''', '', 0)
        helper.addShMock('ls -1 pytests/log_*.ba || true', baLogs.join('\n') , 0)
        helper.addShMock('exit \$(cat pytests/exit_status)', '', 0)

        //Mock log files from test/resources/AutomationTest such as ['log_0.ba', 'log_1.ba']
        resourcesTestDir.eachFileMatch(~/log_\d+\.ba/){ baLogFile -> 
            helper.addReadFileMock(baLogFile.name, baLogFile.text)
        }

        automationTest.beafBAtest("HVS180", "some-test-file")
       //How should I verify that JObData.instance.BAtests has the correct value

    }

My questions are:

1. How can I manage to run the test pass the missing dependencies such as missing dependency antlr/ANTLRException, is there any chance to fill gradle dependencies automatically?
2. Am I doing something wrong? how the test should be done? how should I check the singleton content at the end of the test?, should I put the test inside package org.company?

[pniederlag]

is this still an open issue for you?

[prudhvigodithi]

Hey I have a kind of similar problem

I have a library in the following format

lib = library(identifier: 'jenkins@main', retriever: modernSCM([
    $class: 'GitSCMSource',
    remote: 'https://github.com/prudhvigodithi/opensearch-build-libraries.git',
]))

So inside the libraries repo https://github.com/prudhvigodithi/opensearch-build-libraries.git i have a library vars/gradleCheckFlakyTestChecker that uses the code part of src/gradlecheck in the same repo and I have the following jenkinsfile

lib = library(identifier: 'jenkins@main', retriever: modernSCM([
    $class: 'GitSCMSource',
    remote: 'https://github.com/prudhvigodithi/opensearch-build-libraries.git',
]))


pipeline {
    agent { label 'testhost' }
    options {
        timeout(time: 1, unit: 'HOURS')
        buildDiscarder(logRotator(daysToKeepStr: '180'))
    }
    stages {
        stage('Sample Stage') {
            steps {
                script {
                    gradleCheckFlakyTestChecker()
                }
            }
        }
    }
    post() {
        always {
            script {
                postCleanup()
            }
        }
    }
}

The Jenkinsfile works and the job runs as expected. Now to test the pipeline and jenkinsfile I dont see a way to mock the classes part of src/gradlecheck that is used by library gradleCheckFlakyTestChecker part of jenkinsfile. This throws an error when tested as

super.testPipeline('jenkins/gradle/gradle-check-flaky-test-issue-creation.jenkinsfile',
                'tests/jenkins/jenkinsjob-regression-files/promotion/gradle-check-flaky-test-issue-creation.jenkinsfile')

Is there a way I can mock the external classes part of the gradleCheckFlakyTestChecker when running the JenkinsPipelineUnit?