During CI Pipeline mvn clean package fails creating HTML in target/generated-docs

[dlwhitehurst]

Thank you for taking your time to talk with us!

What is this issue about?
I can build my asciidoc rendering (HTML) locally using mvn clean package, however in my Jenkins pipeline the same call fails. After download of all dependencies (no local .m2/ cache) the mvn goal fails.

• [? ] Bug report
• Feature request
• Question

Description
Here's the actual error in my pipeline console log.
[INFO] --- asciidoctor-maven-plugin:2.1.0:process-asciidoc (asciidoc-to-html) @ labdocs ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.983 s
[INFO] Finished at: 2022-01-11T14:52:32Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.asciidoctor:asciidoctor-maven-plugin:2.1.0:process-asciidoc (asciidoc-to-html) on project labdocs: Execution asciidoc-to-html of goal org.asciidoctor:asciidoctor-maven-plugin:2.1.0:process-asciidoc failed: (ENOENT) No such file or directory - file:/home/jenkins/agent/workspace/labdocs/repo/open-uri-cached-0.0.5.gemspec -> [Help 1]

Environment information

• asciidoctor-maven-plugin version: ___2.2.1
• asciidoctorj version: ___2.5.2
• jruby version ____9.2.17.0

• Maven, Java and OS version: ___
  Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T15:39:06-04:00)
  Maven home: /Users/dlwhitehurst/Tools/apache-maven-3.5.0
  Java version: 11.0.10, vendor: AdoptOpenJDK
  Java home: /Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
  Default locale: en_US, platform encoding: UTF-8
  OS name: "mac os x", version: "10.16", arch: "x86_64", family: "mac"

java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)

This is frustrating because I do a Dockerfile just after the docs are generated and I want the pipeline to put the image with the docs in my Nexus ready to deploy.

Thanks in advance for any help. I hope this is the right place to address this issue.

[abelsromero]

That path is suspicious file:/home/jenkins/agent/workspace/labdocs/repo/open-uri-cached-0.0.5.gemspec. It belongs to one of the gems embedded within AsciidoctorJ jar.

Is the build setting the gempath? Also, it should not affect, but just in case can you validate if Jenkins is setting GEM_HOME environment variable in the build?

[dlwhitehurst]

I dumped the ENV and did not see GEM_HOME or anything related to Ruby, Gems, etc. I can set GEM_HOME prior to the goal but I don't know what the path should be.

[abelsromero]

I can set GEM_HOME prior to the goal but I don't know what the path should be.

You should not. But I wonder how is the path being injected. @robertpanzer do you know how this could be getting into asciidoctorj?

[robertpanzer]

Sorry, no real idea why it should be looking at the file system if GEM_HOME is not set.
Is the class path maybe pointing to this path?

[dlwhitehurst]

I create this path /home/jenkins/agent/workspace/labdocs/repo/ with the pipeline library. Jenkins agent (on Kubernetes) always has workspace and labdocs is the name of the Pipeline job in Jenkins. The repo is just a workdir that I created in the pipeline library code prior to checking out my asciidoc project from Git.

[ciano1000]

Just chiming in as I am having a very similar issue with the plugin when running as part of a Jenkins CI pipeline:

Failed to execute goal org.asciidoctor:asciidoctor-maven-plugin:2.2.2:process-asciidoc (convert-to-html) on project measures_cases: Execution convert-to-html of goal org.asciidoctor:asciidoctor-maven-plugin:2.2.2:process-asciidoc failed: (ENOENT) No such file or directory - file:/var/lib/jenkins/workspace/res_cases_bugfix_reports-jenkins/ruby-rc4-0.1.5.gemspec: (LoadError) no such file to load -- asciidoctor -> [Help 1]

Also like above, I am not altering the gempath in any way, and when dumping the env vars, no references to gempath or gemhome can be found

[abelsromero]

@ciano1000 Can you run maven with -X and attach the output (beware of sensible information that may be contained)? The only thing that comes to my mind now is if there's some extra libraries being injected 🤔

[uniqueck]

Is it possible to see your pom.xml file? Do you have declare some dependencies for the asciidoctor-maven-plugin? Can you paste your tree of project repository, maybe you use some special features from asciidoctor-intellij plugin to support preview of asciidoc content via special extension's?

[wes-johnson]

I ran into a similar issue with my Gradle build and thought others might find this helpful. This was my original error:
Exception in thread "main" org.jruby.exceptions.SystemCallError: (ENOENT) No such file or directory - file:/home/jenkins/agent/workspace/Sparkplug/open-uri-cached-0.0.5.gemspec

I turned on debug for the build and saw classpath paths like this:
/home/jenkins/agent/workspace/Sparkplug/?/.gradle/caches/jars-9/4a3c93384b3944d8466912d1bf5de712/asciidoctor-gradle-jvm-3.3.2.jar

With that - I found that the '?' in the path was a result of not setting GRADLE_USER_HOME. I added this to my startup script along with the JVM argument '-Dfile.encoding=UTF-8' and the build now succeeds in Jenkins. This is the final command I used in the Jenkinsfile:
sh 'GRADLE_USER_HOME="/home/jenkins/.gradle" ./gradlew -Dorg.gradle.jvmargs="-Xmx1536m -Xms64m -Dfile.encoding=UTF-8 -Djava.awt.headless=true" clean build'

I suppose the GRADLE_USER_HOME variable is not used by Maven but seems like a similar pathing issue.

[Datascan-Michael]

Also having this issue in Jenkins Ci pipeline. The build is running in a Maven docker image FROM maven:3.8.5-openjdk-11

[ERROR] Failed to execute goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc (output-html) on project api: Execution output-html of goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc failed: (ENOENT) No such file or directory - file:/var/jenkins/workspace/merge-request_MR-1654/Ascii85-1.1.0.gemspec: (LoadError) no such file to load -- asciidoctor

"/var/jenkins/workspace/merge-request_MR-1654/" is the base path for the Jenkins workspace where build is run from.
Based on the comments from others it shouldn't be trying to look there for the gems that are in a jar.

MAVEN_HOME is set and JAVA_HOME is set. GEM_HOME is not set.

Any ideas to get this working?

Plugin configurations in the pom

     <!-- Run the generated asciidoc through Asciidoctor to generate other documentation types, such as PDFs or HTML5 -->
        <plugin>
            <groupId>org.asciidoctor</groupId>
            <artifactId>asciidoctor-maven-plugin</artifactId>
            <version>2.2.4</version>
            <!-- Include Asciidoctor PDF for pdf generation -->
            <dependencies>
                <dependency>
                    <groupId>org.asciidoctor</groupId>
                    <artifactId>asciidoctorj-pdf</artifactId>
                    <version>2.3.9</version>
                </dependency>
                <!-- Comment this section to use the default jruby artifact provided by the plugin -->
                <dependency>
                    <groupId>org.jruby</groupId>
                    <artifactId>jruby</artifactId>
                    <version>9.4.2.0</version>
                </dependency>
                <!-- Comment this section to use the default AsciidoctorJ artifact provided by the plugin -->
                <dependency>
                    <groupId>org.asciidoctor</groupId>
                    <artifactId>asciidoctorj</artifactId>
                    <version>2.5.10</version>
                </dependency>
            </dependencies>

            <!-- Configure generic document generation settings -->
            <configuration>
                <baseDir>${project.basedir}</baseDir>
                <sourceDirectory>${project.basedir}/src/main/resources/asciidoc/</sourceDirectory>
                <sourceDocumentName>index.adoc</sourceDocumentName>
                <attributes>
                    <doctype>book</doctype>
                    <toc>left</toc>
                    <toclevels>3</toclevels>
                    <numbered></numbered>
                    <hardbreaks></hardbreaks>
                    <sectlinks></sectlinks>
                    <sectanchors></sectanchors>
                    <sourcedir>${project.build.sourceDirectory}</sourcedir>
                </attributes>
                <skip>${output.generated.docs.skip}</skip>
            </configuration>
            <!-- Since each execution can only handle one backend, run
                 separate executions for each desired output type -->
            <executions>
                <execution>
                    <id>output-html</id>
                    <phase>test</phase>
                    <goals>
                        <goal>process-asciidoc</goal>
                    </goals>
                    <configuration>
                        <backend>html5</backend>
                        <attributes>
                            <source-highlighter>coderay</source-highlighter>
                        </attributes>
                        <outputFile>${output.generated.docs.html}/docs.html</outputFile>
                    </configuration>
                </execution>
                <execution>
                    <id>output-pdf</id>
                    <phase>test</phase>
                    <goals>
                        <goal>process-asciidoc</goal>
                    </goals>
                    <configuration>
                        <backend>pdf</backend>
                        <attributes>
                            <source-highlighter>coderay</source-highlighter>
                            <icons>font</icons>
                            <pagenums/>
                            <toc/>
                            <idprefix/>
                            <idseparator>-</idseparator>
                        </attributes>
                        <outputFile>${output.generated.docs.pdf}/docs.pdf</outputFile>
                    </configuration>
                </execution>
            </executions>
        </plugin>

Error stack trace

11:44:16  [ERROR] Failed to execute goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc (output-html) on project api: Execution output-html of goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc failed: (ENOENT) No such file or directory - file:/var/jenkins/workspace/merge-request_MR-1654/Ascii85-1.1.0.gemspec: (LoadError) no such file to load -- asciidoctor -> [Help 1]
11:44:16  org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc (output-html) on project wi-api: Execution output-html of goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc failed: (ENOENT) No such file or directory - file:/var/jenkins/workspace/merge-request_MR-1654/Ascii85-1.1.0.gemspec
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:306)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:211)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:165)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:157)
11:44:16      at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:121)
11:44:16      at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
11:44:16      at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
11:44:16      at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:127)
11:44:16      at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
11:44:16      at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
11:44:16      at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
11:44:16      at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
11:44:16      at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
11:44:16      at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
11:44:16      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
11:44:16      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
11:44:16      at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
11:44:16      at java.lang.reflect.Method.invoke (Method.java:566)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
11:44:16  Caused by: org.apache.maven.plugin.PluginExecutionException: Execution output-html of goal org.asciidoctor:asciidoctor-maven-plugin:2.2.4:process-asciidoc failed: (ENOENT) No such file or directory - file:/var/jenkins/workspace/merge-request_MR-1654/Ascii85-1.1.0.gemspec
11:44:16      at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:301)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:211)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:165)
11:44:16      at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:157)
11:44:16      at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:121)
11:44:16      at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
11:44:16      at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
11:44:16      at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:127)
11:44:16      at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
11:44:16      at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
11:44:16      at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
11:44:16      at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
11:44:16      at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
11:44:16      at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
11:44:16      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
11:44:16      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
11:44:16      at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
11:44:16      at java.lang.reflect.Method.invoke (Method.java:566)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
11:44:16      at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
11:44:16  Caused by: org.jruby.exceptions.SystemCallError: (ENOENT) No such file or directory - file:/var/jenkins/workspace/merge-request_MR-1654/Ascii85-1.1.0.gemspec
11:44:16      at org.jruby.RubyIO.sysopen (org/jruby/RubyIO.java:1255)
11:44:16      at org.jruby.RubyFile.initialize (org/jruby/RubyFile.java:362)
11:44:16      at org.jruby.RubyClass.new (org/jruby/RubyClass.java:931)
11:44:16      at org.jruby.RubyIO.open (org/jruby/RubyIO.java:1174)
11:44:16      at RUBY.open_file (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb:783)
11:44:16      at RUBY.data (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/stub_specification.rb:113)
11:44:16      at RUBY.valid? (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/stub_specification.rb:200)
11:44:16      at org.jruby.RubyArray.select (org/jruby/RubyArray.java:2888)
11:44:16      at RUBY.gemspec_stubs_in (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:786)
11:44:16      at RUBY.map_stubs (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:801)
11:44:16      at org.jruby.RubyArray.each (org/jruby/RubyArray.java:1987)
11:44:16      at org.jruby.RubyEnumerable.flat_map (org/jruby/RubyEnumerable.java:911)
11:44:16      at RUBY.map_stubs (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:798)
11:44:16      at RUBY.installed_stubs (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:791)
11:44:16      at RUBY.stubs_for_pattern (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:856)
11:44:16      at RUBY.stubs (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:819)
11:44:16      at RUBY.find_by_path (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1029)
11:44:16      at RUBY.try_activate (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb:200)
11:44:16      at RUBY.require (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:153)
11:44:16      at RUBY.<main> (<script>:12)
11:44:16  Caused by: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- asciidoctor
11:44:16      at org.jruby.RubyKernel.require (org/jruby/RubyKernel.java:1057)
11:44:16      at RUBY.require (uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85)
11:44:16      at RUBY.<main> (<script>:12)

[abelsromero]

@Datascan-Michael can you provide full logs with -X (beware of sensible information that may be contained). Since I am not able to reproduce we need a detailed view of the paths and libraries to start digging.

Also, can you explain a bit this reference to a gemspec in merge-request_MR-1654/Ascii85-1.1.0.gemspec. The pom does not show extensions, but could it be there's some Ruby environment messing?

[Datascan-Michael]

merge-request_MR-1654 is the folder that was dynamically generated by Jenkins where the project was cloned and the ci pipeline is run from.

I too suspect that there is something missing in the environment as this issue only happens when run thru the CI pipeline where the build is run inside the docker image I mentioned in my original comment. I don't know what it could be. Running maven on my computer the process completes successfully.

I'll see what I can do about getting a full log but it probably reveals too much about our specific project. I'll attempt a simplified version and see what I can come up with.

Until then... maybe some helpful information...

[2023-11-02T15:41:59.473Z] Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
[2023-11-02T15:41:59.473Z] Maven home: /usr/share/maven
[2023-11-02T15:41:59.473Z] Java version: 11.0.15, vendor: Oracle Corporation, runtime: /usr/local/openjdk-11
[2023-11-02T15:41:59.473Z] Default locale: en, platform encoding: UTF-8
[2023-11-02T15:41:59.473Z] OS name: "linux", version: "4.14.287-215.504.amzn2.x86_64", arch: "amd64", family: "unix"

[Datascan-Michael]

I tracked this down and have it working now. Buckle up since I'm going to provide a lot of details about Jenkins and how it runs docker build agents but ultimately, I think you can close this bug as I don't think it's directly related to asciidoctor-maven-plugin.

By default Jenkins will run docker build agents using its user id 1000. You see something like "docker run -u 1000:1000" in the build log where it starts up the docker image for the build agent. If that build image doesn't have a user 1000 or that user doesn't have a home directory for the local maven repository to be created in, then this issue happens because the parts deep in jRuby can't figure out where the local maven repository is to be able to load the necessary gems in a jar. With maven debug turned on you should see something like "[DEBUG] Using local repository at /var/jenkins/workspace/merge-request_MR-1662/?/.m2/repository" where it's setting the local repository to a path in the workspace but note the "?" in the path. It seems that the rest of maven can deal with this in the path just fine and the normal maven builds and java compile will work just fine.
There are a couple options to correct this and make sure a local repository path is set without a "?" in it:
First, you can override the default user that Jenkins runs the docker image with by passing in "-u someValidUser" in the args parameter for the docker agent in the build pipeline. Then the local repository will get set to that users home directory .m2/repository folder.
Second, you can use the "mavenLocalRepo" parameter for the Jenkins withMaven function to set a local repo location in the workspace instead of it trying to use the users home directory. See details here (https://plugins.jenkins.io/pipeline-maven/)
Third, update your Docker image to map the user id 1000 to a valid user with a home directory.

I tried all three of these options and the all resulted in successful runs of the asciidoctor-maven-plugin.