tag:blogger.com,1999:blog-43671086361396995562024-03-13T16:11:15.125+01:00Henrik Lynggaard's blogHenrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.comBlogger21125tag:blogger.com,1999:blog-4367108636139699556.post-2783492493050561382012-02-07T16:40:00.000+01:002016-11-06T23:51:52.782+01:00Ivy based subproject as maven moduleI have previously written about the challenges of integrating an Ivy based subproject (like the play framework) into a build that is otherwise Maven based. After some work it is now working although the support feels a bit rudimentary.Here are the highlights on how to make it work<br />
<br />
<a name='more'></a><br />
<b>Prepare play framework:</b><br />
The play framework must be installed and available to Hudson.<br />
<br />
<b>Dependency management is done by maven</b><br />
Instead of duplicating maven's settings.xml configuration in Ivy, you can rely on the fact that maven will resolve any dependencies and place them in the local Maven repository before any Ivy commands are executed. This way the Ivy settings only need to contain a reference to the local repository and not muck around with remote resolutions.<br />
<br />
Depending on how you create the dependencies.yml (see below), you might need to specify the dependencies twice, once for Maven and once for Ivy<br />
<br />
<b>Clean the extra directories:</b><br />
Since play likes to spread its files more than just the normal src/ and target/ make sure the clean plugin cleans these extra directories as well<br />
<br />
<b>Generate Ivy configuration (conf/dependencies.yml):</b><br />
Instead of checking in the Ivy config file into your SCM, you need to generate this as part of the build.This can be done using antrun to simply echo the file out before the compile stage. The reason is twofold.<br />
<br />
Firstly the path to the local maven repository needs to be set<br />
<br />
Secondly, if the dependencies.yml file is in SCM, then it would still point to the old versions when Maven is doing the release step unless some work is done to have Maven roll these versions forward and commit the changes to this file. It is easier to just generate the file on the fly.<br />
<br />
<b>Compile step:</b><br />
The "precompile" command of the play compiler needs to be invoked instead of relying on the normal java compiler. Using antrun/exec plugin to do this task is easiest. <br />
<br />
<b>Package step:</b><br />
This time the "war" command of the play compiler needs to be invoked. After the play compiler has run the resulting exploded war directory needs to be zipped and replace the war file generated by the normal war plugin so Maven picks up our custom package.<br />
<br />
Care should be taken as to where the exploded war file is generated to. The reason is that you can run into a play bug. If you generate the war file within the Maven module, you must also exclude the output directory in order to avoid endless recursion. That in itself is not the problem, The problem arises if you choose "target" as your output directory. Then you will have a clash between the play bug where the exclude directories are not evaluated from the module directory but on the full path, and the release plugin's checkout of the code into "target/checkout"<br />
<br />
<b>Releasing:</b><br />
Since the Ivy parts are not fully integrated into maven, it does not understand when maven puts target folders on the classpath. This causes a problem during the release:prepare step when maven only executes "clean verify", because then Ivy cannot find the built dependencies. The trick is to reconfigure release:prepare to run "clean install" as preparation goals.<br />
<br />
<b>Outstanding issues:</b><br />
<br />
<ul>
<li>Sometimes the play tests can leave hanging processes</li>
<li>Since play opens a socket, running concurrent build can clash. However the play part is usually so quick that even if SCM polling is set to every minute it is not a problem. </li>
</ul>
Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-28512751653040062192012-01-14T14:19:00.000+01:002012-01-14T14:19:20.206+01:00Why use the Hudson jobcreator tool?It might seem very counter intuitive to start using Hudson's XML format to manage Hudson jobs when it has such a great web interface, especially since it is the web interface which has made Hudson so approachable and easy to use.<br />
<br />
Don't get me wrong. I still love the web interface and think it is the right way to get people to use the tool, however I also think there comes a time when you outgrow the web interface and this is why I built the jobcreator tool.<br />
<br />
These are the requirements and issues that caused me to outgrow the web interface:<br />
<br />
<b>Manual changes doesn't scale</b><br />
Using the web interface to make changes to individual jobs is very easy, but it doesn't really scale if you need to change a lot of jobs on the same time. One example could be changing the git branch from "master" to a release branch for all the jobs related to a environment, Such a change could involve upwards of 30 jobs.<br />
<br />
Another example could be that there are changes to the content and/or structure of the jobs and those must be propagated up through the environments in sync with project code.This would be even more manual changes.<br />
<br />
Managing this with manual changes in the web interfaces introduces a big risk for human error and inconsistencies.<br />
<br />
<b>Hudson jobs are code</b><br />
If you want to be able to reproduce a build or deployment at a later time it is important that you can also reproduce the Hudson jobs. In order to do this you need to store and version your Hudson job configurations.<br />
<br />
This is naturally best done in a SCM like Git or Subversion. Doing so also gives you the option to do branched development of your jobs.<br />
<br />
<b>Testing and more than one Hudson instance.</b><br />
Before making changes to the jobs being actively used the changes should naturally be tested somewhere. This normally means creating the same set of jobs somewhere else or targeted towards a different environment.<br />
<br />
Having the jobs defined as templates makes it easy for a developer to load the jobs into a private Hudson instance to experiment or share it on the testing instance.<br />
<br />
<b>Overall</b><br />
Of course Hudson can be managed via the web interface, but for me it is just another step in the automation.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-48154576114000437712012-01-13T00:24:00.000+01:002012-01-13T00:24:29.214+01:00Announcing the Hudson job creator tool.I have finally had the chance to pull together the last changes before announcing the first version of the Hudson Job Creator tool.<br />
<br />
The idea behind the tool is that you can write FreeMarker based templates and combine those with properties defined in a "pipeline" specification in order to generate Hudson's job config.xml files.<br />
This is mainly useful if you maintain a number of similar jobs, or have a series of jobs that you need to specify for multiple environments.<br />
<br />
I know working with Hudson's xml files directly can seem counter intuitive since one of Hudson's main strengths is its approachability and easy to use web interface, so later this week I will post a more in depth blog post<br />
explaining why I chose to go this route.<br />
<br />
All the information about the tool including download is available on github, <a href="https://github.com/hudson-plugins/jobcreator-tool">https://github.com/hudson-plugins/jobcreator-tool</a> .Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-39188992638611952472012-01-08T13:47:00.001+01:002012-01-08T13:51:41.026+01:00Maven bug fixed making Hudson's Maven3 integration much more usefulBack in July when I reviewed the Maven3 integration in Hudson 2.1, I made a point regarding it not working when doing a maven release. As written back then the cause is a Maven bug (<a href="http://jira.codehaus.org/browse/MRELEASE-577" style="color: #3778cd; text-decoration: none;">release:prepare does not pass argument "--settings" with current settings.xml to inner maven</a>).<br />
<br />
This bug has now been fixed and a new version of the release plugin has been released. So if you update the release plugin to version 2.2.2, Hudson's maven3 integration will now work for releases also.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-18886809893752762532011-11-08T13:12:00.000+01:002011-11-08T13:12:36.146+01:00Mixing Ivy into a Maven build?<b>Current situation:</b><br />
On one of the projects I am working on I have the following "pipeline"<br />
<ol><li>Build environment neutral artefacts with a snapshot version like 1.2.0-SNAPSHOT (using Maven 3 and Hudson M3 integration)</li>
<li>Every hour deploy and test the latest successfully build artefacts to dev #1</li>
<li>Every day deploy and test the latest artefacts which have been successfully tested in dev #1 to dev #2</li>
<li>On demand do a Maven release of the latest artefacts that have passed testing in dev #2</li>
</ol><br />
The tricky part of this pipeline comes in the dev #2 environment because it needs a way to select the latest artefacts which have been tested in dev #1 instead of just the latest artefacts produced. The same goes for making the release, I need some way to identify which artefacts have been tested successfully in dev #2.<br />
<br />
This means I cannot rely on just picking the latest snapshot deployed to the internal repository.<br />
<br />
We have considered different options:<br />
<br />
<ul><li>Publish the snapshot to a internal repository and carry around the timestamp of the specific snapshot version. This would allow us to pin out the version, but gives us a problem with regards to cleaning up unused snapshots as Nexus can only do keep X days or X versions, but not remove a set of artefacts based on a timestamp. Also it has the downside of people questioning why we should even do Maven releasing if we already have a unique identifier.</li>
<li>A second approach would be to use the "copy artefacts" Hudson feature, but then the way we get artefacts would be different between dev and the upper environments.</li>
<li>The approach we have settled on is to not deploy the snapshots to the nexus repo, but use the hudson maven repository plugin. This plugin exposes each build as its own repository. In order to get the right artefacts we use a custom settings.xml to mirror the snapshot repository to the URL of the specific build as exposed by the Hudson plugin. This plugin only works with either the jobs Maven 2 jobs or the new Maven 3 integration, since it relies on Hudson understanding the build and the artefacts. We use the promoted builds plugin to identify the correct build, and we use the promotion status for easy clean up of non used artefacts.</li>
</ul><br />
<br />
We haven't looked too much into using nexus pro's features such as staging repositories or adding metadata, since the above approach works fairly well for us.<br />
<br />
<b>The new challenge:</b><br />
The reason for writing this post and asking for help is a possible change to our process which I am not sure how to best integrate.<br />
There is a wish to integrate a component (playframework based webapp) build using ivy into this framework and specifically into this project. I can see this causing some integration pains<br />
<br />
<ul><li>Firstly the project is one big Maven multi module project and we prefer to keep it that way. As a very least we want to keep things as one build i.e one Hudson job for building. Is it possible to have a Maven submodule defer execution to Ivy ?</li>
<li>If we get the component built using ivy only, then Hudson will not be able to see the outcome as a Maven artefact, and as thus wont be able to expose it as part of the "repository per build". That is at least my strong suspicion. </li>
</ul><br />
<b>What to do ?</b><br />
So does anyone have a suggestions on how to resolve this ?<br />
<br />
<ul><li>Can we cleanly integrate Ivy into our current build, if so how ?</li>
<li>Do we need to find another approach than using the "repository per build" plugin ? if so, what is the suggested alternative.</li>
<li>Would Nexus/Artifactory paid editions make this easier ?</li>
</ul><br />
Just to be clear my requirements are:<br />
<br />
<ul><li>From the outside it must appear as one build i.e. one Hudson job.</li>
<li>For a developers desktop build it is okay to be a multi step process</li>
<li>We need to be able to specify a particular build to deploy in dev #2 and for releasing.</li>
<li>We would like to continue the use the promoted builds plugin to visualize good builds</li>
<li>Clean up strategy should be easy</li>
</ul>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com1tag:blogger.com,1999:blog-4367108636139699556.post-91806914471511124172011-10-26T21:47:00.000+02:002011-10-26T21:47:05.130+02:00Review of "Apache Maven 3 Cookbook" by "Srirangan"This is a review of <a href="http://www.packtpub.com/apache-maven-3-0-cookbook/book">Apache Maven 3 Cookbook</a> written by "<a href="http://www.packtpub.com/authors/profiles/srirangan">Srirangan</a>". I got a free copy from Packt publishing for the purpose of the review. I have been using Maven for some years now and this book is a introductory book, so it was clear from the beginning that I am not in the target audience.<br />
<br />
The style of splitting the book into 50 recipes makes for a good format which is easy to read and breaks the book into small achievements for the reader.<br />
<br />
Instead of focusing solely on how Maven is configured, the author tries to tie some of the subjects to software development practices e.g. covering Nexus and Hudson while explaining team collaboration. It serves the book well to put Maven into a development perspective, but it doesn't always fit with the recipe format. For instance in the Nexus case from above the "How it Work" section becomes more a "why it is good" section.<br />
<br />
I like the fact that the book covers a wide range of different project types and topics. Many times when you read tutorials or other documentation only the simplest project types are covered leaving the reader to add plugins as needed. This book covers many project types and framework and some non-Java areas. It also covers things like setting up Nexus,Hudson, various IDEs. It even has a single chapter on plugin development.<br />
<br />
In the first chapter the level of information in each recipe is appropriate, but as the chapters get more complex the level of information does not. This results in many of the recipes being too simplistic. A prime example the is set-up of remote repositories, it describes in great many screen shots how to install tomcat 7 and deploy nexus, but only has a single line of information on how to set-up the remote repository, plus only mentions (incorrectly) changing the settings.xml and not the required changes to the project object model. So it fails to help the reader define and use the remote repository possible leaving the reader with a broken set-up<br />
<br />
This simplistic approach has another side effect. In many of the recipes there is clear copy-paste'able examples but very little explanation of why things are the they are. An example would be the first recipe which introduce multi-module projects. In the top level project definition the dependencies are placed in "<dependencyManagment>" section instead of the normal "<dependencies>" section without an explanation of why.<br />
<br />
<b>Conclusion:</b><br />
While I like the style and long list of topics covered in this book, I think the decision not the explain the details of why things work like they do e.g. <dependencies> vs <dependencyManagment> or how repositories works, does the book a big disservice.<br />
<br />
I would not recommend learning Maven from this book alone, since I think explaining the "why" is an essential part of learning a new tool. If you want to learn Maven use the free sonatype book <a href="http://www.sonatype.com/books/mvnref-book/reference/index.html">Maven: The Complete Reference</a> and buy a copy of this book if you like a quick introduction to the various project types and plugins.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-36546775632188161742011-10-13T11:24:00.000+02:002011-10-13T11:24:25.280+02:00Using the SoapUI maven integrationI have had the mixed pleasure of using the Maven integration for SoapUI (<a href="http://www.soapui.org/Test-Automation/maven-2x.html">http://www.soapui.org/Test-Automation/maven-2x.html</a>).<br />
<br />
The tool itself is pretty good and makes it very easy to test that our soap based web-services are working as intended. The fact that is actually provides Maven and Junit integration out of the box is even better and fits very nicely with our CI environment.<br />
<br />
There is however a few things that are not obvious when using the plugin.<br />
<div><br />
</div>The documentation page is really old, e.g. it refers to an old (2.5.1) version of the plugin. The trick here is that you should in general use the same version, as the desktop version you are running. In my case that is 4.0.0<br />
<br />
<div>It isn't documented on the page but there here is both a "<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">maven-soapui-plugin</span>" and "<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">maven-soapui-pro-plugin</span>" version of the plugin. In order to fully use project created using the pro version you need the pro version of the plugin</div><br />
<div>The version 4.0.0 of the plugin has a misconfiguration so you will need to manually add some dependencies to the plugin. The version I got working looks like this.<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace; font-size: small;"><plugin><br />
<groupId>eviware</groupId><br />
<artifactId>maven-soapui-pro-plugin</artifactId><br />
<version>4.0.0</version><br />
<configuration><br />
....<br />
</configuration><br />
<executions><br />
....<br />
<executions><br />
<dependencies><br />
<dependency><br />
<groupId>jgoodies</groupId><br />
<artifactId>looks</artifactId><br />
<version>2.2.0</version><br />
</dependency><br />
<dependency><br />
<groupId>fife</groupId><br />
<artifactId>rsyntaxtextarea</artifactId><br />
<version>1.3.4</version><br />
</dependency><br />
<dependency><br />
<groupId>junit</groupId><br />
<artifactId>junit</artifactId><br />
<version>4.4</version><br />
</dependency> <br />
</dependencies><br />
</plugin></blockquote></div><div>The two first dependencies are so that the plugin can run, and the last one is needed if there are any XPath assertions.<br />
<br />
I have tried to upgrade to the latest 4.0.1 version which has just been released but I get the following error, so I suggest people stick to 4.0.0 for now<br />
<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace; font-size: small;">class "com.eviware.soapui.SoapUICore"'s signer information does not match signer information of other classes in the same package</blockquote></div>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com3København, Danmark55.6760968 12.56833710000000855.6214323 12.450636100000008 55.730761300000005 12.686038100000008tag:blogger.com,1999:blog-4367108636139699556.post-6147849629797208802011-07-31T19:37:00.000+02:002011-07-31T19:37:29.512+02:00Announcing the first version of "Hudson Maven Repository Server Plugin"One of the things I highlighted under "the bad" during my review of the new Maven 3 Integration in Hudson 2.1 was the fact that the "Jenkins Maven repository Server" did not work with it.<br />
<br />
This is of course still true but now the first version of the Hudson equivalent has been released. Since it is the first version, my focus has been on making it work in our own pipeline leaving the extra stuff for later. This means<br />
<br />
<ul><li>It only supports normal free-style jobs not multi-configuration</li>
<li>There is no repository chain or git commit functionality, only direct build by number</li>
<li>metadata.xml is not created. </li>
</ul><div>The last item can cause you some issues, as I have seen maven think its local version is newer than the repository version when it cannot find the metadata, so I advice you to manually remove/clean the old artifacts away.</div><div><br />
</div><div>I hope to support both multi-configuration projects and metadata files in the next version, but they might require core changes.</div><div><br />
</div><div>The plugin has gone through the staging in Sonatype's OSS nexus, so it should appear in maven central and the hudson update site shortly.</div><div><br />
</div><div><br />
</div>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0Copenhagen, Denmark55.693403 12.58304599999996855.6387385 12.465344999999967 55.748067500000005 12.700746999999968tag:blogger.com,1999:blog-4367108636139699556.post-75349416438010533422011-07-29T23:45:00.001+02:002011-07-29T23:49:54.909+02:00A review of the new Maven3 integration in Hudson 2.1I have been looking forward to the new Maven3 integration in Hudson for some time now, and with the release of Hudson 2.1 it is finally here.<br />
<br />
My main interest in the new integration is because we rely heavily on the ability to expose a maven build as a maven repository but the current (jenkins) plugin only works with the Maven 2 job type. That setup works nicely, but the Maven 2 job type has some performance and scalability issues<br />
<br />
So we have been looking forward to getting the (hopefully) best of both worlds, fast as the freestyle job type, but still enough metadata to expose the build as a Maven repository.We mostly got it, see the details below.<br />
<br />
<b>The good</b><br />
<ul><li>Our build time dropped from around 35 minutes to 12 minutes.</li>
<li>Our Hudson was acting really slow with regards to page rendering etc, but after switching to the new style integration it has become significantly faster. I think it is because of changes to the metadata stored in memory.</li>
<li>I like the the Maven information browser on the build pages. Right now I have only used it to see how far a build has come, but I imagine it is really useful if you need to look into individual modules.</li>
<li>The new document storages is a great idea which hasn't really come into its own yet. I think its potential is bigger than its current usage. I mean this in a good way because it is already great for managing settings.xml files, but there is so much more you could do with it.</li>
<li>You can now build a Maven job in a matrix build without loosing the metadata.</li>
<li>Many of the Maven options are now exposed as easy to use switches.</li>
</ul><b>The bad</b><br />
<ul><li>Some places it is evident in the user interface that the underlying presentation framework is different from the rest of Hudson. Some of the interface elements feel a bit "out of place", prime examples are the document storage screen and scrollbars.</li>
<li>Currently you cannot use the bundled maven3 to do maven releases if you rely on a custom settings.xml. Technically it is not Hudson's fault as it is merely a victim of a Maven bug (<a href="http://jira.codehaus.org/browse/MRELEASE-577">release:prepare does not pass argument "--settings" with current settings.xml to inner maven</a>). You can use a standalone installation though.</li>
<li>The underlying API exposing the metadata is a bit confusing, and there isn't any documentation yet. To be fair it also has to deal with more things than the original Maven job type APIs,The problem space is simply bigger now that an arfifact can be created in multiple build steps or combinations in a matrix job.</li>
<li>There are a few missing items in the API, e.g. there is no way to get a file link to a archived artifact.</li>
<li>You loose the functionality of the m2release plugin. I quite liked the fact that there was a manual button you could press on a job and have it perform a Maven release of the same job. Then it was visible to all in the build history when the release was done. Now you need to create 2 build jobs making the releases less visible</li>
<li>As far as I can tell you also loose the ability to easily see which modules where previously in a build but has now been removed. I have used that functionality more than once to catch when someone has removed a module but not all dependencies to it causing the release to fail.</li>
<li>You loose the use of "Jenkins Maven repository Server " plugin, but that went "Jenkins only" sometime after version 0.4 anyways and there is already a replacement underway.</li>
</ul><b>The potential</b><br />
<ul><li>Once people start extending the Maven integration there is a lot of potential e.g. why archive war and ejb files if they are included in ear files. Adding that will save tons of network traffic. </li>
<li>The document storage can become a real killer feature once someone implements that the documents can be given to other build steps than just maven3 steps.</li>
<li>And let us not forget the collateral benefits this features has brought with it, there is a new REST api, the GWT frontend framework, JSR 330 based plug-ins.</li>
</ul><br />
<b>Verdict</b><br />
The new Maven integration has provided us with a much needed and very impressive performance improvement both in build time where it went down from 35 to 12 minutes, and in responsiveness of the user interface.<br />
<br />
For us this is the killer and the rest of the features are reduced to "icing on the cake" even though that might be a bit unfair. As you can see from the negative list there is still room for improvement, and I hope to see many of those shortcomings addressed now the rush to integrate is over.<br />
<br />
In conclusion, the new maven integration is a solid improvement, but it is even better as a stepping stone.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0København, Danmark55.693403 12.58304599999996855.6387385 12.465344999999967 55.748067500000005 12.700746999999968tag:blogger.com,1999:blog-4367108636139699556.post-7915230058053715722011-07-25T23:33:00.001+02:002011-07-25T23:33:56.384+02:00Debugging Hudson Git plugin hanging during fetch on Windows slave.Ran into a tricky problem today after switching a couple of our Hudson jobs from subversion to Git. The git plugin was able to clone the repository in a clean workspace, but after that simply hung in the fetch step.<br />
<br />
The job log did not provide much information<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace;">Started by user hlh005<br />
Building remotely on selenium-xp2<br />
Checkout:TestVerification / C:\hudson\workspace\TestVerification- hudson.remoting.Channel@6e7a8d98:selenium-xp2<br />
Using strategy: Default<br />
Last Built Revision: Revision b6d046ffede89c4bcdd5b71b7c6b703583cb6e18 (origin/master)<br />
Checkout:TestVerification / C:\hudson\workspace\TestVerification - hudson.remoting.LocalChannel@7835ec<br />
Fetching changes from the remote Git repository<br />
Fetching upstream changes from git@forge.example.com:my-repo</blockquote><div>Looking at the slave workspace, it was clear that it had actually managed to clone the repository, as there was both .git folder and the working directory content. So in some sense it must be able to contact the server.<br />
<br />
A Google search for the problem returned some results but in the majority people got a error message and not a hanging job. It did however reveal that a incorrect HOME environment variable could be the cause.<br />
<br />
In order to find out our current environment variables for the node I went to "Manage Hudson" -> "Manage Nodes" -> (name of the node) -> "System information". Here you can find both system properties, environment variables and thread dump.<br />
<br />
Besides confirming that we did not have a HOME environment variable, the thread dump revealed something very interesting. One of the thread had a thread name showing the exact git command line being executed and this thread was waiting for input reading on a socket.<br />
<br />
Trying this command on the slave in a normal command promt, confirmed the fact that the git command could not find the key and thus failed to talk to the remote repository. Don't use the git bash you get with msysgit, Hudson does not use this. Use a standard "cmd" promt opened from the start menu.<br />
<br />
The solution was to set the HOME environment variable and restart the Hudson slave, but at least one question remains....<br />
<br />
How was Hudson able to clone the repository when the keys could not be found. Is this another git plugin weirdness or does our gitolite setup have a error ?<br />
<br />
I have created a Hudson issue have it provide better feedback see: <a href="http://issues.hudson-ci.org/browse/HUDSON-8949">Git plugin hangs instead of providing error message when it cannot find the ssh keys (windows slave)</a></div>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-90272914855288746512011-05-08T12:56:00.000+02:002011-05-08T12:56:55.481+02:00My thought on the Eclipse proposalIt has been a very interesting week in Hudson/Jenkins land with the announcement of the Eclipse proposal. The responses have been very varied ranging from the thoughtful and constructive to lets-start-another-mudslinging match. Luckily the mudslinging part is much less this time, and seem to have died down quickly.<br />
<br />
Before I get into the details, let me just say that nothing in the proposal changes my overall perception, I remain fork neutral and any plug-in I develop will work on both if at all technically possible.<br />
<br />
From my point of view I am mostly positive towards the proposal,<br />
<br />
<ul><li>I think this is a good move, bringing it to eclipse can hopefully reduce some of the bad blood and knee-jerk reactions when people mention Hudson/Oracle, and in time this will hopefully manifest as better corporation between the two projects.</li>
<li>I am very pleased that this has made sonatype contribute even more to the open source version instead of hudson pro. :-)</li>
<li>It is nice to see that more companies are investing in Hudson, for people like me that tries to remain fork neutral it is good to see that the effort is not "wasted". </li>
</ul><br />
but it does leave me with some concerns and questions:<br />
<ul><li>One of mentioned reasons for going with Eclipse is the better governance model, but I haven't seen any explanation on how this governance will be performed except that 2 people are listed as project lead. Maybe it is just following a standard eclipse governance ? (which I haven't found the link for yet). Some basic clarification would be nice, like: Will it be the comiters who are in control, or will there be some sort of governance board ? if a board how is the election process?</li>
<li>I can see why Oracle want to move to a neutral 3. party, but I haven't understood why eclipse and not something like Apache ? </li>
<li>How will the re-licensing affect the possibility to port fixes between the forks ? my initial understanding is that since MIT code can be re-licensed as EPL but not the other way, fixes can be ported only from Jenkins to Hudson. Am I correct ? </li>
<li>One of my major concerns with this move is the requirement to use eclipse infrastructure. I haven't been a comitter to eclipse based projects so I am not overly familiar with it, but looking at it does appear somewhat dated compared to java.net + github. I am especially concerned about loosing the easy fork and pull workflow. Are we really loosing that functionality entirely, and if so what is the replacement?</li>
<li>Another of my concerns is that contributing to the core seems more geared towards company sponsored development, leaving less room for individual free-time contributors like me to help out or get a say in the development. Maybe it is just caused by my own lack of knowledge, but perhaps someone can clarify ?</li>
<li>I haven't seen any discussion on the developer mailing list or JIRA on how to replace the LGPL components. This discussion should be started.</li>
</ul><div><br />
</div><div>So in summary: Good move, but more details should be posted</div>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com1tag:blogger.com,1999:blog-4367108636139699556.post-10529413897000719272011-04-21T13:49:00.000+02:002011-04-21T13:59:00.534+02:00Supporting both Hudson & Jenkins - Part 5: Improving the situationI have a number of Hudson/Jenkins issues in order to make improve the dual support situation. I have listed them below for those who wish to follow track them.<br />
<br />
Hudson<br />
<a href="http://issues.hudson-ci.org/browse/HUDSON-8822">hudson-plugin-info macro assumes subversion when it can't find github</a><br />
<a href="http://issues.hudson-ci.org/browse/HUDSON-8823">hudson-plugin-info assumes issues being managed in hudson jira</a> (created this one before seeing the duplicate)<br />
<a href="http://issues.hudson-ci.org/browse/HUDSON-7191">Allow specifying an alternative issue tracking URL in plugin information</a> (this one already existed)<br />
<a href="http://issues.hudson-ci.org/browse/HUDSON-8824"> Update centre json should take information from pom not wiki</a><br />
<br />
<br />
Jenkins<br />
<a href="https://issues.jenkins-ci.org/browse/JENKINS-9467">jenkins-plugin-info macro assumes subversion when it can't find github</a><br />
<a href="https://issues.jenkins-ci.org/browse/JENKINS-7191">Allow specifying an alternative issue tracking URL in plugin information</a> (this one already existed)<br />
<a href="https://issues.jenkins-ci.org/browse/JENKINS-9468">Update centre json should take information from pom not wiki</a><br />
<br />
There is properly going to be more later, but these are the things that are most visible to the end users.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com1tag:blogger.com,1999:blog-4367108636139699556.post-20183592604169681952011-04-18T15:04:00.000+02:002011-04-18T15:06:24.678+02:00Supporting both Hudson & Jenkins - Part 4: Post release workAfter you have done the release there is a couple of tasks left in order for everything to work cleanly.<br />
<br />
<b>Create a stub page on the Hudson wiki.</b><br />
Go <a href="http://wiki.hudson-ci.org/display/HUDSON/Plugins">here</a> and create a child page<br />
- Make sure to add the hudson-plugin-info and excerpt macro.<br />
- Add labels to the page for correct groupin<br />
- Add a reference to where the plugin is hosted.<br />
<br />
<div>You can take a look at <a href="http://wiki.hudson-ci.org/display/HUDSON/Project+Health+Report+Plugin">Project Health Report Plugin</a> as an example</div><div><br />
</div><div><b>Create a stub page on the Jenkins wiki</b><br />
Go <a href="https://wiki.jenkins-ci.org/display/JENKINS/Plugins">here</a> and create a child page. You should follow the same convention as on the Hudson wiki, except that the project info macro is now called jenkins-plugin-info. An example is <a href="https://wiki.jenkins-ci.org/display/JENKINS/Prerequisite+build+step+plugin">here</a>.<br />
<br />
<b>Announce the plug-in</b><br />
Announce the plug-in on the hudson and jenkins developer list. Be sure to mention it is hosted elsewhere because both projects need to adjust their update centre script.<br />
<br />
This is because their scripts try to match the wiki url from the pom with their respective wikis in order to provide a link and description in the update centre. When it can't find the wiki page, the information will be blank.<br />
<br />
<br />
</div>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com1tag:blogger.com,1999:blog-4367108636139699556.post-35452846011498965622011-04-18T14:39:00.001+02:002011-04-18T14:40:19.760+02:00Supporting both Hudson & Jenkins - Part 3: The Release processThe release process is actually fairly simple even though it is not a one click process. I normally does the Hudson release first because then I can validate things in the nexus staging repository.<br />
<br />
My development environment is set up so that I have my SCM (bitbucket) credentials, and the repository credentials, such that I do not need to provide them in the process.<br />
<br />
<b>Step Zero: Prerequisites</b><br />
Make sure that everything you have is committed and you have performed a "mvn clean release:clean"<br />
<br />
<b>Step One: Running release prepare</b><br />
Simply run "<i>mvn release:prepare</i>". I usually run it in interactive mode and answers the version questions, but you can add parameters to run it in batch mode.<br />
<br />
<b>Step Two: Save the release.properties file</b><br />
Now you actually need to run release:perform twice once per profile, but Maven cleans up the directory after the first release perform. Therefore you must provide extra parameters to the second release:perform.<br />
<br />
You will need to look at the <a href="http://maven.apache.org/plugins/maven-release-plugin/perform-mojo.html">documentation</a> on which parameters to specify, maybe you can get lucky and only specify "tag". I however use another short cut, the release.properties that release:prepare generated contains all the needed information, so I save this by making a copy outside the project directory<br />
<br />
<b>Step Tree: Do the Hudson release</b><br />
Now that you have taken note of the needed properties or simply saved the release.properties file, run "<i>mvn release:perform -Phudson-publish</i>"<br />
Now the plug-in should be in the nexus OSS staging repository. Verify and release it as described <a href="https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide#SonatypeOSSMavenRepositoryUsageGuide-8.ReleaseIt">here</a><br />
<br />
<b>Step Four: Do the Jenkins release</b><br />
Now copy the release.properties back into the project directory and run "<i>mvn release:perform -Pjenkins-publish</i>"Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-62939474962952216052011-04-10T23:48:00.001+02:002011-04-10T23:50:27.454+02:00Supporting both Hudson & Jenkins - Part 2: Settings.xml and the pom.xml<b>Overview:</b><br />
The main strategy for supporting both Hudson and Jenkins is to make the main pom structure neutral and then use two profiles to handle the version specifics. These profile which I have called hudson-publish and jenkins-publish is both represented in the settings.xml and the pom.xml.<br />
<br />
<b>pom.xml:</b><br />
In order to create a pom that works for both sides, I have done the following things.<br />
<ul><li>Specified the parent version as 1.392. This is the latest pre-fork parent which do not suffer from the bug which prevents publishing of "requiresCoreVersion". Full specification is org.jvnet.hudson.plugins:plugin:1.392</li>
<li>The groupId of the plug-in must be org.jvnet.hudson.plugins otherwise the publishing to maven central wont work.</li>
<li>The basic items like name, description and url must be set. The Hudson update centre uses the URL and description.</li>
<li>Remember to add license and developer information.</li>
<li>Remember to override scm and issueManagment section.</li>
<li>Don't specify distributionManagment outside the profiles.</li>
<li>Configure the release plugin to only make deploy not site-deploy and use forked-path which is needed by the gpg plug-in.</li>
</ul><blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace;"><plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-release-plugin</artifactId><br />
<version>2.1</version><br />
<configuration><br />
<mavenExecutorId>forked-path</mavenExecutorId><br />
<goals>deploy</goals><br />
</configuration><br />
</plugin></blockquote><br />
<b>settings.xml:</b><br />
There are some things that need to go into the settings.xml which are not profile specific. This is the list of server definitions, which you must add in order to provide username/passwords. I have server definitions for the following list<br />
<ul><li>sonatype-nexus-snapshots</li>
<li>sonatype-nexus-staging</li>
<li>maven.jenkins-ci.org</li>
</ul><br />
<b>jenkins-publish profile:</b><br />
In the settings.xml you need to specify the Jenkins maven repository<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace;"><repository><br />
<id>maven.jenkins-ci.org</id><br />
<name>Jenkins Repository</name><br />
<layout>default</layout><br />
<url>http://maven.jenkins-ci.org:8081/content/repositories/releases/</url><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
</repository></blockquote><br />
In the pom.xml specify the distributionManagment as:<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace;"><distributionManagement><br />
<repository><br />
<id>maven.jenkins-ci.org</id><br />
<url>http://maven.jenkins-ci.org:8081/content/repositories/releases/</url><br />
</repository><br />
</distributionManagement></blockquote><br />
<b>hudson-publish profile:</b><br />
In the settings.xml you need to specify the Hudson maven repositories and your gpg passphrase.<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace;"><repositories><br />
<repository><br />
<id>sonatype-nexus-staging</id><br />
<name>Nexus Release Repository</name><br />
<layout>default</layout><br />
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>sonatype-nexus-snapshots</id><br />
<name>Sonatype Nexus Snapshots</name><br />
<layout>default</layout><br />
<url>https://oss.sonatype.org/content/repositories/snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
<updatePolicy>always</updatePolicy><br />
</snapshots><br />
</repository><br />
</repositories><br />
<properties><br />
<gpg.passphrase>REPLACE_WITH_YOUR_OWNPASSPHRASE</gpg.passphrase><br />
</properties></blockquote><br />
In the pom file, you need to specify both distributionManagment and configure the gpg plugin<br />
<blockquote style="background-color: lightgrey; font-family: 'Courier New', Courier, monospace;"><distributionManagement><br />
<snapshotRepository><br />
<id>sonatype-nexus-snapshots</id><br />
<name>Sonatype Nexus Snapshots</name><br />
<url>https://oss.sonatype.org/content/repositories/snapshots</url><br />
</snapshotRepository><br />
<repository><br />
<id>sonatype-nexus-staging</id><br />
<name>Nexus Release Repository</name><br />
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url><br />
</repository><br />
</distributionManagement> <br />
<build><br />
<plugins> <br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-gpg-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>sign-artifacts</id><br />
<phase>verify</phase><br />
<goals><br />
<goal>sign</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</plugins><br />
</build></blockquote><br />
<b>Full pom examle</b><br />
You can look here for an example of a <a href="https://bitbucket.org/henriklynggaard/prereq-buildstep-plugin/src/db5a75a9a8cf/pom.xml">complete pom file</a>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com1tag:blogger.com,1999:blog-4367108636139699556.post-78422813917404103102011-04-03T17:06:00.001+02:002011-04-10T23:48:58.011+02:00Supporting both Hudson & Jenkins - Part 1: Getting accessWhen I write Hudson and Jenkins plug-ins , I want to support both of them and remain as neutral to the fork as possible. In order to do this I have chosen to host my plug-ins in a neutral place (bitbucket), but I still want my plug-in to show up in both update centres. This series of blog entries is my attempt at documenting how that it done, and this part is about which accounts you need.<br />
<br />
<b>My plugin:</b><br />
My plugin is hosted at <a href="https://bitbucket.org/henriklynggaard/prereq-buildstep-plugin">bitbucket.org</a>, I use it both for source code, wiki and issues. In effect I am not relying on Hudson or Jenkins infrastructure except for publishing to their update centres.<br />
<br />
<b>Jenkins:</b><br />
In order to publish to Jenkins you only need one account. This account is used to access the jenkins Wiki, Jira, Fisheye and publish to their Maven repository. You create it here: <a href="https://jenkins-ci.org/account">https://jenkins-ci.org/account</a><br />
<br />
<b>Hudson:</b><br />
Hudson is a bit more complicated as they use Maven Central repository for hosting their plug-ins, so you will need both access to Hudson and to push to maven push to Maven central.<br />
- First you need access to the <a href="http://wiki.hudson-ci.org/display/HUDSON/Home">Hudson wiki</a>.<br />
- Secondly you will need to have PGP key in order to sign your plug-in. I used Gnomes built in feature but the hudson pages have a guide as well.<br />
- You need an account with Nexus OSS in order to use their staging facility and publish to Maven central repository. Create the account according to <a href="https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide">this</a> page, and make sure to create the Jira ticket. YOu need the later part in order to get the hudson-deployer role.<br />
<br />
References:<br />
<a href="https://jenkins-ci.org/account">Jenkins Account page</a><br />
<a href="https://wiki.jenkins-ci.org/display/JENKINS/Hosting+Plugins#HostingPlugins-Releasingtojenkinsci.org">Jenkins Releasing</a><br />
<a href="http://wiki.hudson-ci.org/display/HUDSON/Home">Hudson Wiki</a><br />
<a href="http://wiki.hudson-ci.org/display/HUDSON/Releasing+Hudson+Plugin">Releasing Hudson Plugin</a><br />
<a href="https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide">Sonatype OSS Maven Repository Usage Guide</a><br />
<a href="https://docs.sonatype.org/display/Repository/How+To+Generate+PGP+Signatures+With+Maven">How To Generate PGP Signatures With Maven</a>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-7193364398225871982010-12-11T19:51:00.001+01:002011-02-09T23:07:14.638+01:00Compact Columns Hudson pluginI just discovered the "<a href="http://http://wiki.hudson-ci.org//display/HUDSON/Compact+Columns">Compact Columns plugin</a>" for <a href="http://hudson-ci.org/">Hudson</a>. It is much nicer to read than the standard layout, so this is my new default.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-20948533646383042722010-11-01T16:01:00.001+01:002011-02-09T23:07:14.640+01:00Annoying problem with JAX-WS webserviceIn order to monitor the connectivity of a webservice I have coded some junit tests which uses a JAX-WS soap client. These tests are fired of using Maven<br /><br />Most of the time they work great but occasionally I get the following exception<br /><br /><code>java.lang.AssertionError: the tube must call the add(...) method to register itself before start copying other pipes,but weblogic.wsee.jaxws.tubeline.FlowControlTube$FlowControlAwareTube@834ed06 hasn't done so<br />at com.sun.xml.ws.api.pipe.TubeCloner.copy(TubeCloner.java:104)<br />at weblogic.wsee.jaxws.tubeline.FlowControlTube.&lt;init>(FlowControlTube.java:31)<br />at weblogic.wsee.jaxws.tubeline.FlowControlTube.copy(FlowControlTube.java:50)<br />at weblogic.wsee.jaxws.tubeline.FlowControlTube.copy(FlowControlTube.java:13)<br />at com.sun.xml.ws.api.pipe.TubeCloner.copy(TubeCloner.java:102)<br />at com.sun.xml.ws.api.pipe.TubeCloner.clone(TubeCloner.java:74)<br />at com.sun.xml.ws.util.Pool$TubePool.create(Pool.java:170)<br />at com.sun.xml.ws.util.Pool$TubePool.create(Pool.java:161)<br />at com.sun.xml.ws.util.Pool.take(Pool.java:78)<br />at com.sun.xml.ws.client.Stub.process(Stub.java:245)<br />at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:135)<br />at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:109)<br />at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)<br />at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:118)<br /></code><br /><br />I don't know why we get that sometimes, but I have managed to hide the problem. The trick is to know that Maven surefire plugin enables assertions per default. In order to disable this particular assertion I configured the following in Maven POM<br /><br /><pre><plugin><br /> <groupId > org.apache.maven.plugins</groupId><br /> <artifactId>maven-surefire-plugin</artifactId><br /> <configuration><br /> <argLine>-da:com.sun.xml.ws.api.pipe.TubeCloner</argLine><br /> </configuration><br /></plugin><br /></pre>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-26589567731483755402010-09-13T22:22:00.001+02:002011-02-09T23:07:14.644+01:00JavaZone 2010 summaryLast week I attended JavaZone 2010 in Oslo, and this is a brief summary of my experiences. I hope to write in more details about the individual sessions later.<br /><br />At the conference there was almost all the time 7 parallel tracks, so while I saw 15 sessions I still missed a lot. This is almost always a problem with such conferences but the JavaZone people has prepared for it, all talks was filmed and the videos will hopefully be available online soon. <br /><br />The logistics around the conference was very good, everything went very smooth even the food service :-) .While the conference did provide both food and soda for both lunch and diner, it would have been nice if there had been a place where one could buy more<br /><br />The flow between the sessions was well planed, although as it is with all such conferences it is difficult to really visit the exhibition floor without missing a session. The sessions was generally of high quality and I certainly got something out of all of them.<br /><br />The exhibition floor was in my opinion the weak spot of the conference at least for me. Yes the major companies like IBM, Oracle etc. was represented but there was also a big group of Norwegian consultant companies looking both to promote and hire people. For the majority of the attendants they are very relevant but for a foreigner like me it sometimes felt a little to "local".<br /><br />The Good:<br />* Planning and logistics<br />* High quality sessions and speakers<br />* The overflow room where one could sit and switch between the talks.<br /><br />The bad:<br />* Sometimes felt a little "local" on the exhibition floor<br />* One or two talk held in Norwegian. While I as a danish person could understand it,I think all sessions should be held in English.<br /><br /><br />All in all I would go again next year if I get the opportunity, and I am looking forward to the videos coming online so I can see all the sessions I missed.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-16536564671815372122010-07-25T22:58:00.001+02:002011-02-09T23:07:14.647+01:00jax-ws maven plugin and wsdlLocation problemI am using wsimport of jax-ws maven plugin in order to build my webservices, but I have run into a minor problem with the plugin.<br /><br />The plugin has two ways of configuring how to find the wsdl files, either using the <wsdlfiles> tag to point out the files or <wsdlurls> to point out the target namespaces using a catalog file to map to wsdl files. The approach chosen has an influence on what is generated as the "wsdlLocation" in the service class' webservice annotation.</wsdlurls></wsdlfiles><br /><br />If I use the <wsdlfile> and <wsdldirectory> tags the calculated full path ends up in the annotation. This doesn't work for me because at runtime this full path is used to find the wsdl and it doesn't exist at that location on that machine. The jax-ws-catalog file is no help here as it normally contains the target namespace as key. </wsdldirectory></wsdlfile><br /><br />I could put a full path in the catalog file, but I would need to keep maintaining that depending on where the build server is building the project which is not very good.<br /><br />If I on the other hand use the <wsdlurls> the target namespace goes into the annotation. This works at runtime because the jax-ws catalog matches the namespace to a relative path inside the jar file.</wsdlurls><br /><br />The problem with using <wsdlurl> is that the maven plugin is not smart enough to know if it should regenerate the classes because of the extra layer of redirection. The result is the classes are allways regenerated. </wsdlurl><br /><br />I have created a issue on the topic: <a href="https://jax-ws-commons.dev.java.net/issues/show_bug.cgi?id=58">classes always regenerated when using wsdlUrls (#58)</a><br /><br />I have not tested just placing the wsdl files in the default directory and letting the plugin auto discover them. I have avoided this because the default directory does not really follow maven standards.Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0tag:blogger.com,1999:blog-4367108636139699556.post-76864323077113566072010-07-09T16:37:00.002+02:002011-02-09T23:07:14.649+01:00Maven compiler accepts "wrong layout" of java code.When you code Java it normal to place the java files in a directory structure matching the package names. Although technically not required I think most IDEs enforce this.<br /><br />However Maven compiler plugin does not enforce this. This can lead to situations where the build server sees everything working but when the developer imports this in a Eclipse or Netbeans the build will fail.<br /><br />Not sure if this is a bug or a feature..<div style='clear: both; text-align: center; font-size: xx-small;'>Published with Blogger-droid v1.4.4</div>Henrik Lynggaardhttp://www.blogger.com/profile/01727790099680191634noreply@blogger.com0