Category Archives: How To

Coding SVN hook in Java

In the How IT of today we create a SVN pre-commit’ hook coded in Java language 🙂

captain.hook

Why we need this

From small teams to really big teams counting more than 100 heads, the source control is almost mandatory. Apache Subversion (SVN) is one of most popular solution for software versioning and release control. Despite that as standard installation the application has no way to validate anything committed by users, SVN is very flexible to accept externals applications which are referred as hooks. Each hook will be executed in a specific step of the process; start-commit, pre-commit, pre-lock, pre-unlock, post-commit, etc.

SVN hooks can be used for almost everything, but commonly they do notification, validation, or replication over each operation. So, this could be the source of power over all bad artefacts submitted by users and prevent the lack of documentation in your application source code.

What we need

Development

  • JDK 1.7+
  • Log4J 1.2.16
  • SVNKit 1.8.12
  • Maven 3.0.5+

Execution – server

  • VisualSVN Server 3.7.1 (including SVN 1.9.7)
  • Microsoft Windows (just because of shell scripts)

Execution – client

  • TortoiseSVN 1.9.7+ (or your favourite SVN clients flavour)

How to

Let’s create a hypothetical scenario where we need to validate the name of files submitted by the team. Also, we need to be sure that no unacceptable word is going to be used in our artefacts contents. In this case, we want to stop the commit before the bad file turns part of the repository.

To achieve this goal we must create two hooks (could be only one, but for educational purposes will be two), which will be executed in pre-commit step. therefore, in case of break of rules, the commit will not be concluded.

A pattern to make it easy

First things first, we need to prepare the following dependencies in pom.xml:

<!-- Log4J -->
<dependency>
  <groupId>log4j</groupId>
  <artifactId>log4j</artifactId>
  <version>1.2.16</version>
</dependency>
<!-- SVN -->
<dependency>
  <groupId>org.tmatesoft.svnkit</groupId>
  <artifactId>svnkit</artifactId>
  <version>1.8.12</version>
</dependency>

The first validation will check if the file name contains only letters and number. It’s not a big challenge, right? Indeed! To do this task we coded the file TickingClockHook.java:

@Override
public void validate(String[] args) throws PreCommitException
{
	//Isolate the file name
	String fileName = args[4];
	log.debug("File name: " + fileName);
	if (fileName.contains("/"))
	{
		fileName = fileName.substring((fileName.lastIndexOf("/") + 1), fileName.length());
	}
	fileName = fileName.substring(0, fileName.indexOf("."));
	
	//Pattern to accept only letters and numbers
	Pattern pattern = Pattern.compile("^[a-zA-Z0-9]+$");
	
	if (!pattern.matcher(fileName).matches())
	{
		throw new PreCommitException("The file name must use only letters and numbers; " + fileName + " is not valid.");
	}
}

That was easy! As you can see, nothing in this method is exotic or hard to understand. Please, take a look at:

  • Line 51: we are reading the file name in method’s argument
  • Lines 53-57: isolate the file name from path and extension
  • Line 60: the regular expression to accept only letters and numbers is created
  • Line 62: verify if the file name matches to the pattern
  • Line 64: when the file name contains any non-alphabetic characters or numerics, the validation fails

Ok, but… is it works? Sure! See this output when trying to commit the file test-123.java:

Error: Commit failed (details follow):
Error: Commit blocked by pre-commit hook (exit code 1) with output:
Error: ==============================================================
Error:
Error: Your commit is blocked since the file name is not accepted.
Error: File: trunk/test-123.java
Error: Reason: The file name must use only letters and numbers; test-123 is not
Error: valid.
Error: You should fix it to try again.
Error:
Error: ==============================================================
Error: If you want to break the lock, use the 'Check For Modifications' dialog or the repository browser.

What about the validation of contents? Let’s see what is coded in BadLanguageHook.java:

@Override
public void validate(String[] args) throws PreCommitException
{
	String contents = loadFileContents(args[1], getUsername(), getPassword(), args[2], args[4]).toUpperCase();
	
	for (String word : bannedWords)
	{
		log.debug("Looking for: " + word);
		if (contents.contains((word.toUpperCase())))
		{
			throw new PreCommitException("Is not acceptable the use of " + word + " in contents of files.");
		}
	}
}

Less lines and nothing difficult:

  • Line 54: the file contents are read
  • Line 56: check for each banned word on the list
  • Line 59: look in contents for the forbidden word
  • Line 61: if the word is there, the validation fails

When we try to commit a dirty file:

Error: Commit failed (details follow):
Error: Commit blocked by pre-commit hook (exit code 1) with output:
Error: ==============================================================
Error:
Error: Your commit is blocked since something in file contents is not
Error: allowed.
Error: File: trunk/JollyRoger.java
Error: Reason: Is not acceptable the use of peter pan in contents of files.
Error: You should fix it to try again.
Error:
Error: ==============================================================
Error: If you want to break the lock, use the 'Check For Modifications' dialog or the repository browser.

After to behold the magic, we can check inside of the top hat.

I created a set of three classes as a pattern to make easy the creation of any hook of pre-commit to SVN.

The complete source code, compiling and shinning is available in my GitHub.

Let’s see the first and “core” of this pattern, the PreCommitInterdiction.java class:

public final static void main(String args[])
{
  //Checking for valid entries
  if (!hasValidArguments(args))
  {
    defineAsInternalError("This execution is invalid. Please, check for pre-commit script and useful messages in your log file " + LOG4J_LOG_FILE);
  }
  else
  {
    //Instantiate the hook
    PreCommitHook hook = createHook(args[0]);
    
    //Executes the validation using the hook
    try
    {
      if (hook != null)
      {
        hook.validate(args);
        printSuccess();
      }
    }
    catch (PreCommitException ex)
    {
      log.error("Could not validate because a pre-commit exception happened: " + ex.getMessage(), ex);
      printFail(ex.getMessage());
    }
    catch (Exception ex)
    {
      log.error("Could not validate because a general exception happened: " + ex.getMessage(), ex);
      defineAsInternalError(ex.getMessage());
    }
  }
  log.debug("...end!");
}

The method main is responsible to validate its arguments and call for the hook:

  • Line 146: check if calling arguments are valid
  • Line 153: instantiate the implementation of pre-commit hook
  • Line 160: execute the validation implemented by the hook
  • Line 167: when the validation fails, a message is printed explaining it
  • Line 172: if an unexpected error occurs, a message explaining it is printed

As you may notice, the highlighted line 160 is the point of interest in this map. The program calls the method validate of hook’s class, it comes from the interface PreCommitHook.java:

public interface PreCommitHook
{
  /**
   * Performs the validation of commit.
   * 
   * @param args Array of parameters to execute the hook.
   * @throws PreCommitException Happens when any or all implemented rules are broken.
   */
  public void validate(String[] args) throws PreCommitException;
}

This interface has to be implemented by all pre-commit hook classes and has only one method to override, it is validate.

The third class is our specific implementation of Exception, the PreCommitException.java:

public class PreCommitException extends Exception
{
	/**
	 * New friendly pre-commit exception.
	 * 
	 * @param message Explanation of the exception
	 */
  public PreCommitException(String message)
  {
    super(message);
  }
}

There is nothing special in this exception besides it is being used to thrown specific issues in the validation process.

Back to hook which is responsible to validate the file contents, in line 54 of file BadLanguageHook.java, what is the method loadFileContents? It comes from its superclass. All pre-commit hook have to extend PreCommitInterdiction, so this method is available. Take a look:

protected String loadFileContents(String repositoryPath, String username, String password, String transaction, String filePath) throws PreCommitException
{
  File repo = null;
  SVNLookClient svnLook = null;
  String contents = null;
  try
  {
    repo = new File(repositoryPath);
    ISVNAuthenticationManager authenticationManager = BasicAuthenticationManager.newInstance(username, password.toCharArray());
    ISVNOptions svnOptions = new DefaultSVNOptions();
    svnLook = new SVNLookClient(authenticationManager, svnOptions);
  }
  catch (Exception ex)
  {
  log.error("Fail while connecting to SVN using \"" + username +  ": " + ex.getMessage(), ex);
  throw new PreCommitException("Unable to connect to SVN: " + ex.getMessage());
  }
  try
  {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    svnLook.doCat(repo, filePath, transaction, baos);
    contents = baos.toString();
  }
  catch (SVNException ex)
  {
    log.error("Fail while loading contents of \"" + filePath + "\": " + ex.getMessage(), ex);
    throw new PreCommitException("Unable to load contents of submitted file: " + ex.getMessage());
  }
  if (contents == null)
  {
    throw new PreCommitException("Contents loaded from file is invalid.");
  }
  
  return contents;
}
  • Line 286-289: connect to SVN repository
  • Line 298-300: read the contents of submitted file

If the instance is unable to connect to SVN or the file contents is unreachable or invalid, a PreCommitException is thrown.

At this point, we have walked for all Java implementation. The pattern is flexible and functional in any Operating System, I guess (I have not tested others than Windows). The source code is available on my GitHub as well entire solution including configurations files and scripts.

However, the Java application (hook) will not work without a little help. The SVN server doesn’t execute applications itself but shell scripts. Therefore, when using Windows as host of the SVN server (not as client) you need to write a couple of .bat or .cmd files to call your hooks.

To accomplish our current scenario are needed three scripts:

  • pre-commit.bat: responsible to call other scripts, in fact, everything could be written here
  • TickingClockHook.bat: call the hook responsible to validate filenames
  • BadLanguageHook.bat: call the hook responsible to validate each file’s contents

These scripts will work only in a Windows host. If you are running your SVN Server on Linux, you need to write bash scripts to do what they do above -it is not a big deal.

Finally, there are two files used to configure the execution:

  • log4j.xml: configuration of log messages
  • svn.properties: authentication parameters used by the hook to connect to SVN

Here we are! I hope this pattern help you to make things a little more organized in your projects.

Download the complete and functional project from my GitHub.

Advertisement

Creating rules dynamically with Drools

This “how to” walks through a solution to dynamically create and execute business rules.

rules

Why we need this

It’s common the necessity to implement some kind of rules to choose between one and other action. Often, these rules are expressed in a sort of business thinking. Drools is a Business Rules Management System maintained by Red Hat and JBoss, and the answer when we need a robust, flexible and open source solution!

What we need

  • JDK 1.7+
  • Drools 6.5+
  • Maven 3.0.5+

How to

First of all, is needed to understand what rules mean in this context. Each rule in Drools works like a if statement in a programming language, thus basically we have two parts; conditional and action. This is written by someone who understood the business rule and adapted it in the schema of Drools Language Rule:

rule "Apply 10% off over all items of $4 or more"
when
    Product(price >= 4.0)
then
    product.discount(10);
end

Pretty easy, right?!

Line 1 says the friendly name of the business rule.
Line 3 is the condition to satisfy. In this case, the price of product has to be equal or more than 4 bucks .
Line 5 shows the action to be executed, so apply 10% discount over product’s price.

We can write how many rules are needed and all of them are saved in a file with extension .drl. But, as a file it is static and the goal here is to create something dynamic, therefore writing files is not an option!

Hands-on mode

Let’s think about the supermarket where we work. It’s time for promotions and the manager asked for applying a set of discounts on some products.

  • Apply 10% off over all items of $4 or more
  • For all items with due date on next 7 days, apply 45%
  • Give 5% off over every kind of Beans

There we go…

Before starting code, just make sure to have the following dependencies in your project file pom.xml:

<dependency>
  <groupId>org.drools</groupId>
  <artifactId>drools-compiler</artifactId>
  <version>6.5.0.Final</version>
</dependency>
<dependency>
  <groupId>org.drools</groupId>
  <artifactId>drools-core</artifactId>
  <version>6.5.0.Final</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
  <artifactId>drools-decisiontables</artifactId>
  <version>6.5.0.Final</version>
</dependency>

Next approach is to use Rule Templates, where we can write the pattern of our rules, and populate it after with concrete rules. Following is what we need to accomplish our mission, the file Product.drl:

template header

name
object
conditional
action

package drools.templates;

global net.itfromhell.howit.dummy.Product product;

import java.text.SimpleDateFormat;
import function net.itfromhell.howit.dynamicdrools.util.DroolsUtility.debug;

dialect "java"

template "Product"

rule "@{row.rowNumber} - @{name}"
when
  @{object}(@{conditional})
then
  product.discount(@{action});
  debug(drools);
end

end template

The structure used to create a template file is similar to regular rule file. Please, take a look on:

  • Line 1: describes this file as a template
  • Line 3-6: parameters expected in this template
  • Line 15: declares the dialect used as java
  • Line 19-25: rule defination
  • Line 27: end of template

Now we have a template to handle all rules about the products in promotion, the next step is to write a couple of rules describing how to apply the discount on them. We will do it programmatically, using a set of classes that I coded to make the dirty job:

  • Rule.class: encapsulate all that is needed to create one single business rule.
  • Condition.class: describe one specific condition to be evaluated.
  • DroolsUtility.class: tool to operate everything in memory.

The complete source code, compiling and ready for duty is available in my GitHub.

In my opinion, you wanna see everything working, who knows if it isn’t a joke… in this case, run ShowMeTheDrools.class:

public class ShowMeTheDrools
{
 public static void main(String args[]) throws Exception
 {
  //List to keep all rules
  List<Rule> rules = new ArrayList<Rule>();
  //Load each business rule
  rules.add(createDiscountOverpriced());
  rules.add(createDiscountSoonDueDate());
  rules.add(createDiscountBeans());

  //Create a session to operate Drools in memory
  DroolsUtility utility = new DroolsUtility();
  StatelessKieSession session = utility.loadSession(rules, "drools/templates/Product.drl");

  //Define the products to be processed using our rules
  Product blackBeans = new Product("Black Beans", 2.20, "30/12/2017");
  Product cannelliniBeans = new Product("Cannellini Beans", 4.15, "05/02/2018");
  Product kidneyBeans = new Product("Kidney Beans", 2.05, "20/11/2017");
  Product rice = new Product("Rice", 1.10, "28/10/2017");
  Product milk = new Product("Milk", 3.50, "10/11/2017");

  /*
  Now, the magic happens!
  For each product to be processed, we have to face it over rules to get, or not, a discounted price.
  */
  System.out.println("Applying over " + rice.getName() + " with price $" + rice.getPrice() + "...");
  session.setGlobal("product", rice);
  session.execute(rice);
  System.out.println("...price after review: $" + rice.getPrice());

  System.out.println("Applying over " + blackBeans.getName() + " with price $" + blackBeans.getPrice() + "...");
  session.setGlobal("product", blackBeans);
  session.execute(blackBeans);
  System.out.println("...price after review: $" + blackBeans.getPrice());

  System.out.println("Applying over " + milk.getName() + " with price $" + milk.getPrice() + "...");
  session.setGlobal("product", milk);
  session.execute(milk);
  System.out.println("...price after review: $" + milk.getPrice());

  System.out.println("Applying over " + kidneyBeans.getName() + " with price $" + kidneyBeans.getPrice() + "...");
  session.setGlobal("product", kidneyBeans);
  session.execute(kidneyBeans);
  System.out.println("...price after review: $" + kidneyBeans.getPrice());

  System.out.println("Applying over " + cannelliniBeans.getName() + " with price $" + cannelliniBeans.getPrice() + "...");
  session.setGlobal("product", cannelliniBeans);
  session.execute(cannelliniBeans);
  System.out.println("...price after review: $" + cannelliniBeans.getPrice());
 }

As result we got:

Applying over Rice with price $1.1...
Triggered rule: 1 - Apply discount on all soon due date
...price after review: $0.605

Applying over Black Beans with price $2.2…
Triggered rule: 2 – Discounting on all beans
…price after review: $2.0900000000000003

Applying over Milk with price $3.5…
…price after review: $3.5

Applying over Kidney Beans with price $2.05…
Triggered rule: 2 – Discounting on all beans
…price after review: $1.9474999999999998

Applying over Cannellini Beans with price $4.15…
Triggered rule: 2 – Discounting on all beans
Triggered rule: 0 – Give some discount on overpriced
…price after review: $3.5482500000000003

In order to verifiy the promotional price, we noted a discount on Rice which started for $1.1 and now is $0.605 (what bargain! don’t forget to check the due date). We know why, it’ s because the rule named “Apply discount on all soon due date” was triggered applying its huge discount.

Black Beans started from $2.2 and finished to $2.09, 5% less because of rule “Discounting on all beans”.

But wait, we got no discount on Milk since the start and final prices are the same 3.5 buks! The simple answer is just because no rule was triggered (you can check it).

Something different happened to Cannellini Beans, there are two rules triggered, is it right? Yes it is. The first rule about Beans is triggered applying 5%, next the rule about overprice is triggered applying more 10%, so the final price is now $3.54. It is intersting since we are able to chain rules, increasing the complexity of our solution.

Back to source code of ShowMeTheDrools on lines:

  • 25-27: all three business rules are created (next section we see details)
  • 31: our tool is used to get a session to operate rules
  • 34-38: a set of products is defined
  • 44-67: we submit each product against all business rules and watch what happens on price

To make it easy to understand, I created a specific method for each business rule. In the real world it doesn’t make any sense. Anyway, let’s see how it works for asked business rule “Apply 10% off over all items of $4 or more”:

private static Rule createDiscountOverpriced()
{
 //First of all, we create a rule giving it a friendly name
 Rule rule = new Rule("Give some discount on overpriced");
 //Here we need to say what kind of object will be processed
 rule.setDataObject(Product.class.getName());

 //As expected, a rule needs condition to exists. So, let's create it...
 Condition condition = new Condition();
 //What data, or property, will be checked
 condition.setProperty("price");
 //What kind of check to do
 condition.setOperator(Condition.Operator.GREATER_THAN_OR_EQUAL_TO);
 //What is the value to check
 condition.setValue(new Double(4.0));

 //Next, is needed to set rule's condition
 rule.setCondition(condition);
 //Finally, this is what will be done when ours condition is satisfied
 rule.setAction("10");

 return rule;
}

What about “For all items with due date on next 7 days, apply 45%”:

private static Rule createDiscountSoonDueDate() throws Exception
{
 Rule rule = new Rule("Apply discount on all soon due date");
 rule.setDataObject(Product.class.getName());

 //Is possible to create multiple conditions, therefore, data range or more complex situations could be expressed
 Condition greaterThan = new Condition();
 greaterThan.setProperty("dueDate");
 greaterThan.setOperator(Condition.Operator.GREATER_THAN);
 greaterThan.setValue((new SimpleDateFormat("dd/MM/yyyy").parse("23/10/2017")));

 Condition lessThan = new Condition();
 lessThan.setProperty("dueDate");
 lessThan.setOperator(Condition.Operator.LESS_THAN);
 lessThan.setValue((new SimpleDateFormat("dd/MM/yyyy").parse("30/10/2017")));

 //You can define as many as necessary conditions to achieve your necessity
 rule.setConditions(Arrays.asList(greaterThan, lessThan));
 rule.setAction("45");

 return rule;
}

Finally we have the third business rule “Give 5% off over all Beans”:

private static Rule createDiscountBeans()
{
 Rule rule = new Rule("Discounting on all beans");
 rule.setDataObject(Product.class.getName());

 //This is the simplest way to define the rule' condition
 rule.addCondition("name", Condition.Operator.CONTAINS, "Beans");  

 rule.setAction("5");

 return rule;
}

Let’s take a look over most important pieces of code of each class, starting from beggining; Rule.class

As we discussed before, the engine understand Drools Rule Language, the method conditionAsDRL transforms all conditions of the rule into textual expressions.

 public String conditionAsDRL() throws IllegalStateException, IllegalArgumentException
 {
  if ((conditions == null) || (conditions.isEmpty()))
  {
   throw new IllegalStateException("You must declare at least one condition to be evaluated.");
  }

  StringBuilder drl = new StringBuilder();
  //For each condition of this rule, we create its textual representation
  for (int i = 0; i < conditions.size(); i++)
  {
   Condition condition = conditions.get(i);
   drl.append("(");
   drl.append(condition.buildExpression());
   drl.append(")");
   if ((i + 1) < conditions.size())
   {
    drl.append(" && ");
   }
  }

  return drl.toString();
 }

The class Condition.class has buildExpression, executed by Rule and responsible to return its own expression of the conditional.

public String buildExpression() throws IllegalArgumentException
{
 StringBuilder drl = new StringBuilder();

 if (value instanceof String)
 {
  drl.append(expressionForStringValue());
 }
 else if (value instanceof Number)
 {
  drl.append(expressionForNumberValue());
 }
 else if (value instanceof Date)
 {
  drl.append(expressionForDateValue());
 }
 else
 {
  throw new IllegalArgumentException("The class " + value.getClass().getSimpleName() + " of value is not acceptable.");
 }

 return drl.toString();
}

For each type of value, there is a specific method responsible to create the specialized expression, as exemple of expressionForNumberValue, used to transform a Number instance:

private String expressionForNumberValue() throws IllegalArgumentException
{
 StringBuilder drl = new StringBuilder();

 if ((operator.isComparable(Short.class)) || (operator.isComparable(Integer.class)) || (operator.isComparable(Long.class))
     || (operator.isComparable(Double.class)) || (operator.isComparable(Float.class)))
{
  drl.append(property).append(" ").append(operator.getOperation()).append(" ").append(value);
 }
 else
 {
  throw new IllegalArgumentException("Is not possible to use the operator " + operator.getDescription() + " to a " + value.getClass().getSimpleName() + " object.");
 }

 return drl.toString();
}

Last but not less important, the class DroolsUtility.class has two methods to see:

/**
 * Loads a session to execute rules in memory using a template file.
 *
 * @param templatePath Relative path to template file describing the rule's pattern.
 * @param rulesAsParameters List of maps representing each rule as a set of parameters.
 * @return Session for execution of rules.
 * @throws Exception
 */
private StatelessKieSession loadSession(String templatePath, List<Map<String, Object>> rulesAsParameters) throws Exception
{
  ObjectDataCompiler compiler = new ObjectDataCompiler();
  //Compiles the list of rules using the template to create a readable Drools Rules Language
  String drl = compiler.compile(rulesAsParameters, Thread.currentThread().getContextClassLoader().getResourceAsStream(templatePath));

  System.out.println("drl:\n" + drl);

  KieServices services = KieServices.Factory.get();
  KieFileSystem system = services.newKieFileSystem();
  system.write("src/main/resources/drools/templates/rule.drl", drl);
  services.newKieBuilder(system).buildAll();

  KieContainer container = services.newKieContainer(services.getRepository().getDefaultReleaseId());
  StatelessKieSession session = container.getKieBase().newStatelessKieSession();

  return session;
}

The highlight line 56 is a trick, it will print in console the concrete generated .drl‘s contents, which is used by the engine in memory. See below:

package drools.templates;
global net.itfromhell.howit.dummy.Product product;
import java.text.SimpleDateFormat;
import function net.itfromhell.howit.dynamicdrools.util.DroolsUtility.debug;
dialect "java"

rule "2 - Discounting on all beans"
when
  net.itfromhell.howit.dummy.Product((name.toUpperCase().contains("BEANS")))
then
  product.discount(5);
  debug(drools);
end

rule "1 - Apply discount on all soon due date"
when
  net.itfromhell.howit.dummy.Product((dueDate > (new SimpleDateFormat("dd/MM/yyyy HH:mm:ss")).parse("23/10/2017 00:00:00")) && (dueDate < (new SimpleDateFormat("dd/MM/yyyy HH:mm:ss")).parse("30/10/2017 00:00:00")))
then
  product.discount(45);
  debug(drools);
end

rule "0 - Give some discount on overpriced"
when
  net.itfromhell.howit.dummy.Product((price >= 4.0))
then
  product.discount(10);
  debug(drools);
end

As suggests its name, this method can be used to get some details of the triggered rule when the engine acts.

/**
 * Debug tool to show what is happening over each triggered execution.
* Name of rule trigger as well the object inspected are printed.
 *
 * @param helper Injected when a consequence is fired.
 */
public static void debug(final KnowledgeHelper helper)
{
 System.out.println("Triggered rule: " + helper.getRule().getName());
 if (helper.getMatch() != null && helper.getMatch().getObjects() != null)
 {
  for (Object object : helper.getMatch().getObjects())
  {
   System.out.println("Data object: " + object);
  }
 }
}

That’s all folks!

Remember it: living on your own rules 🙂

Download the complete project from my GitHub.