When do you think a daily stand up is too crowded?

Recently, looking at the daily stand-ups in my company, it makes me smile a bit.

I think too many = chaos and dispersion. 
Proper and efficient stand-ups should be 4 pigs and occasionally 2 or 3 chickens. 

This is my opinion, what is yours?

Building smart Builders

When building an API, you should always think about who is going to use it.

When the API is simply and clear to use, then the users are happy. When the users are happy then everyone is happy too :).

But great usability is not always easy to achieve. 

There are patterns that help on this, on this post I will focus on the classic builder pattern and how you can enhance it with the step builder pattern in order to build objects with a no brain interface, easy to use, impossible to get wrong.

So lets start painting some context, we have 2 domain objects representing a user configuration to connect to some remote or local server. When remote credentials are required, when local no.

package com.marco.sbp;
public class UserConfiguration {
        private final String name;
        private ServerDetails serverDetails;

        public UserConfiguration(String name) {
                this.name = name;
        }

        public void setServerDetails(ServerDetails serverDetails) {
                this.serverDetails = serverDetails;
        }

        public String getName() {
                return name;
        }

        public ServerDetails getServerDetails() {
                return serverDetails;
        }
}

package com.marco.sbp;
public class ServerDetails {

        private final String host;
        private String user;
        private String password;

        public ServerDetails(String host) {
                this.host = host;
        }

        public void setUser(String user) {
                this.user = user;
        }

        public void setPassword(String password) {
                this.password = password;
        }

        public String getHost() {
                return host;
        }

        public String getUser() {
                return user;
        }

        public String getPassword() {
                return password;
        }
}

We want to abstract the construction of the objects above using 2 different techniques, the classic builder pattern and the step builder pattern.

The classic builder pattern is pretty straightforward, it works masking the creation of the UserConfiguration and the ServerDetails using properly named methods like onLocalHost, onRemoteHost, etc.

package com.marco.sbp.builder;
import com.marco.sbp.ServerDetails;
import com.marco.sbp.UserConfiguration;
public class ClassicBuilder {
        
        private String name;
        private String host;
        private String user;
        private String password;

        
        public ClassicBuilder(String name){
                this.name = name;
        }
        
        public ClassicBuilder onLocalHost(){
                this.host = "localhost";
                return this;
        }
        
        public ClassicBuilder onRemoteHost(String remoteHost){
                this.host = remoteHost;
                return this;
        }
        
        
        public ClassicBuilder credentials(String user, String password){
                this.user = user;
                this.password = password;
                return this;
        }
        
        public UserConfiguration build(){
                UserConfiguration userConfiguration = new UserConfiguration(name);
                ServerDetails serverDetails = new ServerDetails(host);
                serverDetails.setUser(user);
                serverDetails.setPassword(password);                    
                userConfiguration.setServerDetails(serverDetails);
                return userConfiguration;
        }
}

The step builder pattern is still using smart names to construct the object, but it's exposing these methods only when needed using interfaces and proper encapsulation.

package com.marco.sbp.builder;
import com.marco.sbp.ServerDetails;
import com.marco.sbp.UserConfiguration;

/** "Step Builder" */
public class StepBuilder {
        public static NameStep newBuilder() {
                return new Steps();
        }

        private StepBuilder() {
        }

        public static interface NameStep {
                /**
                 * @param name
                 *            unique identifier for this User Configuration
                 * @return ServerStep
                 */
                ServerStep name(String name);
        }       

        public static interface ServerStep {
                /**
                 * The hostname of the server where the User Configuration file is stored will be set to "localhost".
                 * 
                 * @return BuildStep
                 */
                public BuildStep onLocalhost();

                /**
                 * The hostname of the server where the User Configuration file is stored.
                 * 
                 * @return CredentialsStep
                 */
                public CredentialsStep onRemotehost(String host);
        }

        public static interface CredentialsStep {
                /**
                 * Username required to connect to remote machine Password required to connect to remote machine
                 * 
                 * @return BuildStep
                 */
                public BuildStep credentials(String user, String password);
        }

        public static interface BuildStep {
                /**
                 * @return an instance of a UserConfiguration based on the parameters passed during the creation.
                 */
                public UserConfiguration build();
        }

        private static class Steps implements NameStep, ServerStep, CredentialsStep, BuildStep {

                private String name;
                private String host;
                private String user;
                private String password;

                public BuildStep onLocalhost() {
                        this.host = "localhost";
                        return this;
                }
                
                public ServerStep name(String name) {
                        this.name = name;
                        return null;
                }

                public CredentialsStep onRemotehost(String host) {
                        this.host = host;
                        return this;
                }

                public BuildStep credentials(String user, String password) {
                        this.user = user;
                        this.password = password;
                        return this;
                }

                public UserConfiguration build() {
                        UserConfiguration userConfiguration = new UserConfiguration(name);
                        ServerDetails serverDetails = new ServerDetails(host);
                        serverDetails.setUser(user);
                        serverDetails.setPassword(password);                    
                        userConfiguration.setServerDetails(serverDetails);
                        return userConfiguration;
                }

                
        }
}

Lets see now what is the user experience with both of our builders.

The classic builder  will be constructed using the name of the user configuration, then it will expose all of its methods leaving the user a bit too free to choose what's next.

For example, a not careful user could end up with a UserConfiguration set with localhost where no authentication is required, still passing user and password.

This is confusing and it can lead to run-time exceptions.

These are some of the possible combinations of UserConfigurations that the user can end up with, some are correct, lots are wrong:

A complete different story is with the step builder, here only one step at the time is exposed:

If the credentials are not needed they will not be exposed and the build() method is offered only when the state of the object is sure to be coherent and complete:

Only 2 possible UserConfigurations can be built with this pattern, and both make sense and are clear to the user.

 Conclusion

The step builder pattern is not the replacement of the classic Bloch one, sometimes you want to force the user to fill some parameter before advancing with the creation, in this case the step builder is doing the job, otherwise when a more open approach is required than the classic builder is your guy.

Even in the jdk there is bad code.

Java 7, TreeSet and NullPointerException.

Recently I tried to compile with java 7 a project developed with java 6.

Lot of fun happened during tests execution, tests that in java 6 were  running smoothly, with java 7, they were strangely failing!
So, I had to understand why and this is what I discovered...

The context first:
In that project I have a simple Hibernate Entity more or less like the following. 

package com.marco.test;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Table;
import javax.persistence.UniqueConstraint;
import org.hibernate.validator.NotNull;
@Entity
@Table(...)
public class ABean {
        
        ...
        
        private String name;

        @Column(name = "name", nullable = false)
        @NotNull
        public String getName() {
                return name;
        }

        public void setName(String name) {
                this.name = name;
        }
}

note that the field "name" is nullable=false and marked with @NotNull.
This to tell Hibernate to fail the validation in the case a user tries to create or update this column to Null.

I also have a comparator for that Entity.
This comparator uses the name field to compare the Entity ( this is just a simplified version of what I have in the project, of course I don't order a bean based on the string length :) )

package com.marco.test;
import java.util.Comparator;
public class ABeanComparator implements Comparator<ABean> {

        @Override
        public int compare(ABean o1, ABean o2) {
                if (o1.getName().length() > o2.getName().length()) {
                        return 1;
                } else if (o1.getName().length() < o2.getName().length()) {
                        return -1;
                } else {
                        return 0;
                }
        }
}

note that there is no null check on the field name, in my project, Hibernate is already taking care of it.

Now, I have a test that create one empty Entity and it stores it into a TreeSet, and then doees other stuff that we do not really care here.
The beginning of the test is similar to the code below:  

package com.marco.test;
import java.util.SortedSet;
import java.util.TreeSet;
public class SortedTestTest {

        public static void main(String[] args) {

                ABean aBean = new ABean();

                SortedSet<ABean> sortedSet = new TreeSet<ABean>(new ABeanComparator());

                sortedSet.add(aBean);
        }
}

If I run this with java 6, everything is OK.

But, with java 7 I have a NullPointerException.  

Exception in thread "main" java.lang.NullPointerException
        at com.marco.test.ABeanComparator.compare(ABeanComparator.java:9)
        at com.marco.test.ABeanComparator.compare(ABeanComparator.java:1)
        at java.util.TreeMap.compare(TreeMap.java:1188)
        at java.util.TreeMap.put(TreeMap.java:531)
        at java.util.TreeSet.add(TreeSet.java:255)
        at com.marco.test.SortedTestTest.main(SortedTestTest.java:14)

Why?

This is why:

    public V put(K key, V value) {
        Entry<K,V> t = root;
        if (t == null) {
            compare(key, key); // type (and possibly null) check

            root = new Entry<>(key, value, null);
            size = 1;
            modCount++;
            return null;
        }

In java 7 when the first Object is added ( if (t == null) ) to a TreeSet, a compare against itself (compare(key,key)) is executed.  

The compare method will then call the comparator (if there is one) and we will have the NullPointerException on the name property. 

    // Little utilities

    /**
     * Compares two keys using the correct comparison method for this TreeMap.
     */
    final int compare(Object k1, Object k2) {
        return comparator==null ? ((Comparable<? super K>)k1).compareTo((K)k2)
            : comparator.compare((K)k1, (K)k2);
    }

This raises more questions than answers:

• Why running a compare if you know that the Object in the TreeSet is the first and only one ?
• My guess is that what they wanted to do was running a simple Null check.
• Why not create a proper null check method ?
• No Answer
• Why wasting CPU and memory running a comparison that is not needed ?
• No Answer
• Why compare an object against itself (compare(key,key))??
• No Answer

This is the put method of the TreeSet in java 6 and as you can see the compare was commented out.

public V put(K key, V value) {
                Entry<K, V> t = root;
                if (t == null) {
                        // TBD:
                        // 5045147: (coll) Adding null to an empty TreeSet should
                        // throw NullPointerException
                        //
                        // compare(key, key); // type check
                        root = new Entry<K, V>(key, value, null);
                        size = 1;
                        modCount++;
                        return null;
                }

You see the comment?  Adding null to an empty TreeSet should throw NullPointerException
So just check if key is null, don't run a useless comparison!


The conclusion? Always try to analyze the code you use, because even in the jdk there is bad code! 

How to update apt-get repositories in your old Linux distribution?

Are you using a Linux distribution that is really old?.
The apt-get repositories for the distribution are not supported anymore, but you would like to update the repositories so that you can install new packages?

Cool, so here it's how to do that:

You need to update /etc/apt/sources.list with new (valid) urls. e.g. in Ubuntu 11.04 (Maverick, no longer supported) you might want to update the URLs to (for example) "oneiric". You might have to do the following:

1) Backup /etc/apt/sources.list

2) Find and replace (inline) the old distro name ("maverick" in this example) with the new one ("oneiric" in this example)

3) Update apt-get cache to see the changes reflected.

sudo cp /etc/apt/sources.list /etc/apt/sources.list.backup
sudo sed -i 's/maverick/oneiric/g' /etc/apt/sources.list
sudo apt-get update

And that's it, make sure the replaced sources.list file has the same permissions as the original one and bear in mind that you might want to use "sudo" since /etc/apt is usually owned by root.

Easy testing SLAs on distributed components with grep4j

So your distributed architecture looks something like the picture below and you just have received a requirement from the business to make sure that the SLAs of the messages sent by the Producer and then traveling to the downstream systems (consumers) must be fast and never slower than 400 milliseconds.

Requirement says :
The Latency of a message sent from the Producer to any of the Consumers should be never slower than 400 milliseconds.

Sounds familiar? To me yes, and experience taught me that if I want to protect the SLAs in the future, I need as well to automate the test in order to not introduce bottlenecks that then increase the latency of the messages.

But how to do it? Producer and Consumers are in separate machines and some of the consumers are not written in Java.
Also, between the producer and the consumers there is a Queue (or web-service or RMI or an ESB or another component or whatever), so things are not getting easier for me to test.

Well, all components write logs in a similar way, so why not using logs as a data to test?

For example these are 2 sample logs, one from the producer firing a message (id 1546366) and the other from one of the consumer receiving the message (id 1546366):

Producer logs 

......

2013-02-19 10:09:05,795 INFO  [org.grep4j.demo.input.core.InputCoreMessageSender] (pool-19-thread-9) Input::MessageSender::Message(1546366) Sent Successfully

......

Consumer logs

.....

2013-02-19 10:09:06,161 INFO  [org.grep4j.demo.handler.bean.mdb.SingleDestPacketHandler] (Thread-62314 (HornetQ-client-global-threads-989457197)) Handler::Packet::Message(1546366) Received::PacketId(982336426)::State(NORMAL)::Dest(CONSUMER4, 1)::DataLevel(EVENT, 7)::Op(CREATE, C)::GlobalId(1546366)::Priority(1)::Src(GUI, 1::Ids(EventId=1546366,SFBId=1546366,isBirType=false,inBir=false))

.....

And this is how my automated performance test looks like using Grep4j :

package com.gdg.grep4j.demo;
import static com.gdg.grep4j.demo.profiles.Profiles.consumer1;
import static com.gdg.grep4j.demo.profiles.Profiles.consumer2;
import static com.gdg.grep4j.demo.profiles.Profiles.consumer3;
import static com.gdg.grep4j.demo.profiles.Profiles.producer;
import static com.gdg.grep4j.demo.services.TimeService.extractTime;
import static org.grep4j.core.Grep4j.constantExpression;
import static org.grep4j.core.Grep4j.grep;
import static org.grep4j.core.fluent.Dictionary.on;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.number.OrderingComparison.lessThan;
import static org.junit.Assert.assertThat;
import org.grep4j.core.result.GrepResults;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
@Test
public class MessageDistributionPerformanceTest {


        private static final long MAX_ACCETABLE_LATENCY = 400L;

        private long producerTime = 0;

        private GrepResults consumersResults;
        
        @BeforeTest
        public void triggerMessageDispatcher() {
                System.out.println("Producing and firing a Message(1546366) to downstream systems...");
        }
        
        @BeforeTest
        public void extractProducerTime() {
                
                GrepResults producerResult = grep(constantExpression("Message(1546366) Sent Successfully"),     on(producer));
                producerTime = extractTime(producerResult.toString());
        }
        
        @BeforeTest
        public void grepConsumerLogs() {
                                
                consumersResults = grep(constantExpression("Message(1546366) Received"),
                                on(consumer1, consumer2, consumer3));
        }

        public void testConsumer1Latency() {
 
                long consumer1Time = extractTime(consumersResults.filterOnProfile(consumer1).toString());
                assertThat((consumer1Time - producerTime),
                                is(lessThan(MAX_ACCETABLE_LATENCY)));
        }

        public void testConsumer2Latency() {

                long consumer2Time = extractTime(consumersResults.filterOnProfile(consumer2).toString());
                assertThat((consumer2Time - producerTime),
                                is(lessThan(MAX_ACCETABLE_LATENCY)));

        }

        public void testConsumer3Latency() {

                long consumer3Time = extractTime(consumersResults.filterOnProfile(consumer3).toString());
                assertThat((consumer3Time - producerTime),
                                is(lessThan(MAX_ACCETABLE_LATENCY)));

        }
}

A profile is the grep target context, in my case all the profiles are remote machines (for a better understanding of Profiles see Grep4j page).

TimeService is just a simple service class extracting the time in the logs.

package com.gdg.grep4j.demo.services;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.TimeZone;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class TimeService {

        private static final Pattern timePattern = Pattern
                        .compile("([0-9][0-9][0-9][0-9])-([0-9][0-9])-([0-9][0-9]) ([0-9][0-9]|2[0-3]):([0-9][0-9]):([0-9][0-9]),([0-9][0-9][0-9])");

        public static long extractTime(String text) {
                Matcher lm = timePattern.matcher(text);
                if (lm.find()) {
                        SimpleDateFormat sdf = new SimpleDateFormat(
                                        "yyyy-MM-dd HH:mm:ss,SSS");
                        sdf.setTimeZone(TimeZone.getTimeZone("UTC"));

                        String inputString = lm.group();

                        Date date = null;
                        try {
                                date = sdf.parse(inputString);
                        } catch (ParseException e) {
                                e.printStackTrace();
                        }
                        return date.getTime();

                } else {
                        throw new IllegalArgumentException("timePattern not found");
                }
        }
}

In few simple lines of code I have my extremely flexible test (I can test anything that was produced in the logs) .

For the full code : https://github.com/marcocast/grep4j-gdg.git