Sunday, February 7, 2016

Book Review: Building Microservices

The architectural idea of microservices was inspired - in part - by Unix's philosophy of code being short, simple, clear, modular, extendable and that it could be repurposed by developers. The term is currently up there with Internet of Things, Big Data and the Cloud, in contemporary technical lexicon.

Author Sam Newman is an industry expert on the subject. He has:
written for Infoq, presented at Javazone and various other events regarding microservices.

His book 'Building Microservices' does an excellent job introducing the key concepts of microservices:
  • services are autonomous, live on their machines
  • resilience - one service fails, it doesn't impact other services
  • scaling - because services are independent they can be independently scaled
  • deployment - deployment should be easy. A change to a service means that's all that's deployed.
It is worth pointing out it isn't just a book about microservices. Many of the ideas and best practises detailed (for example HATOES in your ReST approach and to check OWASP for security references) are equally applicable to non - microservices architectures. But, that said, what I really like about this book is anytime a general architectural or software engineer concept (and there are a lot) is explained it is explained very well.  Some examples:
There are - of course - many tips regarding how you approach microservices. For example:
  • Don't get too hung up on DRY. It may make it harder to keep your services independent.
  • Consider using CDC's for your testing approach
  • Canary releasing / Blue, Green testing for releasing.
  • Using bulkhead and circuit breaker patterns to make services ore resilient.
As an overview to microservices, it's a great book. However, the reader must bear in mind there is no one size fits all approach to microservices. The finer details will depend on your project, your team and even a bit of trial and error. It's difficult to critique this book. So instead, I'll just flag some concerns not about the actual book but about microservices in general and how the industry and developers are reacting to them.

My first concern with microservices is more a practical than a technical one. Doing modular design for everything from your schema, service layer, end points, configuration isn't as easy as some people might think. Especially in teams of varied skill level, varied backgrounds and inevitable commercial pressure that happens in every project. It requires a lot of technical aptitude, leadership and discipline. If a project is not good at achieving modular design in a monolith - for whatever reason - I think it will really struggle at modular design in microservices when the complexity of the network has to be also considered.

The second concern I'd have is that when explaining the principle ideas of microservices, it is often compared with a monolith to the point the word monolith is a pejorative term.  The two approaches: monolith and microservices are presented as if it's either one or the other.  I think this a false dichotomy fallacy. There are other approaches available.

Thirdly, microservices are no doubt,  a clever idea.  But that doesn't mean there are a panacea. In some projects they will be a good fit, in others they will not be worth it. One obvious factor to bear in mind is your non-functional requirements.  One useful point of reference,  that is worth considering is the project James Lewis (another Thoughtworks guru) described in his talk about microservices in 2012.  In this presentation, three non-functional requirements for the project caught my attention:
  • One component had to handle 1,000 TPS
  • Another had to support a user base of 100 million users
  • Another had to support batch loads of 30 - 90 million records
While I am not saying you should be in this ballpark before considering microservices, I am just trying to suggest that most projects don't have these sort of demands and there's a lot of merit of considering something like a very well structured modular monolith first using DDD principles and then to migrate towards microservices should the benefits justify it. This strategy is well explained by Martin Fowler in his Monolith first article.

Some other references:


Until the next time enjoy yourselves.



Fail Safe / Fail Fast

When developing a rapid prototype it can make sense to put the emphasis on the 'happy path' and not consider things like exception handling, edge cases and failure.  Perhaps, once the prototype phase of the project is over the code will be thrown out or it will be refactored to deal with the real world.

In a production system we simply don't have luxury to only consider the happy path.  The more production system I have worked on the more types of failure I have being exposed to - some of it very painful. As my hair went greyer from these experiences, I couldn't help thinking more and more about how to make failure less painful. Who wants their pager going off for some silly reason? Nobody. If dealing with production code, we simply must think about how best to deal with various forms of failure.

In this presentation I consider two engineering techniques that I think should always be on the architectural radar:
* Fail fast
* Fail safe



Sunday, January 24, 2016

Developer Productivity

Recently I gave a presentation regarding developer productivity - something that has always interested me.  Generally, when a system gets out of control with spiralling technical debt and software entropy it becomes impossible to make changes or add new functionality to that system in a reasonable and predictive way.   At a critical level, this can be the difference between project success and project failure.  If processes are smart, good structures are in place, developers are productive not burning out from late hours debugging crazy code, terrified of the impacts of any change.

Everyone wants a path to project delivery that is smoother.  No matter what the job or the task it is generally far easier to get a sense of satisfaction from a sense of productivity.   But let's face it; software entropy exists in nearly every project.  Why?  Are we not smart enough to use the machines we designed?  Do commercial realities inevitably mean bad code happens more than good code?

These are questions that spark lively discussions.  In this presentation, I outline some of my own ideas on developer productivity.




Saturday, September 12, 2015

Custom User types in GORM

Recently, I wanted to model a Merchant which like many things in a domain model had an Address. I thought it made sense that the Address was embedded inside the Merchant. Reasons:
  • It had no lifecycle outside the Merchant. Merchant dies so should the address.
  • It only ever belonged to one and only one Merchant
So pretty obvious this was a composition relationship.

Now, it is possible to model composition relationships in GORM. See here. However, this approach comes with the caveat that the Address must be a GORM object. I didn't want the Address being a GORM object because GORM objects are powerful in Grails. With all their dynamic finders and GORM APIs they are essentially like really a DAO on steroids. If a developer gets their hands on can do lots of things (not always good things). I didn't want or need any of this. In addition, a good architecture makes it difficult for developers to make mistakes when they are working under pressure at fast speeds. That means, when you are making design decisions you need to think about the power you need to give, should give and will give.

So with that in mind, I looked into wiring up a custom type for Address. This would just be a data structure that would model the address, could be reused outside the Merchant (thus promoting consistency and again thus promoting good design) and wouldn't come with the power of the GORM. There is some documentation in the GORM doc's for custom types but there isn't a full working example. I had a look at some Hibernate examples and then put managed to put this together and get working.

Here is my address object.

@Immutable
class Address {

    private final String city;
    private final String country;
    private final String state;
    private final String street1;
    private final String street2;
    private final String street3;
    private final String zip;

    public String getCity() {
        return city;
    }

    public String getCountry() {
        return country;
    }

    public String getZip() {
        return zip;
    }

    public String getState() {
        return state;
    }

    public String getStreet1() {
        return street1;
    }

    public String getStreet2() {
        return street2;
    }

    public String getStreet3() {
        return street3;
    }
}
Here is my AddressUserType object:
class AddressUserType implements UserType {

    public int[] sqlTypes() {
        return [
            StringType.INSTANCE.sqlType(),
            StringType.INSTANCE.sqlType(),
            StringType.INSTANCE.sqlType(),
            StringType.INSTANCE.sqlType(),
            StringType.INSTANCE.sqlType(),
            StringType.INSTANCE.sqlType(),
            StringType.INSTANCE.sqlType()
        ] as int[]
    }

    public Class getReturnedClass() {
        return Address.class;
    }

    public Object nullSafeGet(ResultSet rs, String[] names, SessionImplementor session, Object owner) throws SQLException {
        assert names.length == 7;
        log.debug(">>mullSafeGet(name=${names}")
        String city = StringType.INSTANCE.get(rs, names[0], session); // already handles null check
        String country = StringType.INSTANCE.get(rs, names[1], session ); // already handles null check
        String state = StringType.INSTANCE.get(rs, names[2], session ); // already handles null check
        String street1 = StringType.INSTANCE.get(rs, names[3], session ); // already handles null check
        String street2 = StringType.INSTANCE.get(rs, names[4], session ); // already handles null check
        String street3 = StringType.INSTANCE.get(rs, names[5], session ); // already handles null check
        String zip = StringType.INSTANCE.get(rs, names[6], session ); // already handles null check

        return city == null && v == null ? null : new GAddress(city: city, country: country, state: state, street1: street1, street2: street2,  street3: street3, zip: zip);
    }

    void nullSafeSet(java.sql.PreparedStatement st, java.lang.Object value, int index, org.hibernate.engine.spi.SessionImplementor session) throws org.hibernate.HibernateException, java.sql.SQLException {
        if ( value == null ) {
            StringType.INSTANCE.set( st, null, index );
            StringType.INSTANCE.set( st, null, index+1 );
            StringType.INSTANCE.set( st, null, index+2 );
            StringType.INSTANCE.set( st, null, index+3 );
            StringType.INSTANCE.set( st, null, index+4 );
            StringType.INSTANCE.set( st, null, index+5 );
            StringType.INSTANCE.set( st, null, index+6 );
        }
        else {
            final Address address = (Address) value;
            StringType.INSTANCE.set( st, address.getCity(), index,session );
            StringType.INSTANCE.set( st, address.getCountry(), index+1,session);
            StringType.INSTANCE.set( st, address.getState(), index+2,session);
            StringType.INSTANCE.set( st, address.getStreet1(), index+3,session);
            StringType.INSTANCE.set( st, address.getStreet2(), index+4,session);
            StringType.INSTANCE.set( st, address.getStreet3(), index+5,session);
            StringType.INSTANCE.set( st, address.getZip(), index+6,session);
        }
    }


    @Override
    public boolean isMutable() {
        return false;
    }

    @Override
    public boolean equals(Object x, Object y) throws HibernateException {
        // for now
        return x.equals(y);
    }

    @Override
    public int hashCode(Object x) throws HibernateException {
        assert (x != null);
        return x.hashCode();
    }

    @Override
    public Object deepCopy(Object value) throws HibernateException {
        return value;
    }

    @Override
    public Object replace(Object original, Object target, Object owner)
            throws HibernateException {
        return original;
    }

    @Override
    public Serializable disassemble(Object value) throws HibernateException {
        return (Serializable) value;
    }

    @Override
    public Object assemble(Serializable cached, Object owner)
            throws HibernateException {
        return cached;
    }

    public Class returnedClass() {
        return Address.class;
    }
}
And here is my Merchant which has an Address.
class Merchant {
    UUID id;

    String color;
    String displayName;
    //...
    //...

    Address address
    
    static mapping = {
        address type: AddressUserType, {
            column name: "city"
            column name: "country"
            column name: "zip"
            column name: "state"
            column name: "street1"
            column name: "street2"
            column name: "street3"
        }
    }

}
As stated, with this approach, the Address data structure could be used in other GORM objects. Until the next time take care of yourselves.