Agile IT Organization Design

General impressions

Agile promises to deliver solutions through collaborative effort, cross functional team design, modern programming methods and probably many more things. Because of that, it is hard to distinguish between part and parcel of Agile, and optional techniques that were developed over time. Agile IT Organization Design by Sriram Narayan is a bird’s eye view on Agile topics and tries to organize them.

Agile IT Organization Design

As the author mentions multiple things that are somehow related to Agile, starting with project estimations, through project finance and software development practices, ending with team room layout, the reader should be ready to jump over the wide variety of topics. Putting so many subjects in one book causes that specific aspects are discussed from time to time without any case-studies or detailed guidance how to kick start those ideas. It is valuable if you want to get a general idea, but mediocre if you want to get in-depth knowledge. There is nothing to worry about, because there are a lot of references to other publications or books, which may help you to explore more.

The good thing is that at the end of each chapter there is a summary, which after short skimming, helped me to create my own book reading order.

Author very often, explain reasons for doing things in a certain fashion. This approach helped me to reflect what and how I did in the past, and hopefully, it will help me make better decisions in the future.

Recommendations

This book is written for people from all walks of life. You can read it if you are a leader, a product owner, a software engineer or a stakeholder. It would be favourable if every person read that book, but it is not possible. If you work in a modern organization and you have the general understanding of bolts and nuts of Agile processes, you will probably not lose much by skipping that book. It is a good book for someone who would like to take a break from technical books and read something without code listings inside. Also, I would recommend this book to reach out for a particular chapter which are in the area of your interest.

Key takeaways

I have highlighted over 50 sentences in this book. Some of them to remember, other to think about or to discuss them with colleagues. I have selected 9 that stuck in my mind:

  1. End-to-end cycle time is more important than team development velocity.
  2. Doing sprints doesn’t make it iterative development.
  3. An engineering team without a clear leader may have trouble settling down on a solution design.
  4. The product is never mature; it is always maturing until it is finally obsolete.
  5. Move from a project centric model of execution to a product-centric model.
  6. Labeling people with job titles like an engineer, a manager, a QA, may have side effects.
  7. Spoken language influences the way we think.
  8. People say tl;dr to anything longer than a tweet.

Three levels of TDD

Introduction

I’ve been using TDD technique for a few years. Most of the time with satisfactory a result. But it wasn’t an easy journey; it was a trip full of ups and downs. During this period my thinking about TDD has changed dramatically, or maybe I have changed my perception of testing and software development during this time? Indeed, yes I have.

Lasse Koskela in his book called “Test Driven: TDD and Acceptance TDD for Java Developers.” wrote that “TDD is a technique that evolves together with the practitioner.” In this blog post, I would like to describe my own evolution in this matter.

Red Green Refactor Circle

Level 1 - Fundamentals

You begin your journey with TDD. When you are new into something, you want to follow the rules strictly. One is TDD circle, which is “RED, GREEN, REFACTOR”. You have also heard about three laws of TDD defined by Uncle Bob:

  • You are not allowed to write any production code unless it is to make a failing unit test pass.
  • You are not allowed to write any more of a unit test than is sufficient to fail, and compilation failures are failures.
  • You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

You are very confused about TDD because all examples that you can find relate to some mathematical/algorithmic problems. But in your daily job, you are obligated to write features, talk to DB and the other systems. You probably are struggling with complex dependencies, maybe you have to use mocks.

But finally, after some practice you start to see the benefits, which are:

  • You’ve noticed short feedback loop. Immediately, when you complete your implementation, you can launch the test to verify the correctness of your code.
  • Test code coverage gets higher. No matter how you measure it, it will be higher.
  • Regression is not a problem. Because when you break previous functionality during refactoring, you will instantly know that.

Level 2 - Requirements

Task lists

Task lists work perfectly for me. When I implement a business requirement, each small step or each corner case is represented by one task in my task list.

Then for each task I write one test, often I use parametrized tests to extend tests quickly. Finally, after a few TDD circles, my task is finally completed, and I can move on.

But sometimes during my work new system requirements appear. Often because the domain is so complicated that it’s hard to predict all the functionality up front. There is a big temptation to do it now, during the work on the current task, but it is dangerous. By doing it, you can lose your focus on your current goal.

I’ve practiced the habit which consists of adding this new requirement as a new task to my task list and complete it after the current one. Then you gain some time to think about this need, to decide if it is an essential functionality to do.

BDD

At some day, you will discover Behaviour Driven Development. For example, look at this specification:

Scenario
1
2
3
4
5
Scenario: Customer has a broker policy so DOB is requested
Given I have a "Broker" policy
When I submit my policy number
Then I should be asked for my date of birth

It is a very well written test scenario. Moreover, it is an executable scenario. This text can be executed with the tool called Cucumber. You don’t have to use it. You can use standard test framework and write your test using fluent test libraries or you can build your fluent API for tests if needed.

Start writing tests that will not only check your code but also be valuable documentation for your system.

Level 3 - Understanding

Show me your tests and I will tell you everything about your code.

TDD sometimes can also mean “Test Driven Design”. When you start thinking about it, your main reason for writing the tests is to refactor and re-engineer your codebase freely. For me, it is the highest value which you can get from TDD. How to achieve it? Try not to “cement” your code. Try to test interfaces or facades but not bolts and nuts of the implementation.

How to check if your tests are correct? Remove production code and try to rebuild it in a different way basing only on tests.

Summary

In this article, I presented fundamental rules of TDD. The topic of requirements were also discussed. In the end, I told you about Test Driven Design which for me is a valuable part of this technique. I hope that your understanding of TDD will improve and you will start writing better tests and better systems.

I gave a speech about TDD. Slides available at tdd.lewandowski.io

Photo credits: Banner, Thumbnail

Fish shell - Load environment variables from file

In many places, you can find environment files, with the following structure:

1
2
FOO1=BAR1
FOO2=BAR2

When you try to evaluate, this file using source command, you get an error with the fish shell.

1
2
$ source web.env                                                   
Unsupported use of '='. In fish, please use 'set FOO1 BAR1'.

This is very annoying, so I’ve decided to write a function that can read this file and load those variables. Here is how you can use it:

1
2
3
$ posix-source web.env
$ echo $FOO1
BAR1

The source code of this function. Enjoy:

1
2
3
4
5
6
7
$ cat .config/fish/functions/posix-source.fish                      
function posix-source
for i in (cat $argv)
set arr (echo $i |tr = \n)
set -gx $arr[1] $arr[2]
end
end

Photo credits: Banner, Thumbnail

Lessons learned from Decision Maker

In the past few weeks, I’ve read Getting Things Done, Technical Leadership, Elon Musk Biography and The Decision Maker.

Each of these books was good. But “The Decision Maker” is a game changer and I can’t stop thinking about this book. It was worth reading - for sure. I’ve decided to write a short book review and note the most important facts that I’ve learned from this book.

Review

This book is a story about a company and its new owners who have left the corporation and decided to build a great place to work. It is full of dialogues, issues, and situations.

By observing those scenes, the author presents ideas and values that matter when you have to lead the team or the company.

Is this book only for managers or bosses? Certainly not. If you work with other people or deal with non-trivial tasks, this book is for you. For me, it is an appropriate supplement for any “Agile” book.

Blueprint presented in this book is a good starting point for setting up company culture.

The story did not take place in reality. Each scene looks genuine, but as a whole, it seems artificial. Like a romance from 90’s, when you know they will live happily ever after.

Lessons Learned

Spoiler Alert!

In the following section, some elements of the book are about to be revealed.

People

To begin with, you have to change your thinking about other people.

People:

  • are unique,
  • are creative,
  • are able to learn,
  • have different strong points,
  • have different needs,
  • like a challenge,
  • are capable of changing the environment,
  • are capable of making contribution,
  • can be trusted.

Among some people, you can see those values. Among others, you have them hidden, and you have to unlock them.

But there is always somebody who disagrees with it and this is important to remember it. Do you see any similarities with Theory X and Y employees?

Decision Maker

Secondly, you have to choose the Decision Maker. It is a person who makes a decision. How to find them? It is simple.

The Decision Maker is a person, who is closest to the action. Bosses or leaders are not often deeply familiar with the situation. Usually, team members are often closer to the problem.

The Decision Maker has to be capable of listening and understanding other people. Making a decision is a process, in which you have to talk and listen to the others.

The Decision Maker should be aware of what is going on. Awareness of facts and consequences is crucial. If the person does not have basic data for making decisions - like company current finance status - you are responsible for unlocking that data.

Wisdom and knowledge are desirable qualities of that person.

It is a leader’s job to choose the Decision Maker. The leader should also observe and monitor the Decision Maker to see if he makes good decisions. If not, something should be done by the leader.

Results of making decision

It turns out that your employees’ decisions are often as good as or even better than yours can ever be.

People who are allowed to make the decision feel the ownership, because of that they will do everything to make the best possible decision.

Advisory process

The purpose of the advisory process is to look for a wider perspective.
The Decision Maker should ask at least a few people what they think about the decision.
He or she should ask:

  • team members,
  • other people with experience,
  • subordinates and superiors,
  • anyone who can help.

But the Decision Maker should take the final call.

Silver bullet

The decision maker process is not a silver bullet. It is only one tool or technique. The bigger picture is not straightforwardly visible in the book.

Between the lines, you can see many behaviours and dialogues which look familiar in “Teal Organizations”. If your organization is not ready, the decision maker process is definitely not the road to follow.

Photo credits:
Banner
Thumbnail

Formatting Java Time with Spring Boot using JSON

The aim of this post is to summarize and review ways of formatting Java Time objects using Spring Boot and Jackson library.

This post is organized in five steps. Each step represents one aspect of the issue and it is also related to one commit in example project repository.

Step 0 - Prerequirements

Versions and dependencies

This tutorial is based on Spring Boot version 1.3.1.RELEASE with spring-boot-starter-web. It uses jackson-datatype-jsr310 from com.fasterxml.jackson.datatype in version 2.6.4, which is a default version of Spring Boot. All of these is based on Java 8.

The Code

In the example code repository, you can find one HTTP service made with Spring Boot. This service is a GET operation, which returns a class with Java Time objects.
You can also find the integration test that deserializes the response.

Step 1 - The goal

I would like to return class Clock, containing LocalDate,LocalTime and LocalDateTime, preinitialized in constructor.

Clock - Service response class
1
2
3
4
5
6
public final class Clock {
private final LocalDate localDate;
private final LocalTime localTime;
private final LocalDateTime localDateTime;
...
}

Response class is serialized to JSON Map, which is a default behaviour. To some extent it is correct, but ISO formatted Strings in response are preferable.

LocalDate - response as JSON Map
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{  
"localDate":{
"year":2016,
"month":"JANUARY",
"era":"CE",
"dayOfYear":1,
"dayOfWeek":"FRIDAY",
"leapYear":true,
"dayOfMonth":1,
"monthValue":1,
"chronology":{
"id":"ISO",
"calendarType":"iso8601"
}
}
}

Integration testing is an appropriate way to test our functionality.

Example of integration test
1
2
3
4
5
6
ResponseEntity<Clock> resp = sut.getForEntity("http://localhost:8080/clock", Clock.class);

assertEquals(OK, resp.getStatusCode());
assertEquals(c.getLocalDate(), resp.getBody().getLocalDate());
assertEquals(c.getLocalTime(), resp.getBody().getLocalTime());
assertEquals(c.getLocalDateTime(), resp.getBody().getLocalDateTime());

Unfortunately, tests are not passing, because of deserialization problems. The exception with message is thrown can not instantiate from JSON object.

Step 2 - Adds serialization

First things first. We have to add JSR-310 module. It is a datatype module to make Jackson recognize Java 8 Date & Time API data types.

Note that in this example jackson-datatype-jsr310 version is inherited from spring-boot-dependencies dependency management.

Dependency in pom.xml
1
2
3
4
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</dependency>

Response is now consistent but still, not perfect. Dates are serialized as numbers:
Dates serialized to numbers and integers
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
{  
"version":2,
"localDate":[
2016,
1,
1
],
"localTime":[
10,
24
],
"localDateTime":[
2016,
1,
1,
10,
24
],
"zonedDateTime":1451640240.000000000
}

We are one step closer to our goal. Tests are passing now because this format can deserialized without any additional deserializers.
How do I know?
Start an application server on commit Step 2 - Adds Object Mapper, then checkout to Step 1 - Introduce types and problems, and run integration tests without @WebIntegrationTest annotation.

Step 3 - Enables ISO formatting

ISO 8601 formatting is a standard. I’ve found it in many projects. We are going to enable and use it.
Edit spring boot properties file application.properties and add the following line:

application.properties file - disabling timestamps write
1
spring.jackson.serialization.WRITE_DATES_AS_TIMESTAMPS = false

Now, the response is something that I’ve expected:

Dates serialized as Strings
1
2
3
4
5
6
7
{  
"version":2,
"localDate":"2016-01-01",
"localTime":"10:24",
"localDateTime":"2016-01-01T10:24",
"zonedDateTime":"2016-01-01T10:24:00+01:00"
}

Step 4 - Adds on demand formatting pattern

Imagine one of your client systems does not have a capability of formatting time. It may be a primitive device, or microservice that treats this date as a collection of characters. That is why special formatting is required.

We can change formatting in response class by adding JsonFormat annotation with pattern parameter. Standard SimpleDateFormat rules apply.

Using @JsonFormat annotation
1
2
3
4
5
@JsonFormat(pattern = "dd::MM::yyyy")
private final LocalDate localDate;

@JsonFormat(pattern = "KK:mm a")
private final LocalTime localTime;

Below there is a service response using custom @JsonFormat pattern:

Custom response style
1
2
3
4
5
6
7
{  
"version":2,
"localDate":"01::01::2016",
"localTime":"10:24 AM",
"localDateTime":"2016-01-01T10:24",
"zonedDateTime":"2016-01-01T10:24:00+01:00"
}

Our tests are still passing. It means that this pattern is used for serialization in service and deserialization in tests.

Step 5 - Globally changes formatting

There are situations where you have to resign from ISO 8601 formatting in your whole application, and apply custom made standards.

In this part, we will redefine format pattern for LocalDate. This will change formatting of LocalDate in every endpoint of your API.

We have to define:

  • DateTimeFormatter with our pattern.
  • Serializer using defined pattern.
  • Deserializer using defined pattern.
  • ObjectMapper bean with custom serializer and deserializer.
  • RestTemplate that uses our ObjectMapper.

Bean ObjectMapper is defined with annotation @Primary, to override default configuration.
My custom pattern for LocalDate is dd::MM::yyyy

Object mapper bean with custom pattern
1
2
3
4
5
6
7
8
9
10
11
12
13
14

public static final DateTimeFormatter FORMATTER = ofPattern("dd::MM::yyyy");

@Bean
@Primary
public ObjectMapper serializingObjectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
JavaTimeModule javaTimeModule = new JavaTimeModule();
javaTimeModule.addSerializer(LocalDate.class, new LocalDateSerializer());
javaTimeModule.addDeserializer(LocalDate.class, new LocalDateDeserializer());
objectMapper.registerModule(javaTimeModule);
return objectMapper;
}

Definitions of serializer and deserializer for all LocalDate classes:

Custom serializer and deserializer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
public class LocalDateSerializer extends JsonSerializer<LocalDate> {

@Override
public void serialize(LocalDate value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
gen.writeString(value.format(FORMATTER));
}
}

public class LocalDateDeserializer extends JsonDeserializer<LocalDate> {

@Override
public LocalDate deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
return LocalDate.parse(p.getValueAsString(), FORMATTER);
}
}

Now, the response is formatted with our custom pattern:
Formatted response
1
2
3
{  
"localDate":"01::01::2016"
}

Tests

When we define custom serializer, our tests start to fail. It is because RestTemplate knows nothing about our deserializer. We have to create custom RestTemplateFactory that creates RestTemplate with object mapper containing our deserializer.

Custom RestTemplateFactory
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
@Configuration
public class RestTemplateFactory {

@Autowired
private ObjectMapper objectMapper;

@Bean
public RestTemplate createRestTemplate() {
RestTemplate restTemplate = new RestTemplate();
List<HttpMessageConverter<?>> converters = new ArrayList<>();
MappingJackson2HttpMessageConverter jsonConverter = new MappingJackson2HttpMessageConverter();
jsonConverter.setObjectMapper(objectMapper);
converters.add(jsonConverter);
restTemplate.setMessageConverters(converters);
return restTemplate;
}
}

Conclusion

Custom formatting Dates is relatively simple, but you have to know how to set up it. Luckily, Jackson works smoothly with Spring. If you know other ways of solving this problem or you have other observations, please comment or let me know.

Photo credits: Banner, Thumbnail