Making good software

My 7 principles to design the architecture for a software project.

Software architecture is quite a grey area. It involves taking decisions that you are not likely to revisit in the future. These decisions usually involve frameworks, programming languages, application servers, databases, e.t.c.
Software architecture is about coming up with the foundation from where developers are going to build up the application. This also includes: development environments: (source control, building tools, IDEs…), QA environments, Continuous integration environments, etc.
Bad architectural decisions can make the development of your project a complete failure, and are the most difficult decisions to revert.
Good architecture, in the other hand, for most projects, doesn’t really bring any advantage, it only allows for its normal development. Good architecture usually remains hidden under normal circumstances.
When designing the architecture, is essential to avoid bad decisions that are going to tax your development in the future. What follows is a list of principles to help you come up with safe and scalable architectures.

1.- Start with the minimal necessary architecture.

Avoid unnecessary complexity like unnecessary new technologies or frameworks.
Any architectural element needs to have a very good reason to be used in the project. If the benefits or necessity of an architectural element can’t be proved, then that architectural element should be removed.

2.- Consider the specifics of your project: Constraints and risks.

Defining the architecture is something that has to be adapted to the constraints and risks of your specific project.
You shouldn’t aim for the same kind of architecture if it is a start-up or a project for a multi-national company. You should also consider the expertise of your team, if they are very experienced in a particular programming language/framework… you should probably use it even if it is not the one you prefer.
Risk areas, like a requirement to have some service with a high availability, are going to require additional considerations in the architecture that you need to keep in mind.
Constraints and risks are the element that are going to move you away from your comfort zone, but they need to be looked at, to make sure that if necessary, they get addressed from the architecture.

3.- Grow as you need.

Delay any architectural element as much as you can, especially if they are complex.
Sometimes requirements seems to call for complex integrations/frameworks… like rules frameworks, ESBs… for these cases, I find it better to ignore them at the beginning and try to complete the first end-to-end scenario as soon as possible. When this first scenario is completed, then grow from there, one step at a time, adding new functionality as requested by the user. This way, it will become obvious if that architectural element is really necessary, and if it is, it will also be obvious how you should integrate it with the rest of the architectural elements in your project.

4.- Deliver continuously.

Continuous delivery is the vehicle to allow you to refine your architecture as you go.
Continuous delivery is based in four principles:
Deliver soon: The purpose of delivering soon is to shorten the feedback cycles with the customer, allowing for changes in the requirements if necessary.
Deliver many times: Delivering many times guarantees that the feedback cycle is smooth.
Bug free: The code must be clean of critical bugs.
Production ready: It should be deployable at any time.

5.- Require a customer.

Having someone acting as a customer, making decisions about priorities and signing off releases is critical for the good development of your project.
Failure to have someone representing correctly the role of the customer causes misdirection in the development and consequently causes that the final product developed won’t meet expectations.

6.- Avoid waste.

Architecture can lead to an excess production of artefacts, inexperienced architects may pretend to foresee the entire architecture of the application and have it specified in paper.
This is especially true in companies following waterfall like processes, and sometimes it cannot be avoid, forcing the architects to produce such and such document/diagram.
Excessive initial architecture/design, aka Big design upfront (BDUF), is not only bad because is wasteful, its main risk is to let the development team believe that they cannot deviate from the original specifications, making it very difficult to be adaptive to future changes.

7.- Maximise feedback and transparency

Feedback and transparency guarantees no surprises for the customer, which in turn helps building a more trustworthy and productive relationship.
Maximising the feedback and transparency are one of the ultimate goals of the development process for any project, this allows for early and informed reaction when change is necessary, whether something is going wrong, or whether the customer wants to introduce a change in the specification

Agile: The good, the bad and the ugly.

Most companies are adopting Agile as the “de facto” process for their new projects, while is clear that overall is better than waterfall, it still has its defects. This article does a brief overview of what is good, bad and ugly about agile.

The Good.

Embrace change. Change is the only constant in software development; not embracing this is like swimming against the flow. Embrace change is precisely Agile’s motto. Agile supports change by developing in short iterations, which allow for faster feedback and consequently allows for changes in the features to be developed.
Empower programmers. Good, empowered programmers make good software. But that’s not easy, agile empowers programmers by giving them the most important role in software development moving it away from contracts, schedules and management.
Disregard BDUF. Big design upfront is one of the major signature features of the classic waterfall developments, not a single line of code is produced until a complete design of the application is completed. This is not only bad because it goes completely against embracing change, is also not realistic since any complex design has too much uncertainty making it impossible to produce a complete correct design upfront.

The Bad.

In some cases, an extreme approach to Agile may cause some issues, which from all of them IMHO the next ones are the most relevants:
Too much focus on the code. All code sucks, pushing too hard to produce beautiful code is pointless. This doesn’t mean that there aren’t big differences between different codes, there is code that is terrible and there is code that is alright, but there isn’t beautiful code. Unfortunately extreme agilists tend to push the idea that the beautiful code is achievable causing developers to spend too much time looking after it.
Complete disregard of design. BDUF is bad, but not doing any design at all is also bad. In extreme agile there is a tendency to let the code drive the design, which is the opposite of BDUF. The ideal situation is a healthy balance where some informal design and thinking are performed before coding, then some coding, then some more design and so on.

The Ugly.

Apart from tendencies which are clearly bad, agile also promotes some other practices, which while not being extremely bad feel like a waste or like they add very little value.
Holy events. In agile there are many activities that the team performs religiously, like stand ups, planning meetings and so on. These activities are performed like mass, forcing the same structure and same formulas all over again. These “holy events” usually add very little value and may even become a communication anti pattern.
Alternative estimation methods. Accurate software estimation is impossible, for two main reasons, change and uncertainty. Ironically some agile families still try to provide with some sort of reliable estimation mechanisms.

Managing crises effectively in software development projects.

In a green field project, there is always an initial euphoric phase where everything is great.
Unfortunately, sooner or later things turn for the worse and the first crisis hits the project.

You cannot avoid having a crisis, but you must manage them. There hasn’t ever been any project that didn’t have a crisis, and yours is not going to be an exception.

Crises are about not being been able to meet expectations and they involve different actors, including “the business”.

Manage expectations, make sure that “the business” understands that having crises is normal, and do it before they hit you; the best period is the honeymoon that is the start of the project. Sit with the business and explain what your crisis plan is, availability, backup plans …

Crises can be sometimes prevented beforehand, the worst type of crises are the ones caused by development not being honest with the business, common examples are not telling that the development is late or lying on the real set of features already developed.

Be transparent. Managing a project is to take responsibility when things go wrong and making sure that all the stakeholders are well informed of what is the current status of the project. Hiding information is never good in the long run.

Unfortunately, one of the most difficult activities is to learn what the real status of the project is. There are many projects that died because it was discovered too late that the project was heading the wrong direction and it was too late.

Be aware of what the real status of the project is. Many project managers are so much obsessed with only reporting green light to upper management that they live in denial. Gather as much information as possible, disregard subjective information as “this should be fine”, or “we are almost done”.

The budget and or the schedule, can’t be both sacred, that’s one of the toughest realizations in software development and one of the most difficult things to explain for project managers, the main reason for this is uncertainty, there is always a lot of uncertainty in software development that causes a lot of change.

Manage change and uncertainty. Be aware when the conditions of the project change and make sure that the necessary actions are taken to accommodate the project to the new conditions.

Even when the project is under control, you still are going to be face with crises, but they are going to be crises of a different nature, mainly accidental nature: data lost, hardware failures, bugs…

Be always ready for accidental crises. Have an emergency plan, you need to know who, how and when can be contacted in case of emergency and what availability they are supposed to have. You also need a data restoration plan. Most importantly remember to do fire drills: Simulate crises in a production like environment and make sure that everything works as you expect.

As already mentioned, there is nothing that can fully prevent you from having a crisis, so when they do appear, make sure you react accordingly.

Be professional. React calmly but with decision. Focus your energies in finding a solution for your crisis.

When there is a crisis, the project mentality switches to emergency mode. In emergency mode everyone should be given the 110% of themselves, this is very productive, but also, very stressful.

Try to spend as less time as possible in emergency mode. When you spend too much time under a high quantity of pressure you will start becoming less productive.

Too many crises in a short period of time is usually the sign of a major underlying issue, the worst remedy is to keep the team in perpetual emergency mode trying to extinguish all the fires at the same time.

On perpetual crises. Stop, analyze the situation and tackle one crisis at a time. This is sometimes very difficult to explain to the stakeholders, which may expect immediate solutions, but is the only way out in the medium-long term.

Unfortunately crises are also usually recurrent, and they keep reappearing.

When tackling a crisis be sure not to focus only in the consequences, but also the causes. A crisis should be fixed with guarantees that it won’t happen again in the future.

One of the biggest differences between software development teams is how they react to crises, being able to manage them makes a huge difference. Keep in mind that many times making mistakes is not as important as how you manage the situation after a mistake is being made.

Top 10 basic java advices.

1.- Make the classes that contain the logic of your application stateless.

A stateless class is a class which doesn’t have state, or which state is formed by injected dependencies used to delegate behaviour. Since these classes don’t have state, the outcome of their methods is only dependant on their input. Stateless classes have the following advantages.

Thread safe friendly.
Deterministic behaviour
Easy to test.

2.- Use unmodifiable beans as much as you can.

Unmodifiable beans are classes which state are set on creation and from them are prevented to change; in java, using the modifier final in your member fields best approaches this.
Unmodifiable beans offer the following advantages.

Thread safe friendly. Actually: stateless logic classes + unmodifiable beans = Thread safety!
Robustness. Unmodifiable beans eliminate the risk of holding references to objects that keep changing.

3.- Try to have your classes matching one of the following categories.

There are two main concerns a class can deal with: logic and data modelling. Having classes that deal with logic or data that don’t belong to them make your code more confusing and cluttered. Based on these two concerns, there are mainly 5 types of classes that you should try to adhere to:

Service. 100% logic, publishes the main public end points of your application, they usually delegate to managers through dependency injection.
Bean. Mainly data model but also logic, used to carry data and perform operations, the operations the bean can perform should be self contained.
DTO. 100% data model, is a particular type of bean used to transport data betweeen different mediums.
Business logic manager. 100% logic, encapsulates the implementation of the business rules defined for your domain.
Integration manager. 100% logic, glues together business logic manager + external frameworks used in your project + beans.

4.- Avoid static methods.

Static methods usually contain miscellaneous logic required in your application, because is difficult to find an already existing class in the code to host them, they sometimes are created as static methods in abstract classes. It is much better to encapsulate the static methods as instance methods in utility classes such as AppUtils… The disadvantages of using static methods are:

They are more difficult to mock out in tests
Their implementation can’t be hidden behind an interface
They can’t be injected.

5.- Use composition over inheritance.

Inheritance heavily violates encapsulation, and easily generates obscure code. One clear example is the java io library which entirely designed with the decorator pattern. Composition allows you to have completely equivalent functionality, but without breaking encapsulation, it is also the core concept of the most important design pattern from all them all, the strategy pattern.

6.- Design in layers.

Almost every single business application has a few broad main areas of concern that are completely unrelated from each other. These areas of concern should be translated into layers in your design to allow for a cleaner code. The most common layers are:

Persistency
Model
Integration
UI

7.- Avoid child to parent, or child to sibling relationships.

Design is the art of abstracting a problem into its different entities and their relationships. These relationships most of the time express some sort of parent-child relationship, as entity A contains entity B, or entity A manages entity B. Having these relationships implemented only from the parent to the child allows for clearer an less coupled code. One clear example of this would be the way Lists are implemented in Java.

8.- Keep the invariants to be maintained by the programmer to a minimum.

Invariants are conditions that need to be kept in order for an algorithm to work as expected. For instance, constants are usually defined with Simple data types as integers, strings… and if they are used anywhere in your code, is to be expected that you will receive not any value, but the ones specified by your constants. Other examples of invariants are not so obvious, imagine that there is a service with a method process, that retrieves an order, the only parameter that should be accepted is the order id, if the method requires something else, like the creation date of the order, we are adding a unnecessary invariant to be kept by our consumers.

9.- Avoid side effects in methods.

A method should only do one thing. Methods with side effects are dangerous, as the output of the method is never clear, you should always aim for deterministic methods with no side effects and which only read their input, never change them.

10.- Test.

Test, test, test… In software development is more productive to produce 7/10 of a robust application than a completely flaky application. Based on the previous classification of classes I would recommend the following approach for testing.

Don’t test your beans.
Unit tests your business logic managers.
Integration tests for services and integration managers.
And finally… always spend some time doing manual end to end test.

10+1.- Your judgement must be above everything.

Software development is not an enginery; there isn’t a set of well-defined problems that have well-defined solutions. Is not like building a bridge, or a building… every development is unique, what applies to one project doesn’t necessarily apply to other. What this means, is that there isn’t absolutely any rule that has to always be applied, all are best practices, or advises, but is up to you to decide what approach is better suited for the problem you have to solve now.

Top 7 programmers bad habits

1.- The all code is crap, except mine, attitude.

I have bad news for you buddy, all code is crap. No matter how much effort you put on it, there is always a majority of programmers who are going to think that your code sucks and that they could have done it 10 times better. I have already covered this topic in previous posts, you can find more information of what exactly I mean when I say that all the code is crap here and here.
How to fix it: Don’t criticise others people code, it could be yours the one in the spotlight, try to make objective and professional observations instead, but don’t judge. Be humble and try to learn from everyone around you, hopefully then your code won’t be so bad.

2.- The “I fix that in a second” catastrophe.

Taking shortcuts is very tempting, everyone has done it. There are actually situations where they are necessary, but in overall, they are dangerous, very dangerous and should be avoided. A shortcut which goes wrong may save you a few hours, but may cause months of pain.
How to fix it: Don’t trust yourself when carrying delicate activities. Ask someone else to review what you are doing. Make sure that if you are about to take a shortcut, you make very clear to the stakeholders what the reasons and the risks are. Try to get a manager to sign off every time you are about to take a shortcut, aka “cover your ass”.

3.- The “That will only take a second” misconception.

Being Barcelona my hometown, I am very proud of the Sagrada Familia Cathedral, which is very well know for its beautiness, and also for the time is estimated it will take to complete, (in construction since 1882), but that’s probably because they didn’t ask a programmer to estimate, otherwise the estimate would probably have been somewhere around 2 weeks.
How to fix it: For starters, is important to understand that accurate estimations in software development for non trivial solutions are impossible, we can only guess. Also remember that is very likely that you will find so many things which you didn’t foresee when you start developing that is worth multiplying the estimate to cover for those, I usually go with 1.5 or 2.

4.- The ego spiral.

The last design meeting...

Many programmers discussions look more like rooster fights than human discussions. This usually happens in design and architectural meetings. It is actually quite easy to detect this ego spirals, you just have to substitute most of what the contenders are saying with COC! COC! COCOCOOCCC! COOC!
How to fix it: Leave your ego at home. Big egos are one of the biggest non technical issues for any programmer. Keep in mind some basic considerations when making decisions.

5.- “It wasn’t me!”

In my opinion, other bad habit from most programmers is the lack of accountability. We always have an excuse… It’s like if we were saying that under normal conditions we would never make a mistake, which honestly is quite hard to believe.
How to fix it: No need to cry, or to perform seppuku, (aka harakiri), when we make a mistake. Having a healthy attitude where we can you just say something like: “yeah, sorry, now we need to do this to fix this issue, my fault” is a very professional attitude, and it will help you to build a reputation and to be better considered by your colleagues.

6.- The demotivated genius.

Repetitive and simple tasks are not the best motivators, but they need to be done, programmers tend to get demotivated and very unproductive when they need to complete them.
How to fix it: Discipline. Unfortunately, there isn’t any other remedy I can think of.

7.- The premature programmer.

If programming was sex, there would be a lot of unsatisfied computers. You can just not go in, do things halfway through and then fall sleep. One of the concepts that I find most programmers struggling with is the concept of “Done”. Remember that done means: tested (and not only unit tested), documented, committed, merged…
How to fix it: This one is tricky, the complexity of non apparent necessary tasks to fully complete some functionality is quite high and usually requires discipline and training. Probably the two easiest ways to help a programmer understand if some code is done are peer reviews and demos.

Progamming a chess engine with Java (I) – Finding where a piece can move.

A few weeks ago, I posted an article describing some Java code which would be able to solve any Sudoku using tree data structures. After this article, I thought it would be nice to continue developing around tree data structures, and that’s when I decided to code a chess engine in Java.
This is the first installment of a series of articles that will describe this chess engine as I am developing it. It will also use, the tree library that I created for the Sudoku resolver. You can find the complete source code in the link provided at the bottom of the article, any feedback on the code will be very much appreaciated.

The Chess Board class

view plaincopy to clipboardprint?

public class ChessBoard {
private final Squares squares;
public ChessBoard() {
this (new Squares());
}
public ChessBoard(Squares squares) {
this.squares = squares;
}
public ChessBoard performMovement(Piece piece, Location locationFrom, Location locationTo) {
Squares copy = squares.copy();
copy.setContent(SquareContent.EMPTY_SQUARE, locationFrom.getCoordinateX(), locationFrom.getCoordinateY());
copy.setContent(piece, locationTo.getCoordinateX(), locationTo.getCoordinateY());
return new ChessBoard(copy);
}
public ChessBoard performMovement(PieceOnLocation pieceInBoard, Location locationTo) {
return performMovement(pieceInBoard.getPiece(), pieceInBoard.getLocation(), locationTo);
}
public ChessBoard performMovement(Movement movement) {
return performMovement(movement.getMovingPiece(), movement.getFrom(), movement.getTo());
}
public SquareContent getSquareContent(Location location) {
return squares.getContent(location.getCoordinateX(), location.getCoordinateY());
}
public ChessBoard addPiece(Piece piece, Location location) {
Squares copy = squares.copy();
copy.setContent (piece, location.getCoordinateX(), location.getCoordinateY());
return new ChessBoard(copy);
}
public List<PieceOnLocation> getPieces(Color movingColor) {
List<PieceOnLocation> pieces = new ArrayList<PieceOnLocation>();
for (Location location : Location.values()) {
SquareContent content = squares.getContent(location.getCoordinateX(), location.getCoordinateY());
if (! content.isEmpty()){
Piece piece = (Piece) content;
if (piece.getColor() == movingColor) pieces.add(piece.on(location));
}
}
return pieces;
}
public List<PieceOnLocation> find(Piece pieceToFind) {
List<PieceOnLocation> pieces = new ArrayList<PieceOnLocation>();
for (Location location : Location.values()) {
SquareContent content = squares.getContent(location.getCoordinateX(), location.getCoordinateY());
if (! content.isEmpty()){
Piece piece = (Piece) content;
if (piece == pieceToFind) pieces.add(piece.on(location));
}
}
return pieces;
}
public PieceOnLocation getPieceOnLocation(Location location) {
return new PieceOnLocation(getSquareContent(location).asPiece(), location);
}
public ChessBoard empty(Location location) {
Squares copy = squares.copy();
copy.setContent(SquareContent.EMPTY_SQUARE, location.getCoordinateX(), location.getCoordinateY());
return new ChessBoard(copy);
}
}

About the Chess Board class is important to notice 3 things:

Is thread safe. All is member variables are private and final, and is immutable, all the “changing operations” create a copy of the object with the new state.
It delegates all the functionality to Squares. Squares is the main class, it contains all the logic for holding SquareContent objects (Pieces and empty squares). It is wrapped by the Board to achieve thread safeness.
It is easy to create and customise a board, we only have to create a new board, and call the method addPiece. (See example below).

Creating a new board with a few pieces.

view plaincopy to clipboardprint?

board = new ChessBoard().
addPiece(Piece.BLACK_PAWN, Location.C5).
addPiece(Piece.WHITE_PAWN, Location.D5).
addPiece(Piece.WHITE_KING, Location.D4);

The Chess Analiser

This class contains the main entry points for all the main logic, in particular, we are interested in the method findReachableLocations and in the default implementation ChessAnaliserImpl.

view plaincopy to clipboardprint?

public interface ChessAnaliser {
[...]
public List findReachableLocations(PieceOnLocation pieceOnLocation, ChessBoard board, Movement previousMovement);
}

view plaincopy to clipboardprint?

public class ChessAnaliserImpl implements ChessAnaliser {
@Override
public List<Location> findReachableLocations(PieceOnLocation pieceOnLocation, ChessBoard board, Movement previousMovement) {
Piece piece = pieceOnLocation.getPiece();
List<Location> toReturn = new ArrayList<Location>();
for (MovementLine movementLine : chessReader.findMovementLines(pieceOnLocation)) {
if (movementLine.isApplicable (board, previousMovement)){
List<Location> potentiallyReachablePositions = movementLine.filterPotentiallyReachablePositions (pieceOnLocation.getColor(), board);
for (Location location : potentiallyReachablePositions) {
ChessBoard nextPossibleBoard = nextBoard(board, movementLine.getType(), pieceOnLocation, location, previousMovement);
if (!isInCheck(nextPossibleBoard, piece.getColor())){
toReturn.add(location);
}
}
}
}
return toReturn;
}
}

Take into consideration:

[Line 07]. The base of any movement is the movement line. The MovementLine class defines the type of movement and the different locations for that movement type that a piece can potentially move to given a starting position. Movement lines don’t take into consideration other pieces in the board.
[Line 08]. The method isApplicable, in the movement line, checks if that movement line is applicable for a given board taking into consideration the previous movement. The clearest example here would be the movement line for a pawn to do an “en passant” movement, this method will check if all the conditions for an en passant movement are true.
[Line 09] The method filterPotentiallyReachablePositions checks from all the possible locations of a movement line, how many are reachable, so if there is a piece that is actually standing in the middle of the board for that movement line, it will filter out all the positions after that piece . (See examples below).
[Lines 11 and 12] For all the potential locations that given piece can move to, it finds what the next board would be, and then it checks wether if the movement would cause the pieces for that color to be in check. This would be an illegal movement, so the location would be discarded.

Tying everything together

The following are real examples to find what the possible locations for the given pieces are:

view plaincopy to clipboardprint?

public class ChessAnaliserImplIntegrationTest {
private ChessAnaliserImpl testObj;
private ChessReader chessReader = new ChessReaderImpl();
private ChessBoard board;
@BeforeMethod (groups="integration")
public void setup (){
testObj = new ChessAnaliserImpl (chessReader);
}
@Test (groups="integration")
public void testFindReachableLocations_forRook_withSameColorObstacles (){
//GIVEN
board = new ChessBoard().
addPiece(Piece.WHITE_ROOK, Location.A1).
addPiece(Piece.WHITE_QUEEN, Location.A5).
addPiece(Piece.WHITE_KNIGHT, Location.C1).
addPiece(Piece.WHITE_KING, Location.C8);
//WHEN
List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.A1), board, null);
//THEN
Assert.assertEquals(actualReachablePositions.size(), 4);
Assert.assertEquals(actualReachablePositions.get(0), Location.B1);
Assert.assertEquals(actualReachablePositions.get(1), Location.A2);
Assert.assertEquals(actualReachablePositions.get(2), Location.A3);
Assert.assertEquals(actualReachablePositions.get(3), Location.A4);
}
@Test (groups="integration")
public void testFindReachableLocations_forRook_withDifferentColorObstacles (){
//GIVEN
board = new ChessBoard().
addPiece(Piece.WHITE_ROOK, Location.A1).
addPiece(Piece.BLACK_QUEEN, Location.A5).
addPiece(Piece.WHITE_KNIGHT, Location.C1).
addPiece(Piece.WHITE_KING, Location.C8);
//WHEN
List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.A1), board, null);
//THEN
Assert.assertEquals(actualReachablePositions.size(), 5);
Assert.assertEquals(actualReachablePositions.get(0), Location.B1);
Assert.assertEquals(actualReachablePositions.get(1), Location.A2);
Assert.assertEquals(actualReachablePositions.get(2), Location.A3);
Assert.assertEquals(actualReachablePositions.get(3), Location.A4);
Assert.assertEquals(actualReachablePositions.get(4), Location.A5);
}
@Test (groups="integration")
public void testFindReachableLocations_forRook_whenCantMove (){
//GIVEN
board = new ChessBoard().
addPiece(Piece.WHITE_KING, Location.D1).
addPiece(Piece.WHITE_ROOK, Location.C2).
addPiece(Piece.BLACK_QUEEN, Location.A4).
addPiece(Piece.WHITE_KNIGHT, Location.C1);
//WHEN
List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.C2), board, null);
//THEN
Assert.assertEquals(actualReachablePositions.size(), 0);
}
@Test (groups="integration")
public void testFindReachableLocations_forPawn_whenInFirstRow (){
//GIVEN
board = new ChessBoard().
addPiece(Piece.BLACK_PAWN, Location.A7).
addPiece(Piece.BLACK_KING, Location.D4);
//WHEN
List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.A7), board, null);
//THEN
Assert.assertEquals(actualReachablePositions.size(), 2);
Assert.assertEquals(actualReachablePositions.get(0), Location.A6);
Assert.assertEquals(actualReachablePositions.get(1), Location.A5);
}
@Test (groups="integration")
public void testFindReachableLocations_forPawn_enPassant (){
//GIVEN
board = new ChessBoard().
addPiece(Piece.BLACK_PAWN, Location.C5).
addPiece(Piece.WHITE_PAWN, Location.D5).
addPiece(Piece.WHITE_KING, Location.D4);
Movement enPassantEnabler = new Movement(Piece.BLACK_PAWN, Location.C7, Location.C5);
//WHEN
List<Location> actualReachablePositions = testObj.findReachableLocations(board.getPieceOnLocation (Location.D5), board, enPassantEnabler);
//THEN
Assert.assertEquals(actualReachablePositions.size(), 1);
Assert.assertEquals(actualReachablePositions.get(0), Location.C6);
}
}

Source code

All the source code can be found in the following SVN URL http://subversion.assembla.com/svn/making-good-software/tags/mgsChess/blog20110515/

10 characteristics of a robust application or service.

Robustness is one of the most important qualities of any application, is also one of the most difficult to achieve. What follows is a list of what in my opinion are the 10 principal characteristics to be considered when designing any robust application.

1.- Transactions.

Having the right transactionality in your application is critical to guarantee the integrity of the data. No matter which framework you use to manage the transactions, is important that you consider making transactional the system calls that require ACID properties. Some key areas to look for:

Transaction modes.
Transaction frameworks and APIs.

2.- Database access.

Modern frameworks usually try to abstract the access to the underlying database of the application, while this is certainly good, is also causing an increasing lack of understanding of what is happening behind the scenes. Being able to explain what is the SQL likely to be executed by your framework, and when is going to be executed will help you to create better database access code, and to optimize the database access, specially if you don’t have a DBA in your project. Some key areas to look for:

SQL
Indexes
Lazy loading.
ORM frameworks and APIs
Caching

3.- Multithreading.

Multithreading is the next big thing in software development, is been there now for ages, but with the new hardware architectures going multi-core, multithreading is emerging as the new must-know skill. It is also critical to acknowledge that your code is probably already running in a multithreading platform so you need to consider it to prevent your code to fail. Some key areas to look for:

Stateless services
Unmodifiable beans
Race conditions
Locks

4.- Logging and auditing.

Logging and auditing are one of the most underrated qualities of an application, especially for server applications. Is easy to forget in development how useful a log is when the application has already been rolled into production. Some key areas to look for:

Consistent identifiers. Try to use the same identifier in every log message so is easy to track one thread of execution.
Consistent format: If you have to grep or quey your logs, you will appreciate using always the same format for your log messages.
Completion: It should be possible to track an entire thread of execution from the log being able to see the main events.

5.- Exception management.

An exception that gets thrown out to the user level is usually a sign of a fragile application, a proper exception handling can make you application more robust. I already wrote an article with some basic considerations regarding exceptions.

6.- Invariants.

Invariants are conditions that need to be kept across several data structures to guarantee the integrity of the logic of the application. There are several design techniques or language features, as encapsulation, generics, enums… to help keeping the invariants safe. Something so simple as changing a list of constants to an enum or encapsulating two attributes that need to change in synchronization can make a huge difference building a more robust application.

7.- Disabling/Enabling/Swapping services or modules.

Shit happens, so you better be ready, being able to disable a service, or a module, or being able to write a dummy implementation to overcome a temporary incident, accounting for this in development, may save your day.

8.- Ubiquity and low coupling.

Being able to run chunks of your application independently, and at any time, is a sign of good design, and can also help you recover for unforeseen errors.

9.- Self diagnosis.

One very valuable feature of any service call is to provide with self diagnosis on error, so the cause for the issue becomes obvious and it facilitates the search for a solution.

10.- Documentation.

Any non trivial system requires documentation. Good documentation made available to the right people can save you from many errors.

What is the mission of any software developer?

What is important in an application?

In software development there are only three measurable areas that encapsulate all that is important in an application: Rentability, quality and maintainability.

Rentability	Ultimate goal of any application. Determines if the amount of time and resources spent in maintaining the application are worth. Main responsibles: Business, sales, marketing.
Quality	Measures how well an application does what is supposed to do. Main responsibles: System administrators, DBA, developers, QA.
Maintainability	Measures how long it takes and how much it cost to successfully deploy a change/new development from the change request until is available for use. Main responsibles: Management, developers, QA.

From these metrics, is important to understand two things:

Rentability is the most important metric. No rentability = failure.
Quality and maintainability are only important because they have an impact in the rentability. They basically represent the cost of maintaining and fixing the application.

Some examples of these metrics would be:

Bank legacy applications. Very high rentability, very high quality but very bad maintainability.
Successful web start-ups. High rentability, low quality and good maintainability. New web-apps that manage to get a lot of attention, with a lot of bugs, but with a small core that is easy to change.

The mission

I find it quite interesting how many developers seem to ignore these three previous areas, the mission of any software developer should be: Increase the rentability of the application through the quality and maintainabilaty.
Is important to note how code is not mentioned. The code is not an end, is a mean. If what you are doing doesn’t improve the quality or the build process, then, don’t do it. This maybe interpreted as an attack against some programming techniques, in particular refactoring, but is not. Refactoring plays an important role in maintainability, as any other engineering practice, they serve a bigger purpose, and is not the code, is rentability.

Maximising the quality

Some of the main aspects of quality to look for are:
Correctness: Correctness is about expectations. A software feature is correct if it fulfils the expectations of the customer.
Traceability: Traceability is about being able to explain what happened in the application in the past and why it happened.
Supportability: Supportability is about how easy is to fix possible errors/corruptions in the application
Usability: Usability is about how good the overall user experience is on the application, is usually about performance, UI design…
Robustness: Robustness measures how gracefully the application handles edge cases.
Scalability: Scalability is the ability to handle growing amounts of work in a graceful manner

Improving the maintainability of the application

One of the main specific areas where developers can make a bigger difference is the build. Having a bad build process is one of the biggest productivity killers in software development. A build process comprehends all the stages a change goes through since a developer commits it until it reaches production, including all the QA and deploys activities. The output of a successful build is a set of new features that work as expected in production. Any build process has two main characteristics.
Reliability. The reliability of the build process is a measure on how frequently builds that were supposed to be successful turned out to have bugs, and how critical these bugs were.
Performance. Performance is a measure of how long it takes to make a build attempt.
But is worth remembering that a non reliable build is useless, no matter how fast it is.

The obsession with beautiful code, the refactor syndrome.

with 16 commentsre is lately an emerging obsession with beatiful code, up to the point where some programmers are overprioritizing it over more important goals such as:

Correctness
Robustness
Traceability
Supportability
…

When this is taken to the extreme, and all that the programmer cares about is the beauty of the code, can cause him/her to fall under the refactor syndrome.
The refactor syndrome has the following symptoms:

Complete disregard for other’s people code.
Obsession with naming variables and methods.
Overuse and overvaluation of unit testing.
Complete disregard for any sort of documentation, design or analysis.
Ignoring end to end, and manual testing.

And usually causes:

An explosion of services, managers, helpers or other utility like classes.
Integration bugs.
Lack of robustness.
Loss of productivity.

The main issue with the refactoring syndrome is that is based in the beauty of the code, and this, upfront, has three big issues.

Not achievable. All code is crap. This is one of my favorite quotes, and to me is proven by the following facts:
- Exclusion principle. All programmers tend to think that only their code is the only good code in the universe.
- Legacy code principle. As a programmer, as you learn new stuff, you even consider your not so new code to be crap.
- Experience principle.I have never seen, or produced any real code which I thought it was beautiful.
Not so important. Even if beatiful code was achievable, it doesn’t prove that the code is correct, and among other things, correctness is way more important than beauty.
Wrong perspective. Focusing in the code itself, and not in the application makes the programmer forget about other more important areas.

This refactor syndrome seems to be associated with some programmers taking the last agile tendencies as religious dogmas. This happens when programmers confuse the tools and engineering practies like TDD, refactoring, pairing and unit testing… with what needs to be achieved. This would be comparable to a builder who believes that having shiny, new and clean tools, is more important than the house he is building.
It is important to understand that none of the tools or engineering process that the different agile family of processes propose are bad per se, they are usually great. Is just that when we decide to overuse them that we might be getting close to the refactoring syndrome.
In summary, is important to remember that:

It is not important how beatiful is your code if it doesn’t work.
Refactor only when is necessary, not only when you don’t like the code or you think you are going to make it more beatiful. Not liking something IS NOT a reason enough to change it.
Unit tests have very limited value when it comes to integration and end to end issues. Do real end to end test!!
Know all the different tools and engineering practices, but don’t take them as dogmas, learn what is good and bad from each and apply them when necessary.
Good programmers are gonna keep doing good software, bad programmers are gonna keep doing bad software, no matter what processes they follow.

Update

Apparently this article is causing some controversy, so I want to make clear some points which I see after the comments I didn’t made clear enough.

I do think that Unit Test is great, I just want to point out that it CAN’T and it SHOULDN’T pretend to substitute integration and end to end test from the developer.
When I say that a bad programmer is going to write bad software, no matter what process he follows, I didn’t mean to say as in, a bad programmer also will always be a bad programmer, we all have been very bad programmers when we started, and we only hope that we improve as we aquire more experience.

Solving Sudokus with Java and tree data structures

A few weeks ago I published Java and tree data structures a post where I mentioned that I would try to put together some code to handle tree data structures… Well, this is the first installment of what I hope it will become a series of articles/code that will look into creating this tree library.
This first article/code is going to be about building trees dynamically. It is going to be illustrated by an example on how to use dynamic tree building to find the solution for a Sudoku. All the source code is available for downloading in a SVN repository, the URLs are:

Dynamic tree building to find a solution

One widely used technique in computer science is to use a tree data structure to find the solution for something that needs to be solved but which solution is not obvious and requires many steps. One example could be chess, or a puzzle, a crossword, or like in this case, a sudoku.
The premise is simple, given an initial state, start building a tree of developed states from this original step until we find the state we were looking for. In the specific case of a sudoku, start building a tree where the branches are the different valid developments for that sudoku and stop when we find the solution.
The key element for building a tree dynamically is recursivity, starting from the original state, we find what the next candidate are, and them we do the same with each of them. In the mgsTrees library this can be easily accomplished by implementing an interface:

view plaincopy to clipboardprint?

public interface ExpandTreeStrategy<T> {
public ExpandAction<T> expand (T toExpand);
}

The return type of the ExpandTreeStrategy is an ExpandAction. An ExpandAction has 3 flavors.

stopBranch: It signals that this node is not expandable. In the Sudoku solver this is used when an ilegal Sudoku is reached.
stopTree: It signals that is not necessary to keep building the tree. In the sudoku solver this is used when the solution is found.
continueWith: This is used to specify what are the next states to look at. In the sudoku solver this is used when a intermediate sudoku is reached to specify what the next candidates are.

This is how to create the ExpandTreeStrategy Implementation for the Sudoku solver.

view plaincopy to clipboardprint?

public class ExpandSudokuLookingForSolutionStrategy implements ExpandTreeStrategy<Sudoku> {
private final SudokuAnaliser sudokuAnaliser;
public ExpandSudokuLookingForSolutionStrategy(SudokuAnaliser sudokuAnaliser) {
this.sudokuAnaliser = sudokuAnaliser;
}
@Override
public ExpandAction<Sudoku> expand(Sudoku toExpand) {
SudokuAnalysis sudokuAnalysis = sudokuAnaliser.analise(toExpand) ;
if (sudokuAnalysis.isSolved()) return ExpandAction.stopTree ();
if (! sudokuAnalysis.isLegal()) return ExpandAction.stopBranch ();
if (sudokuAnalysis.hasDirectSolutions()) {
return continueWithSudokuWithAllDirectSolutionsFilled(sudokuAnalysis);
} else if (sudokuAnalysis.hasNextCandidates()){
return continueWithSudokuVariantsForSquareLocationWithLessPossibleValues(sudokuAnalysis);
}else{
throw new RuntimeException("Is not possible to have a sudoku not solved, legal and without direct soluctions or candidates");
}
}
private ExpandAction<Sudoku> continueWithSudokuVariantsForSquareLocationWithLessPossibleValues(SudokuAnalysis sudokuAnalysis) {
List<SudokuMovement> movements = sudokuAnalysis.getPossibleMovementsForSquareWithLessPossibleValues();
List <Sudoku> nextSudokus = new ArrayList<Sudoku>();
for (SudokuMovement movement : movements) {
nextSudokus.add(sudokuAnalysis.getSudoku().with(movement));
}
return ExpandAction.continueWith(nextSudokus);
}
private ExpandAction<Sudoku> continueWithSudokuWithAllDirectSolutionsFilled(SudokuAnalysis sudokuAnalysis) {
List<SudokuMovement> movements = sudokuAnalysis.getAllDirectSolutionMovements();
return ExpandAction.continueWith(sudokuAnalysis.getSudoku().with(movements));
}
}

At the end the entire tree building logic is wired in the TreeFactory class, which has a method, buildTree, that takes the starting point for the tree, and which constructor takes the ExpandStrategy to be used to build the tree. The result of this call is a TreeResult, which contains three main methods;

getTree(): The resulting tree
isBuildInterrupted(): Will tell you if the build is interrupted, in the case of the SudokuSolver this would mean that the solution was found.
getLastBranch(): A sequential ordered list of all the last branch nodes startging from the root node, the original node. In the case of the sudoku solver this is used to retrieve in case that the solution is found (isBuildInterrupted == true) the sequence of Sudokus from the original one to the solution.

view plaincopy to clipboardprint?

public class TreeFactory<T> {
private final ExpandTreeStrategy<T> expandTreeStrategy;
public TreeFactory(ExpandTreeStrategy<T> expandTreeStrategy) {
this.expandTreeStrategy = expandTreeStrategy;
}
public TreeResult<T> buildTree (T seed){
System.out.println(seed);
ExpandAction<T> expandActionResult = expandTreeStrategy.expand (seed);
if (expandActionResult.isStopBranch ()) return TreeResult.stopBranch (seed);
if (expandActionResult.isStopTree ()) return TreeResult.stopTree (seed);
Node<T> rootNode = new Node<T> (seed);
List<Node<T>> currentBranch = new ArrayList<Node<T>>();
currentBranch.add(rootNode);
Iterator<T> childIterator = expandActionResult.getChildIterator();
TreeResult<T> lastSubTreeResult = null;
while (childIterator.hasNext()){
Node<T> next = new Node<T> (childIterator.next());
lastSubTreeResult = buildTree(next.getContent());
rootNode.addChild(lastSubTreeResult.getTree().getRootNode());
if (lastSubTreeResult.isBuildInterrupted()) {
currentBranch.addAll(lastSubTreeResult.getLastNodeBranch());
return TreeResult.interrupt (rootNode, currentBranch);
}
}
currentBranch.addAll(lastSubTreeResult.getLastNodeBranch());
return TreeResult.buildCompleted (rootNode, currentBranch);
}
}

The final code to solve any given Sudoku will look like:

view plaincopy to clipboardprint?

public static void main(String[] args) {
Sudoku sudoku = new Sudoku(
new SquareValue [][]{
{__, __, __, __, __, __, _6, __, __},
{__, __, _3, _7, __, __, __, __, _1},
{__, _8, __, _1, __, _6, _9, _4, __},
{_7, __, __, _5, __, __, __, __, __},
{__, __, _5, __, _4, __, _8, __, __},
{__, __, __, __, __, _2, __, __, _7},
{__, _2, _9, _6, __, _4, __, _3, __},
{_6, __, __, __, __, _8, _4, __, __},
{__, __, _4, __, __, __, __, __, __}
}
);
IteratorStrategy rowNavigator = OneLineIteratorStrategy.horizontal();
IteratorStrategy columnNavigator = OneLineIteratorStrategy.vertical();
IteratorStrategy matrixNavigator = MatrixIteratorStrategy.instance();
SudokuProcessor sudokuProcessor = new SudokuProcessorImpl(rowNavigator, columnNavigator, matrixNavigator);
SudokuAnaliser sudokuAnaliser = new SudokuAnaliserImpl(sudokuProcessor );
ExpandTreeStrategy<Sudoku> expandTreeStrategy = new ExpandSudokuLookingForSolutionStrategy(sudokuAnaliser);
TreeFactory<Sudoku> treeFactory = new TreeFactory<Sudoku>(expandTreeStrategy);
TreeResult<Sudoku> buildTree = treeFactory.buildTree(sudoku);
List<Sudoku> lastBranch = buildTree.getLastBranch();
System.out.println("Solution: " + lastBranch.get(lastBranch.size()-1));
}

Linktocse