Initially, agile techniques were targeted only at small, co-located teams usually, but now larger teams that are geographically distributed have adopted agile. How has that change come about and does distributed agile really work?

Siddharta: Right, so agile always talks about co-located teams and even now most people agree that co-located teams are always going to be more productive than distributed teams. But there are a whole lot of other things which can decide to make you go in for a distributed team – for example availability of skill sets or strategic decisions by the company. There are many things which tell you, you need to go in for a distributed agile project. We’ve seen over time, distributed agile has been becoming bigger and bigger. In fact, there are a lot of things being distributed.

Jeff Sutherland, one of the scrum founders, talked about teams in the US and Russia doing distributed agile and how productive they are. So there are a lot of people who are really succeeding in doing distributing agile. But the thing is to succeed in distributed agile – you need to make a few changes. Obviously, communication is not going to be as good as if everyone is co-located; similar bunch of practices which ensures that you try to maximize the amount of communication available, it might be a shorter iteration so that you can get further feedback.

You try to split groups not amongst teams – so you don’t want to have developers in one location and testers in another. But along functions or bits of functionality. So you might have developers and testers working on one part of the system in one location, and developers and testers working on another part of the system in another location. And you want to maximize the amount of communication bandwidth that you’re going to build through chat, voice messaging or through cameras and things like that. If you make these kinds of changes you can succeed. I know a number of companies, not just out of India, who are doing distributed agile and are successful.

Selina: Ok. So if you look at it, the nay-sayers of agile say that agile practices don’t give too much important to design, or the design aspects. But design is vital to a product’s long-term success. What do you have to say on this?

Siddharta: This is something that you hear very often – that agile teams don’t do design, there’s a similar thing that agile teams don’t do documentation. But these are kind of misconceptions. A lot of people understand agile to be coding without design, without documentation and equate it to adhoc development. And nothing could be further from the truth.

We need to make a slight distinction between not doing design and not doing Upfront design. So the term used in agile circles is Big Up-Front Design (BUFD). What agile does not do is to get all the details upfront and do a BUFD. So we don’t have a design phase like your typical but agile teams do do design.

As and when they keep incrementing features they revisit the design. And design is something that you look at throughout the project, it’s not technically something at the beginning alone. At the end of every iteration, it is re-evaluated; re-factoring is done constantly to improve the design. This is something which happens continuously. So agile does continuous design and evolving design, but they don’t do BUFD.

So it’s not to say that agile doesn’t do design, in fact it does it right through out the project development. Constant evaluation and redesign.

Selina: So how is design planned? In every sprint or whenever it’s required, you take a look at it?

Siddharta: Whenever someone is recommending a feature, they think about whether it fits in the existing design or not. If it doesn’t fit in, then assuming you are all co-located, you can take a few people and decide these are the kind of changes you need to make and then go ahead and implement them. So it happens on a day-to-day basis, week-to-week, sprint-to-sprint basis.

Read the rest of the interview here or listen to the audio here

Agile has its own advantages and people who vouch for it. So does a completely different way of doing things – the CMM (Capability Maturity Model). What are the compelling reasons for a person who is practicing CMM processes to move to agile?

Siddharta: I’d say there’s only one compelling reason – and that is, if what you are doing currently is not working for you. If you’re following CMMI and it is working for you, that’s great, then there is really no need to change. You don’t have to change because it’s the in-thing. That’s something which I’m quite against.

But a lot of people do have problems when it comes to CMMI when requirements are unstable, because of the way it is structured, and with testing and user acceptance right at the end, which creates a lot of issues with respect to changing requirements. Or cases where you’ve not got the requirements exactly or there’s also the case where the customer sees a product and then gets lots of ideas on how it can be done. So when you have a market like this, then you find that CMMI tends to cause issues because you get bugs right in the end, you get change requests right in the end. It can be difficult to cope with it.

Whereas agile is perfectly suited for those kinds of project. You make frequent releases, there is a lot of feedback is involved, there is a lot of collaboration involved. So in these cases, agile is well suited for these kinds of projects. If you’re doing what you’re doing, and its working then fine, I’d say continue with it. But if it’s not working, then agile could be an alternative.

Right, but is it true that agile is more applicable for consumer kind of applications and products compared to enterprise-class systems?

Siddharta: No, I wouldn’t say it’s true. In fact it’s kind of interesting because where agile originated from – if you look at it – a lot of it is from enterprise applications. If you look at extreme programming, it developed out of a project at Chrysler. If you look at feature-driven development, it came out of a project in Singapore and so all these projects are enterprise projects. The history of agile is actually quite the opposite – it’s come out of an enterprise background, where if you look at it, in an enterprise setting – requirements can change often and things like that.

Now of course, if you really think about agile, it started a lot before Web 2.0 came into the scene. So that’s the background from which it has come and it just so happens, now that we talk about web 2.0, it turns out that agile its perfect for those kind of applications as well because you have releases going out very quickly. But I wouldn’t say that that’s the only type of project that it is suitable for because the history of agile is really coming out from the enterprise group. Even today if you look at it, most agile projects are Java oriented enterprise projects.

That’s a very interesting take. Can you outline some of the biggest challenges you see when people try to make the transition to Agile?

Siddharta: Yeah, there are so many. The number one is getting into a mindset and cultural change. You know, agile is not really about processes if you think about it. What agile promotes is a set of values and process is something which just comes out of a value and that’s why we have so many different processes that can all call themselves agile.

The values we are talking about is like, for example, collaboration between customer and developers and management and testers and so on, we are talking about self-management, about being adaptable. These kinds of values are quite the opposite of what traditional project managers have been used to. They are used to plan-driven projects, they are used to command & control hierarchies, they are used to contract negotiations with the customer. So coming out of this mindset and coming into the culture of agile – that’s the biggest stumbling block.

We see a lot of people who try to adopt agile without changing the values and it runs into some dysfunction or the other. For example, instead of having a self-organizing team, they do iterations but the manager tells everybody what to do. So they think they are doing agile but actually speaking I wouldn’t really call it agile. So you come across these kind of dysfunctions because the cultures as a mind-set change has not been done and that’s really the number one issue. Once you do the mind-set change then you can pick on any number of practices to do what ever you need to do but if you do practices without doing the mind-set change, you could be in some trouble.

Hence having a product roadmap even when following an Agile approach is a must. A
product roadmap is a planned approach that helps with strategic project planning and communication of that plan with respect to product releases, functionality listing etc. But it should be formulated by first understanding the target users, the market, and the underlying technologies. Product roadmap forms an integral part of any product strategy and provide the framework for plan changes and the impact they would have on the product. It’s not just about the specific features or functionality of the product, but the long term product vision/goal of how far one would go with it.

The general perception today is that in Agile, unlike the waterfall model, it is difficult to chart out and manage an effective long –term product roadmap. However, Agile reflects the reality better because the planning is done only for the current items and the immediate iterations that may follow. And this necessarily avoids one from making a long-term product goal (say 6-10 months ahead), which may fail because priorities/requirements of the products often change. Moreover, making a long range roadmap has always been a real challenge in general.

In Agile, the roadmap and approximate dates are calculated based on the product backlog and the knowledge of the team velocity. Many a time over-commitment to the customers has been a major issue here. But product managers can certainly employ a centralized system that would ascertain development team velocity, all previously committed requests, and based on that, calculates the next available commit date for a task of importance. This always keeps one in check. A good agile product roadmap should invariably help one deliver the right products with the right features at the right time to the right customers.

Read the entire digest contents

Many in the enterprise software community assume that Agile techniques are targeted only at small, co-located teams of about ten people. This assumption makes sense to a certain extent as time zone differences, language barriers, cultural differences, and lack of synchronization of development processes make implementation of Agile techniques in offshore teams a great challenge. However, these challenges can be overcome by reinforcing the following practices:

Rapid and Frequent Communication : One of the most important principles of Agile is active and regular communication between all teams involved in getting the software to market. So, if there is one or more offshore teams involved, a daily status update informing the other teams of the day’s progress (what they did that day, what they are the planning to do the next day and the problems faced) can be done by making some adjustments for the time zone difference. Instead of the entire team attending the stand-up meetings, every team can have a local stand-up meeting and the team lead alone can convey it to the other team(s). Weekly status meeting with the entire team and face-to-face meetings whenever possible (so you need to budget for that) should also be given priority. All this will ensure that team members are on the same plane and project is progressing as desired. Besides teleconferencing, other collaborative communication methods should also be considered and made use of as appropriate, including video conferencing, wikis, instant messaging, VoIP solutions etc.

Automated Builds and Continuous Integration: The practice of automating builds at regular intervals is advised as several teams develop simultaneously and since the team members are geographically distributed, it becomes a massive task to ensure that each change is integrated into the build. So, if there is continuous integration, any integration issues can be sorted out immediately rather than solving huge issues at the end. An effective integration should include practices like compiling, database integration, unit testing, deployment, functional testing, notifications, and reporting.

Short Iterations: Product requirements keep changing due to a better understanding of what the market really needs. Hence, short iterations that last from one week to four weeks, with working software at the end of each iteration, is advisable. In the case of distributed teams working on different functional areas, having a demo at the end of the iteration will help clarify whether offshore teams have really understood the requirements communicated to them, instead of waiting for months to see the end results.

Efficient Team Organization: Getting the entire team together at the beginning of the project results in everyone getting to interact with the other team members and can result in better camaraderie and collaboration. Project managers should involve distributed team members to discuss release dates, associated costs, and risks involved in the project. Team members at different locations sharing the responsibility of different functional areas (instead of owning different phases of the project) and modules, and collective code ownership can foster a strong sense of pride and ownership among the whole team. Cross functional teams and Scrum masters present at all major team locations also helps to a great extent.

Read the entire digest contents

As Agile teams manage their work differently, they typically need a different set of tools to support their agile approaches. Simple and adhoc tools like index cards and spreadsheets for requirements management and prioritization, wikis for collaborative documentation, commercial or opensource project management and bug tracking tools are usually sufficient for small, co-located teams. However, when teams and projects get larger and Agile methods need to be scaled up, integrated tools become useful.

New agile project and product management tools enable better coordination and collaboration between the various teams involved in the agile process (development, customer support, marketing, management, customers, etc) and also provide better reporting capabilities. Some of these tools even include requirements management, test and defect management and integration with development, test and build environments. They can bring about further improvement and acceleration of existing agile processes.

X-planner, RallyDev, VersionOne, TargetProcess, Yoxel and Mingle are some available Agile project management tools. .NET shops can also consider using MS team SCRUM plug-in for its agile project management functionalities. Some of these tools require you to spend time learning to use it initially while others allow easy integration with existing tools. Finding and using the tools that are well-integrated, facilitate meaningful interaction between your different teams and give more visibility can ultimately help you produce better software faster.

Read the entire digest contents

There are dozens of Agile methods, a detailed list of which can be found at Wikipedia and the Agile alliance. Interestingly, even these two lists are not identical and there is on-going debate on which processes can be considered agile, and almost everyone adapts and implements the same agile processes differently. After all, you cannot use the same agile approach with a team of six and a team of six hundred. The difference between building a data warehouse and an online ordering system on the other hand may be more subtle and harder for stakeholders to understand that it potentially requires different agile approaches.

So choosing the right agile method depends on the type and size of your product/ project, the criticality of the project, the location of the team members, and even the technology that is being used.

To get started with Agile, teams should equip themselves with knowledge about the different agile methodologies and the practices that form a part of these methodologies. Attending conferences and workshops where practical examples and best practices are shared is also encouraged.

What’s important next is to figure out which practices to adopt based on how it would address your requirement and what is the business value that can be derived from it. For example, if your goal is to reduce time-to-market, you can consider the agile practices of iterations, iteration backlogs and stand-up meetings. If increasing quality is the primary goal, automated developer tests, pair programming, user stories and continuous integration can work well for you.

There can be a lot of overlap between the various Agile processes and many successful practitioners adapt and adopt practices on an individual basis. Trying a pilot project can give teams a chance to understand and adapt to agile practices, make mistakes and learn from them. Also, instead of a big-bang approach to agile, a gradual transition (say from 9 to 6 to a 3 month release cycle) can cause less confusion and allow the teams involved to better adapt to agile.

Read the entire digest contents

Before considering transitioning to SaaS however, ISVs should accept the fact that making the move to on-demand will have a profound impact on every aspect of the business: including pricing, delivery, sales channels, product architecture, support and infrastructure.

The architectural changes involved in getting your product SaaS-ready can be quite complex, hence the engineering aspects of building a SaaS product should not be overlooked. On-demand products need to have user customization designed-in and remote development capabilities. Also, as SaaS allows for instant upgrades to be done across the customer base, it would typically follow agile or iterative methodologies.

Adopting agile or iterative methodologies in turn ensures that your product will not be over-engineered and will have just enough features for you to go live with just enough technology. The most important aspect of any SaaS architecture will be allowing for reusability, extensibility, availability, and a service oriented approach.

One limitation of on-demand software is that it is configurable by end-users for their individual needs, but there is a single code base for all customers (or tenants) which is not customizable for any one customer. Because of this, competitive advantage or differentiation based on the software itself becomes negligible.

However, SaaS vendors have woken up to this and have developed platforms that can accommodate customizations. This rich customization allows SaaS vendors to serve larger corporations, in addition to their typical customer base of small and mid-sized companies. All of this and a few other things promise to change the SaaS landscape soon – the next wave of SaaS is coming!

Read the entire digest contents

It’s a fact that testing in an Agile environment is different from testing in the Waterfall or other development approaches – so where do traditional test automation tools fit in? Although test automation specialists may be comfortable using their chosen traditional test automation tool, these tools may not be the best bet when it comes to Agile test automation.

Some of the reasons for the mis-match between traditional test automation tools and Agile testing are: these tools encourage the test-last workflow, which cannot work for agile teams; the scripts that traditional tools create are typically dependent on various sources of information and are therefore not easily maintainable; traditional automation tools are expensive and learnt exclusively by the automation specialist as a QA tool, which will not work in an integrated agile testing team.

However, this does not mean that Automation tools cannot be used in an Agile environment. A new breed of test automation tools like Fitnesse, Green Pepper, StoryTestIQ, Selenium, Watir and others are more optimized for Agile teams and solve the challenges of collaboration, reducing waste and increasing the speed of feedback that Agile teams face.

Also, just as in any test automation effort, the key thing is to find areas/scenarios that would benefit the most from automation. Judicially applying test automation efforts to those areas will enable one to reap the most benefit!

Read the entire digest contents

Many teams that have adopted agile tend to leave out other key roles (for instance business users or testers) out of, or on the fringes, of the agile team. With testing becoming an integrated activity and all members on agile teams writing their own unit tests, it is easy to think that testing as a role is no more significant in an agile world.

However, although the role of testers will change considerably in agile development, the importance of having professional testers in agile teams should never be under-estimated. Rather than doing pure testing, agile testers take the role of Test Analysts and perform quality assurance for the team. They bring in a different perspective, help the development team produce testable code and play an integral part of the continuous feedback loop that keeps the team on track.

All of this in no way implies that agile testing is easy. In fact, Agile testing calls for more judgement from a tester: it requires more expertise about what’s good and what’s not, the ability to be more flexible and having the confidence to work from your knowledge and intuition of what is good. It’s certainly not just a case of following a test script and making sure the software does what it says in the spec.

Agile testing is therefore certainly not for dummies!

Read the entire digest contents

1) Continuous Integration (CI) : Whether you are following a Dedicated or Time & Material or Fixed-Bid type of engagement, CI is a very useful and proven Agile tenet. The Root Cause Analysis of the issues that come up during the last stages of a typical release will point to Integration issues as one of the root causes.
There are tools around which can be used for automatically building and deploying your code at very frequent intervals (or on a need basis) so that the integration issues are closed as and when they are found, instead of accumulating till the end.
One good example is CruiseControl, which has versions for Java as well as Dotnet environments. You could visit the thoughtworks site for more info on CruiseControl.

2) Pair programming: Pair programming is an agile tenet which advocates having a pair of programmers work together on all coding aspects, instead of working individually. Compare this with a typical scenario where the programmers work individually for all
their work. 100% pair programming is a controversial topic, and we will not get into that debate in this post. But what is not controversial, and indeed beneficial, is to use pair programming in a contextual manner. That is, whenever you are faced with a difficult problem, have 2 people work on it. That has proven to be much more productive than the aggregate work done by them separately. Also, there is a knowledge backup within the team for all the difficult sections within the codebase.

3) Small releases: Have an overall fixed-bid contract which allows about 20% deviation in the scope. Then have smaller iterations where small chunks of feature requirements will be identified, zoomed-in, designed, implemented, tested, and be ready to be deployed.

In other words, it is very common to find the fixed-bid billing model tied to the waterfall methodology. There are inherent problems with the traditional waterfall software developement life cycle (SDLC) methodology, and it is better not to mix waterfall and fixed-bid. Pictorially it can be represented this way:

Requirements->Design->Coding->Testing->UAT->Golive

Feature set #1—>Feature set #2 —>Feature set #3
Where, the first line indicates the typical approach using Waterfall model the second line indicates the approach using smaller closed iterations (Agile model), with the various phases of an sdlc compressed into each iteration.

The main difference is that in the waterfall model, the compartmentalization is in terms of phases, whereas in Agile, the compartmentalization is in terms of features—ie, in Agile all the phases will be implemented for each set of features in the order of decreasing priority of features. We will talk more about the mechanics of this, in a future post.
In summary, some of the Agile tenets can indeed be used in a Fixed bid model, actually reducing the heartburn for all the stakeholders in the client’s as well as the vendor’s side.