Here are my notes from the agility panel at today's user conference.
Panelists:
- Marc Strohlein, chief agility officer, Outsell (moderator)
- Larry Tunks, CIO, Congressional Quarterly
- James Wonder, director of emerging technology, American Institute of Physics
- Alex Humphreys, director of business technology services, Oxford University Press
- Kurt Cagle, managing editor, XML.COM
About 30-40% of the audience said they’ve tried
agile methodologies. A lesser number characterized their companies' culture as agile.
Company culture must enable agility; it must come from the top down. There must be a willingness to make mistakes and learn from them. Hard in blame-oriented culture.
There is a fear of lack of control with agile. How can you be flexible and have control? How can you mitigate that far?
What should you do if you want agility and the company culture doesn't support it? Quit? Disagree with the notion that agile culture needs to come top-down. It's about how people work together. It's a hurdle that you go from a predictable process to a less predictable one. Way to overcome is to be clear about what that process is. Communication is key.
Trust is important in agile. It comes from having a cohesive team and a well defined vision. Trust gets built as well. Be sure your first agile project works as that will start the cycle of building trust. Trust needs to be built. Where is it built already? Can you put a core of people together and build around that?
Work hard to break down silos in the company. Programming staff speaks in the language of the newsroom, not the language of technology. Less structure is more intimidating; what is my role; how am I defined here?
It's impossible to over-stress the issue of trust. Agile is predicated on the notion that software development is unpredictable. Things are unpredictable. Everyone involved has to trust that everyone involved is doing what they need to be doing all the time. Agile can fail when there is a pre-existing system predicated on adversarial relationships. Hard issue when dealing with outside service providers -- i.e., how much is this going to cost? Should a consultancy say "I don't know" (the agile answer) or $247,000. Which is the better answer?
Can get around this by adopting outsiders on the team and treating as part of the team (i.e., don't do traditional fixed-bid consultancy but more time and materials on join-the-team basis).
Mentoring is an important part of this, comes from
XP (extreme programming). Mentoring isn't just having two guys look over each others' shoulders. Mentor should act as filter and guide to the person who's new to the organization. Tap the mentor to get the person either to a point where they're comfortable or to a place where we say "Joe's not cutting it." Need a way to establish trust for people coming into the organization at any given point. This works way better than
team-building management exercises where we go build bridges in the rain.
Agile needs to take into account business realities. These businesses need to make money. Getting business people to buy into this notion that the budget is either $250K or $5M isn't going to fly. What will fly is delivering projects. (Sense the agile means unpredictability and that's good but you can't go so far as to put your hands in the air and never commit to anything.)
Given a potential history of IT over-promising and under-delivering how do you get senior management to buy in? Start on per-project, per-team basis. Make some waterfall-y concessions where necessary. Have a design and development contract to make legal happy. Allow for the fact that it won't be a model of agility but there is some benefit in acknowledging that we don't know everything up front.
Agile tends to work differently on the XML front. In imperative development, you're building incrementally, component by component. With XML, you're having to shift away from building components and move closer to the idea that we are designing incrementally what we are trying to do. With XML the design part is often the most difficult part of the process. In normal databases, you have large numbers of tables, that using created along the way as part of the development process. With XML, you say I have the semantics of the workflow worked out. With XML, the workflow tends to be very much oriented to publishing and moving entities around, regardless of what those entities are.
Isn't it
waterfall that requires trust and a humongous leap of faith? I'm going to make a big spec and leave me along for 12 months and I'll deliver --
that's trust! Isn't agile really about less trust -- constant iteration, constant guidance, working together on trade-offs between resourcing, funding, and requirements.
Yes, waterfall requires trust, but when you move to a cross-functional team with less well-defined interactions, where people swap hats, the marketing people need to trust that the development people are doing that for the right reasons, that they're not trying to encroach or take anything away from there. Great thing about agile is there are no six-month black holes.
Needs to come down from a conceptual discussion to a real business problem that needs to be solved.
The chasm between waterfall and agile doesn't really exist. Agile is the reflection of what has been happening over the past several years, contracting modularization of content, workload, expectations, and requirements. Don't forget that the reality of the development process has changed. 15 years ago, it really did take 2 years to build some apps. With the new reality of the Internet, with services orientation, you don't need to build all the infrastructure. Your role as a development changes from from-scratch builder to integrator. Development cycles themselves have been contracting dramatically.
What's happening to the science of information management, which should also be a part of the development of content applications?
Consider open source development. Two things are happening -- distributed open source teams just can't do waterfall and the level of abstraction of programming has gone way, way up. You now have non-programmers in some cases working on these products and systems. You can get stuff done fast in Jango and Rails, but how do you make sure you're not rebuilding the wheel every time because it's over-componentized. In effect, today, we're moving back to the centralized computing of the 1990s.
The biggest benefit of agile is the simplicity. You can communicate. You can see, touch, and feel it. Build up on it. Rather than over-design something and then whittle back in an adversarial manner.
Check out
Bob Glushko and his work / book on
document engineering.
I've seen agile work and I've seen it fail miserably. Two things I've seen. When it works, there is a long-term roadmap. Sometimes, marketing says need X product by Y date. Then development says, I guess that's what we need to do. Any product needs a given set of features to be successful. With agile, you think of those features on a progressive scale. You can see a lot of waterfall in agile.
Yes, it goes back to the roadmap. It's easier to develop a mature product with agile than a brand new product. What's needed is a good common understanding of what's on that roadmap. As you're creating XML you have to have some idea of what you're going to do with it, even if you're going to change it, or you won't define a structure or schema that's helpful.
XML is ultimately about data modeling, and to a lesser extent about process modeling. It's the
Tao of the programming world. Imperative programming is "go." XML programming is "be." If you ask a programmer how long it will take to write a program, they will tell you any number that comes into their mind. Realistically, until they know what the model is, until then, you don't know what you're building. That model becomes your roadmap. One problem we create is to conflate the process with what we're trying to model. Just
as in building a house, until we understand it, it doesn't matter what methodology we're using. Need to get to declarative, be-full modeling constructs.
Understand your model. Understand what you're trying to do.
Two years from now it will be great to see the impact of Vista on this whole debate. Won't be the first time that a Microsoft initiative changed the corporate approach to development.
"I like agile projects because they're fun." Marc Strohlein. That says it all!