I recently heard this question:
"Is too much focus on testing benefits a bad thing overall?"
TDD done well can improve readability. TDD done poorly, that is without consideration of other important principles, can reduce readability.
A guy I worked with in the mid-90s would say "You can always make a system more flexible by adding a layer of indirection. You can always make a system simpler by removing a layer of indirection." Both flexibility and simplicity are important qualities of a system. The two principles can often live together in harmony, but often they work against each other. If you go too far towards one extreme or the other, you move away from the ideal that exists where these two principles are balanced.
TDD is partly about testing, partly about design. TDD done poorly can tend too much towards either flexibility or simplicity. It can push towards too much flexibility. The objects become more testable, and often simpler, but the inherent complexity of the domain problem then is pushed out of the objects into the interaction of the objects. We gained flexibility, and to the naïve eye, it can look as though we've gained simplicity because our objects are simpler. The complexity, however, is still there. It's moved out of the objects, and into the object interaction, where it's harder to control. There are code smells that can act as red flags here - a system with hundreds of small objects and no larger objects is one, lots of objects with only one-line methods is another.
TDD done poorly can move in the other direction as well, that is, towards too much simplicity. So, we do TDD by writing the test first, but it has little impact on our design. We still have long methods and huge objects, and those are code smells that can red-flag this problem.
Now TDD will not by its nature knock you off-balance in either direction, provided it's well-applied. Use other practices to keep you on track. For example, draw pictures of what you're doing before you do it. Obviously, not all the time. Some things are far too simple for that. Some pictures are worth saving, some are just sketches that help us to visualize the problem, and we are, by varying degrees, mostly visual learners. If you can't draw a picture of the problem, you don't understand it.
How will this help with TDD? It will help to keep a system from going too far on the flexibility side, away from the simplicity side. If you draw a picture and it's ugly, that's a red flag. Sometimes it's necessary, but often when you draw the picture, your mind will quickly see things that can be simplified. The solution becomes more elegant and simplified, easier to maintain, and more enjoyable to work on. If you can't or won't draw pictures of your system, you're losing this opportunity to make your software more solid, more elegant, more beautiful to see and easier to maintain.
Applying this comes with experience, and some coders will never understand the value that a good balance provides. There's no metric that you can run that tells you you're in the right place. If someone gives you a prescribed method to arrive at that harmonious point, he's lying to you. More importantly, he's probably lying to himself without realizing it.
So, my answer to his question is 'yes': test everything without forgetting the other good principles.
Any good practice will throw you off-course if it's not balanced with other good practices.
Sunday, May 24, 2009
Saturday, May 23, 2009
Creating A Remote Branch With Git
If you're still doing this the way I suggested. Stop. Stop right now. Back away from the keyboard, and nobody gets hurt. Also, read the comments below, and do it the way Graham shows.
Original post:
----------------------------------------------------------------------
Assume that you're in a clone of a remote repository. We'll create a remote branch named "mybranch." First create the branch locally:
>git branch mybranch
Switch to that branch:
>git checkout mybranch
Switched to branch "mybranch"
At this point, list all your branches:
>git branch -a
master
* mybranch
origin/HEAD
origin/master
Now, here's the command that creates the remote branch:
>git push --all
Total 0 (delta 0), reused 0 (delta 0)
To /tmp/remote.git
* [new branch] mybranch -> mybranch
It's just that easy. Now, in another clone on another machine, I can self-collaborate by creating a local branch that mirrors the remote branch. First, list all the branches:
>git branch -a
* master
origin/HEAD
origin/master
origin/mybranch
And create a local branch that tracks with the remote branch:
>git checkout -tb mybranch origin/mybranch
Branch mybranch set up to track remote branch refs/remotes/origin/mybranch.
Switched to a new branch "mybranch"
List the branches again to see the change:
>git branch -a
master
* mybranch
origin/HEAD
origin/master
origin/mybranch
Run gitk to see visually that your local branch is tracking the remote branch. Use git push and pull as you change machines to move the changes around. When the branch is done, you can merge it.
I like git. I tried bazaar and mercurial, but git "just works" and is easier to get started with than bazaar. Plus, there's github. Visit me there at https://github.com/DonBranson.
Wednesday, May 20, 2009
DonsProxy on github
I've been thinking about getting DonsProxy out on github for a bit now, but needed to break out the good stuff from the slough. So, it's finally out there. My next DonsProxy task is to get the SSL part working better - maybe by injecting Mina. Still trying some things out. You can clone DonsProxy from github at http://github.com/DonBranson/DonsProxy/
Thursday, May 7, 2009
Favorite Project Series - Datagate (14 Years of SOA)
SOA is one of the big buzzwords these days, and a lot of people have started doing SOA for the first time in the past few years. I have some good memories of the "good old days of SOA," and now it seems like everybody's using the groundwork that was laid in the 80s and 90s. Now, people can just go download Weblogic and Systinet or JBoss, and and the various pieces, build a system using those frameworks, and they're "doing SOA." There are so many powerful tools; the IDEs such as Eclipse and Intellij do a lot of the tedious work for you.
My first introduction to SOA was in 1995 on a project called Datagate at Southwestern Bell. I came on the project when it was about six months old. All of the things that people take for granted now, that they use and it "just works" (or not), we had to build our own. There was no app engine, no UDDI, no SOAP. The things that people download and use today, we built and used back then. If you're a computer geek, this is the project of a lifetime. It was all message-based (that's a MOM for all you acronymophiles). Now I can look around and see SOA everywhere, but we were, in our own way, pioneers of the technology, because we did things no one had ever done before. Sure, not everything we did was new - Tuxedo had been around, so MOMs weren't exactly new.
What we did in the mid-90s that was new was to make the technology more accessible. It was hard to write Tuxedo services back then. We made it accessible to every C, VB, and PowerBuilder coder out there. There are some ways in which today's technology is better. But, what we built was so smokin' fast, and there's nothing that SOA folks are doing today that even comes close to the performance we delivered on those old, slow machines. We had a great, usable middleware product that was the framework for developers to construct clients and reusable services on a variety of platforms - about 12 unixes, MVS running on IBM 390s, Windows 3.0, Windows NT, and Tandem. It's one of those experiences that you looked back on, and think, we did it before most anybody else. What people today use, we built.
I wasn't one of the visionaries for that group. I was brought in to develop an interface so that VB and PB programmers could write clients on top of our C framework. In the end, I help design and architect that, the Directory Service Replicator, and the Dashboard for this system, and we did good work.
There was no SOAP back then. The Datagate team developed its own protocol. It was message-based, so was much more performant, scalable, and simpler than the RPC-style calls are today. Yes, even the message-driven services are RPC under the covers, not the true MOMs like what we built. A true MOM lets you fire off multiple messages to services without waiting for the ack that the message was received, then process the responses as they come back in. All this with one thread, not multiple threads that RPC pushes you towards for this kind of processing.
There was no UDDI then for service lookup back then. There was no LDAP, either. We built our own distributed, replicated X.500 directory service. Infrastruture and business services registered and reserved leases dynamically. If a service went down, its service entry went away, and clients would call one of the other instances that was still registered. And of course, it was fast - 10s of millis to do a service lookup, not 100s or 1000s like you see with UDDI. Smokin' fast.
I architected and led the development of the Directory Replication Service along with my friend Dave C. This cell-based, scalable solution allowed us to run multiple Directory Services, and replicate data between them in a scalable, available fashion.
We used XDR for marshalling. That's one piece we did download. I lead the charge to unit test the whole thing - and found only one bug. There was an endian issue on that affected one of the platforms, and we found it and fixed it.
There were no application engines. What we built was a resource manager that would control service lifecycle and heartbeat services to quickly detect outages.
There was no PKI, or even SSL. We bought licenses for the encryption libraries, then designed and built our own secure message protocol (with mutual authentication where needed), plus the infrastructure to support it. Certificate Revocation Service, Certificate Authentication Service, and al the APIs for services to use to load their certs, encrypt their messages. We built the services and the GUIs so that administrators could manage the PKI we built.
There was monitoring software available - Tivoli - but it was far more than our budget would allow. We built our own Dashboard that allowed us to closely monitor infrastructure and business services running over much of the United States, including Missouri, Texas, California, and more.
We did training to teach developers how to build reusable business in C, and how to build clients in C, in VB, and in PowerBuilder. I taught the VB and PowerBuilder classes.
Our team consisted of three subteams - Infrastructure, which was the team I was on, the Service Writers team, and System Adminstration, which handled the care and feeding of production systems.
At A.G. Edwards, we leveraged SOA frameworks for 6-1/2 years as part of a broker workstation, to develop the pretty scalable agedwards.com (which, last I heard some years back, had about 300,000 users signed up). I was a member of the Lead Architect Team that designed and developed agedwards.com. We used SOA for some other pretty cool stuff. One of my favorites was the BLServer, that used a modified version of what is now called the "Competing Consumers" pattern in a highly-available cluster of message-based services that self-allocated using leases to process roughly 1.5 million messages a day for about 4-1/2 years before it was mothballed in February of this year.
The industry has come a long way since then. Like I said, what we built, you can now go download, and the SOA business has experience explosive growth. After 2-1/2 years at Bell on the Datagate project, I did 6-1/2 years of SOA work at A.G. Edwards, continuing to architect systems using new SOA technologies as they emerged - Tengah, (now Weblogic) Dynamo, Novell's LDAP server, SiteMinder, WSDs, Big/IP, and various JMS providers. Some of this stuff I had more exposure to than others, but that gives you an idea of how things have changed over the past 14 years. Now there's CXF and JAXB and other exciting new tools coming along.
Every Tom, Dick and Harry does SOA now. Years after that work at Southwestern Bell, a small company hired a "SOA expert" (after I was already there!). This poor fellow didn't even know what LDAP is. When I described how it could be used for service lookup, he said you can't use it for that - it's for authentication. I explained to him that it's just a database that's optimized for reads. You can use it for anything that falls into that category. Now, he's an expert, downloading what other people have built and putting it all together. One boss at that company tried to get me to join a project where I would have been writing architecture documents all day for six months, and building nothing. His angle? That the SOA experience would look good on my resume. I kid you not.
On Datagate, we built SOA sooner, faster, more scalable, and at least in some ways better than what exists today. The primary advantage I see in today's world is standards. Almost everything is standards-based, and that means vendors can provide tools to do many of the things we used vi for. It's a change for the better, in most every way, because SOA is much more accessible today that it was in 1995. Plus, now it's an acronym. ;)
My first introduction to SOA was in 1995 on a project called Datagate at Southwestern Bell. I came on the project when it was about six months old. All of the things that people take for granted now, that they use and it "just works" (or not), we had to build our own. There was no app engine, no UDDI, no SOAP. The things that people download and use today, we built and used back then. If you're a computer geek, this is the project of a lifetime. It was all message-based (that's a MOM for all you acronymophiles). Now I can look around and see SOA everywhere, but we were, in our own way, pioneers of the technology, because we did things no one had ever done before. Sure, not everything we did was new - Tuxedo had been around, so MOMs weren't exactly new.
What we did in the mid-90s that was new was to make the technology more accessible. It was hard to write Tuxedo services back then. We made it accessible to every C, VB, and PowerBuilder coder out there. There are some ways in which today's technology is better. But, what we built was so smokin' fast, and there's nothing that SOA folks are doing today that even comes close to the performance we delivered on those old, slow machines. We had a great, usable middleware product that was the framework for developers to construct clients and reusable services on a variety of platforms - about 12 unixes, MVS running on IBM 390s, Windows 3.0, Windows NT, and Tandem. It's one of those experiences that you looked back on, and think, we did it before most anybody else. What people today use, we built.
I wasn't one of the visionaries for that group. I was brought in to develop an interface so that VB and PB programmers could write clients on top of our C framework. In the end, I help design and architect that, the Directory Service Replicator, and the Dashboard for this system, and we did good work.
There was no SOAP back then. The Datagate team developed its own protocol. It was message-based, so was much more performant, scalable, and simpler than the RPC-style calls are today. Yes, even the message-driven services are RPC under the covers, not the true MOMs like what we built. A true MOM lets you fire off multiple messages to services without waiting for the ack that the message was received, then process the responses as they come back in. All this with one thread, not multiple threads that RPC pushes you towards for this kind of processing.
There was no UDDI then for service lookup back then. There was no LDAP, either. We built our own distributed, replicated X.500 directory service. Infrastruture and business services registered and reserved leases dynamically. If a service went down, its service entry went away, and clients would call one of the other instances that was still registered. And of course, it was fast - 10s of millis to do a service lookup, not 100s or 1000s like you see with UDDI. Smokin' fast.
I architected and led the development of the Directory Replication Service along with my friend Dave C. This cell-based, scalable solution allowed us to run multiple Directory Services, and replicate data between them in a scalable, available fashion.
We used XDR for marshalling. That's one piece we did download. I lead the charge to unit test the whole thing - and found only one bug. There was an endian issue on that affected one of the platforms, and we found it and fixed it.
There were no application engines. What we built was a resource manager that would control service lifecycle and heartbeat services to quickly detect outages.
There was no PKI, or even SSL. We bought licenses for the encryption libraries, then designed and built our own secure message protocol (with mutual authentication where needed), plus the infrastructure to support it. Certificate Revocation Service, Certificate Authentication Service, and al the APIs for services to use to load their certs, encrypt their messages. We built the services and the GUIs so that administrators could manage the PKI we built.
There was monitoring software available - Tivoli - but it was far more than our budget would allow. We built our own Dashboard that allowed us to closely monitor infrastructure and business services running over much of the United States, including Missouri, Texas, California, and more.
We did training to teach developers how to build reusable business in C, and how to build clients in C, in VB, and in PowerBuilder. I taught the VB and PowerBuilder classes.
Our team consisted of three subteams - Infrastructure, which was the team I was on, the Service Writers team, and System Adminstration, which handled the care and feeding of production systems.
At A.G. Edwards, we leveraged SOA frameworks for 6-1/2 years as part of a broker workstation, to develop the pretty scalable agedwards.com (which, last I heard some years back, had about 300,000 users signed up). I was a member of the Lead Architect Team that designed and developed agedwards.com. We used SOA for some other pretty cool stuff. One of my favorites was the BLServer, that used a modified version of what is now called the "Competing Consumers" pattern in a highly-available cluster of message-based services that self-allocated using leases to process roughly 1.5 million messages a day for about 4-1/2 years before it was mothballed in February of this year.
The industry has come a long way since then. Like I said, what we built, you can now go download, and the SOA business has experience explosive growth. After 2-1/2 years at Bell on the Datagate project, I did 6-1/2 years of SOA work at A.G. Edwards, continuing to architect systems using new SOA technologies as they emerged - Tengah, (now Weblogic) Dynamo, Novell's LDAP server, SiteMinder, WSDs, Big/IP, and various JMS providers. Some of this stuff I had more exposure to than others, but that gives you an idea of how things have changed over the past 14 years. Now there's CXF and JAXB and other exciting new tools coming along.
Every Tom, Dick and Harry does SOA now. Years after that work at Southwestern Bell, a small company hired a "SOA expert" (after I was already there!). This poor fellow didn't even know what LDAP is. When I described how it could be used for service lookup, he said you can't use it for that - it's for authentication. I explained to him that it's just a database that's optimized for reads. You can use it for anything that falls into that category. Now, he's an expert, downloading what other people have built and putting it all together. One boss at that company tried to get me to join a project where I would have been writing architecture documents all day for six months, and building nothing. His angle? That the SOA experience would look good on my resume. I kid you not.
On Datagate, we built SOA sooner, faster, more scalable, and at least in some ways better than what exists today. The primary advantage I see in today's world is standards. Almost everything is standards-based, and that means vendors can provide tools to do many of the things we used vi for. It's a change for the better, in most every way, because SOA is much more accessible today that it was in 1995. Plus, now it's an acronym. ;)
Tuesday, May 5, 2009
Migration from Subversion to Git
I'm migrating from Subversion to Git. It's about time, of course - decentralized SCMs have some significant advantages over the old-style SCMS such as CVS, Clearcase, Subversion and the like. I looked at Bazaar, which touts "Bazaar is a distributed version control system that Just Works." Sadly, I couldn't get it to work. It needed a lot of python modules that I, not being a python guy, don't really know where to get. Found some, not others. Funny thing is, Git "just works." Sure, had some hiccups with converting from Subversion before realizing that I had made an incorrect assumption about the repository structure in one case. Looks cool, though, especially when you throw github in the mix. Good stuff.
Recommended reading if you're thinking about converting from your current SCM to something new, whatever that may be: http://whygitisbetterthanx.com/
Recommended reading if you're thinking about converting from your current SCM to something new, whatever that may be: http://whygitisbetterthanx.com/
Subscribe to:
Posts (Atom)