testing

As you’ve probably noticed by now, we’ve posted a couple of release notes over the past 2 weeks. You’re probably wondering, “How do I update?” The reality is, we’ve tweaked our update platform to amplify testing and gain additional real world use prior to making new builds available to all ISPs using Powercode. Let’s dive into some of the upcoming changes.

There are 2 main elements to this, but let’s first back up and briefly explain what our current internal testing looks like.

Internal Lifecycle of New/Changed Code

When we develop Powercode and move our code base forward, the author first tests each and every change they’ve made. Next, that change is submitted to the entire development team and is reviewed before it is accepted as “part of Powercode.”

As different changes from all our developers get submitted, reviewed, and accepted into Powercode, it accumulates in a new build. We decide when to cut off new changes from entering the “next build” by evaluating how much “risk” is created by all the changes in the build.

Now, when we use the word “risk” here, I don’t want anyone to think Powercode gambles with these releases (not since I’ve taken over at least).

What we’re talking about is simply:

• What is the scope of the changes?
• If something were to break, where could it potentially happen?
• If something were to break, what is the worst case impact of that break?
• How will the amount of changes impact our support burden?

Though there a few more factors we think about, the above about covers it.

So now we have our build and we decide this is our release. The next step is to make sure our support and sales departments understand the changes and then have them use their real world experience with our customers to try to break things, offer new perspectives and generally do some more targeted testing.

Support has a fully built-out lab for them to test against, with various hardware pieces, topologies, etc.

After another round of tweaks and changes based on the support team’s perspective, we have a build ready.

New Update Tracks

Now, this is where the bigger public facing changes come in. We recognize that Powercode is used in very different ways across many ISPs. There are tons of options that create many forks in the road. Quite truthfully, some scenarios and situations are not only impossible for us to test, but some are inconceivable to us. There are number of creative option/setting combinations out there that our customers employ to achieve some… let’s just say… “secret features”.

Stable vs Beta

With that said, it’s very important to us to get some diverse, real-world use cases tested before we unleash these builds out into the world. We’re going to accomplish this with two main tracks. One is a customer facing track (you). The other is an internal track for us.

In the system settings of Powercode of the latest release and onwards, you’ll notice a new setting that allows you to upgrade from either the “Stable” track or the “Beta” track. It’s a simple dropdown you use to choose and save.

You’ll always be able to jump back and forth without any consequences. The only limitation is that if you choose “Beta,” and have a version that is ahead of the “Stable” track, you cannot switch back to “Stable” and expect to be downgraded. You’ll need to wait until the “Stable” track gets ahead of you before you see any more updates.

Update Groups

In combination with choosing which update track to follow, we’ve also implemented update groups, an internal setting accessible to us at Powercode. As we track different support issues, use cases, and other variations between our ISP customers, we build groups to get a sampling of different environments we want Powercode to be released against. For example, we have a number of ISPs that run a test instance of Powercode for themselves (This is free by the way. Just call and ask.) to use for training staff, API testing, and more. Internally we will put all of those test servers into “Group 1.” So if you operate a test server, and set it to track the Beta releases, you’ll get all of our releases immediately.

We’ll take any and all feedback from our “Beta Group 1” servers and fix, tweak, or add anything that is necessary then release to a wider audience, “Beta Group 2.”

Only once we’ve gone through this amount of rigorous, real-world testing, will we push the build out to “Stable Group 1” and subsequently, “Stable Group 2.”

Why Multiple Groups For Stable?

You might think, “Well, if you’re calling it ‘Stable,’ why split it into groups?” Splitting it into groups during the “Stable” release isn’t about testing, but rather about making sure the service and support we provide is efficient. Not only has Powercode grown significantly recently, we have some fairly ambitious features in the pipeline. We will be implementing another phase of these update changes which will require these stable groups to be differentiated. The biggest piece of that next phase is attended upgrades, where you will have a Powercode representative on standby during all of your upgrades. This is not “just in case something goes wrong,” but because we want to make sure you implement new features to their maximum return.

Some of this is still up in the air as we iron out the internal processes. However, our goal is to make the pain of updating as low as possible so we can enable everyone to take advantage of the new features, fixes, and enhancements Powercode is about to unleash.