Intro to Insane Startup QA (Part II)

Thanks for checking back in.

If you recall we last left our hero on the doorstep of an Internet startup company that had brought him in to "QA" their product. Well, what they really meant was they wanted someone to come in and test like heck for two months and find as many issues as humanly possible and get them fixed. Silly rabbits. QA is far more than just testing and verifying bugs as those "in the know" know. Is know know a no-no? Hmm, weird.

Anyways, QA encompasses requirements reviews, risk analysis, project coordination, test plans, automation, metrics, blahblahblah...there are many other websites that can tell you what it is and isn't, I won't bore you with a lecture or argue with you about what the differences are between QA'ers and testers. For the purpose of our story, our QA guy found himself in a situation that called for rapid assessment and exploratory testing. Real QA will come later, once the bodies have been hidden.

Rapid exploratory assessment testing...huh?

Not quite. Simply put, you have to understand what type of environment it is that you're in and figure out what approach you're going to take in order to be effective. You see, time is not on your side. Here are a some tips that can help you with the assessment part...but first, a quick joke...What does the acronym SPECS stand for? Simple Papers Engineers Can't Supply. (Haha...ok, I'm done, no more dumb jokes)...here we go...

• Where's the info?
• What development methodology are they attempting to follow? XP, Agile, Waterfall (gasp!) or the chaotic hybrid, XPAgileFall? This will clue you in quickly on how organized the team is (or isn't) and help you identify areas where subtle improvements can have a massive impact on productivity.
• Ask the question...how does a feature originate, get through dev and then make it's way into the application?
  
• Do they have any documented processes for checking in code, doing builds, running unit tests, deploying code, etc...? More importantly, do they follow them if they do?
• Do they use a Wiki? If so, dig into it. Read and comment on pages that aren't clear to you (a guy or gal that knows nothing about the app yet). If not, get one set up a.s.a.p. It's a valuable tool.
  
• Go through whatever they're using for a bugdb (this will be very educational as it will illustrate how others on the team communicate). 10 to 1 says it's doubling as their requirements tracking system too (and that's only ok if it's JIRA).
  
• What kind of environment are you going to have to test in? Believe it or not, you may be testing in a production environment at first, don't be shocked.
  

• State of the application
  
• This is where exploratory testing comes in. You will absolutely have to seek out someone to walk you through the basics of the application. After that, it's time to skin your knees on the app a bit. I believe this definition works just fine for exploratory testing.
• You will have to come to your own conclusions here. Typically a lot of the bugs that are caught at this stage in a startup are found by end-users. Maybe the CEO, VP of Marketing, CTO or whoever else outside of development has been hitting the app prior to you getting there. Chances are great that whatever bugs are in the system might be one-liners and/or just a screen shot. Don't get discouraged, my advice is to move on past them until you're more comfortable with the app. Just remember, not everyone can write up a kick-ass bug report like you can.
  

• Who is in play?
  
• This is one that you typically only learn by experience. Every company is unique. But in startups, there is typically an undocumented (like everything else) hierarchy. Here's how it may work...
• Founder(s) - They call the shots and they damn well should get to, after all, it's their company. They're the visionaries, you should spend any time you can with them to get their input on what's most important to them.
  
• The First Born - These are the ones the founder(s) first hired to help them. These folks have a tremendous amount of pull when it comes to adding features and are also very in tune with the business, the market and the product. Seek out their help with testing. They're the ones that have been or will have to demo the app. They have a vested interest in helping this app & company succeed. These are often the most valuable testers as they use the application (hopefully) as much as anyone.
  
• The Center of the Universe - Every startup has one. It's that one person that everyone goes to for answers or is there keeping all the duct tape in place. They know every nook and cranny of the app and how it works. They're the glue that holds it all together and are typically so damn busy they often can't be counted on to give you the time and attention you'll need as a QA'er (note...fear not, over time, you may elevate to this position...if you really want it that is...say goodbye to your social life). Important note here...offer whatever help you can to this person, they will be one of your most valuable assets.
  
• The Young Guns - Ah, the energy, the optimism, the talent, the bravado, the inexperience, the arrogance. I'm joking honestly here. Most startups recruit younger staff because it's cost effective and they can get more hours out of them. What they lack in experience they more than make up for in a willingness to experiment. Tap into that eagerness by challenging them to help you out by doing some unit testing, build automation or test harnesses.
  
• Customers - Do you have any? No. Move on. Yes? Congrats. As a QA'er, I can't say that this will be a valuable asset at the beginning, but I'll hedge that with "it depends". You may have one single customer that you're building your app for, if so, make them very happy and involve them in the testing and discovery as much as you can. In my current case, we have a ton of them and a support team to handle the front end. Some day, when we're a grown up QA team, we'll look to engage them more.
  

• What needs my attention the most?
  
• Technically, you need to identify the areas of greatest risk. You'll need to engage the developers and get their input on what areas of the code are 'iffy'. Most developers will admit a hack is a hack. Startup code has lots of them because they build a feature, tweak it, change the feature, tweak it, etc... By the time they're done, it's completely different than what they started with and unless they scrap the code and re-built it, it's probably a mish-mash by now.
• Look in the bugdb for counts v. the different components? This can tell you a) what features the end-users are looking at the most (most bugs) and b) where there are no bugs, this might be a newer feature and very likely, untested.
• Be very sensitive to "high visibility areas". They may not contain the most complex code, but these are the areas where execs hate to see sloppy bugs. It is amazing what a clean UI can do for an application as the first impression is very, very important. This is also important because these same execs will be demo'ing the wonder-app and if it looks like crap, it likely will be hard to sell. I'm not suggesting that Todd Garland build your site (although he would for the right price), but the on-screen controls should all work correctly and the page layout should be clean and legible.
• Wherever the app is doing the heavy lifting (web or data layer), have developers build unit tests that exercise the core code so you can focus on the functional integration and system-level tests.

Phew! Well, that pretty much summarizes your first day at a startup.

See ya on Day #2.