Mobile QA Strategy

With the emergence of mobile app development, companies often rushed to be the first to market with their new apps, often overlooking steps critical to ensuring a successful application.  As this market is more established, businesses now realize that, just as any other media for reaching customers, a mobile app must be a well planned, properly designed, and quality piece of software in order to be successful.  

Quality Assurance (QA) testing is one of these previously overlooked steps that companies now realize is critically important.  QA activities take on a new level of complexity with mobile applications that can make this a daunting task.  With a wide variety of devices, manufacturers, operating systems, mobile browsers, screen sizes, screen resolutions, and screen proportion, there are literally thousands of unique combinations.  Each combination can result in a different user experience or functional impact, making the scope of testing needs tremendous.

An important first step related to the QA efforts that should happen early in, or prior to, the design and development phase, and before any formal QA testing begins, is to identify a matrix of the target devices and Operating Systems/browsers.  The business should determine a list of all of the different devices that they feel are important, and provide a priority ranking for each.  Similarly, they should provide a list of all mobile operating systems and browsers targeted and provide a priority ranking for each.  Data is available online that gives an overview of the market usage and prevalence of operating systems, browsers, and devices (see http://www.netmarketshare.com/).  Along with the general market information, the business should evaluate their target market, demographic usage patterns and requirements, and business needs in determining their device/platform prioritization.  As there is a direct cost associated with compatibility development, changes, and defect resolution specific to individual devices, this prioritized matrix allows the business to evaluate where money should be spent to enable devices, and which devices are of lower priority and not worth the investment.

Although the application is meant for mobile, the easiest testing method is on the desktop through a standards browser (for mobile website) or development tools (for mobile applications).  Basic functional testing to ensure the code is defect free, meets business requirements, and data entry is appropriately handled can be addressed on the desktop.  Providing feedback to developers is simplest when tested on the desktop, and the easiest to resolve.  This ensures the application is functional in its own state and allows future device specific testing to focus on defects and issues related specifically to the device and software.  Some browsers, such as Firefox, have a setting allowing control of the identified browser agent, which allows the server to provide mobile versions to the desktop browser.  

Emulators provide a convenient and easy way to provide pre-device testing.  As nearly all mobile manufacturers provide development device emulators, developers and QA resources should utilize these services for initial testing across the full spectrum of prioritized devices.  Emulators don't always exactly mimic the device, and certain usability testing can be difficult to test, however this can provide an ability to initially eliminate errors related to the device.  

The final step, which is critical to fully ensuring compatibility and usability, is to test on the actual devices.  The tester should perform tests across the full spectrum of categories (functionality, usability, data entry, security) directly on the devices using a combination of manual testing and automated testing directly against the device.  

Testing on each of the target devices can be a logistical challenge.  Especially if a large variety of devices has been identified, securing devices with mobile data connections across the spectrum could be very costly.  Depending on the nature and frequency of apps being created, outsourcing device testing could be a viable option.  There are three major categories of outsourcing: outsourced testing, device outsourcing, and crowd sourcing.  

Outsourced testing is, as it sounds, completely outsourcing the testing of a mobile application to a vendor specializing in mobile testing.  The testing organization is responsible for test case creation and full execution of testing on mobile devices.  Results of the testing are provided back to the development organization for correction.  

Device outsourcing retains the QA activities within the development organization, however utilizes devices owned/maintained by the outsourcing vendor.  Depending on the vendor, this could involve actually renting devices which are shipped to the development organization, utilizing devices within a lab setup provided by the outsourcing vendor, or remote access to the actual devices.  In the later scenario, you have full control of the actual devices via remote control, and can view the screen through both a screen share, and an actual camera feed of the device.

Crowd sourcing represents an interesting option for testing.  Vendors coordinate real-world users based on the types of devices and configurations they use for their personal mobile communications.  Development organizations can specify the exact category of users to test, and the vendor will provide the application to users to test.  This is generally not thorough testing – simple user interaction testing to provide user feedback and error logs from each type of device.   Although this testing does not provide a full QA implementation, it is an effective method to get initial field results.  

   

Although mobile testing represents significant additional challenges above traditional software testing, it is critically important to ensure thorough testing.  Mobile users have little tolerance for defects, and will not continue to use an application that does not function well or intuitively on their device.  The best chances for success in your mobile application involve creating a well planned and designed, fully tested application.  

Successful IT Project Management

Survey’s show that over 60% of IT projects fail. Failure can refer to numerous things; going over budget, beyond schedule, or not meeting the requirements of the business. Another survey (http://www.genecaresearchreports.com/) asked business and IT professionals their perspective on IT projects and found that 75% felt development projects would either always or usually fail, 78% said they felt the business is usually or always “out of sync” with project requirements, and 80% responded they felt at least 50% of the time during a project is spent on rework. These statistics represent a tremendous amount of wasted time, money, and resources.

What causes projects to fail? Often, this comes down to weak processes, structure, and project management. In this article, we’ll explore the top areas to focus efforts and evaluate how strong project management and adherence to a methodology can dramatically improve project success.

Before exploring opportunities for improvement, it is important to take a deeper look at what defines success. Traditionally, the constraints triangle has set the standard for defining project success with scope, schedule, and budget. Companies are now realizing that the traditional metrics don’t reflect all important success criteria. The project’s overall impact to the business, how effectively it solves the organization’s needs, and the general perception of the project define true project success.


#1 – Business vs. IT Projects
Think in a larger scope for your projects. An IT project is designed to implement a system with a very narrow scope. Focusing solely on IT requirements may ignore the true metrics for success within the overall organization, causing the project to fail in the eyes of the business. A business project, however, is designed to solve an organizational problem. This approach brings together the appropriate expertise from throughout the organization, gains top management buy in, and ties goals and metrics back to business needs. The broad perspective approach of a business project is an underlying theme within all areas of improving project management.


#2 – Proper Planning
Perhaps one of the most common, and most obvious types of project failures arises from poor planning. Before a project can begin, there must be careful planning to ensure the solution solves the business problem. This involves carefully understanding the needs, business drivers, success criteria, stakeholder impact, and numerous other factors related to the environment, organization, and situation. It is critical to have a clear view of where you are heading and the full problem.

A less obvious planning concern is around over-planning. There is a balance that must be identified between identifying enough details to ensure you are moving in the proper direction and taking proper steps, while not fully documenting and exploring every possible detail. Over-planning can result in budget and schedule overrun before the first piece of code is ever written.


#3 – Poor Team
A major cause of project failure is having a poor team. This could involve the most obvious issues – such as a team that doesn’t have the skillset or skill level to complete the work at the needed level of quality, or a team that doesn’t communicate or work well together. Thinking of the team in a broader sense as all who have influence over the project – those actually completing the work, the business leaders, the project manager, the executive stakeholders, etc. – shows there are a lot of areas for breakdowns and failures. All team members and stakeholders must be operating effectively, communicating appropriately, possessing the necessary skills, aligned with the business goals and needs, and motivated to reach the same goals in order for a team to be high performing. Organizing a high performing team is both an art and science, requires significant attention and work by all involved, and can be very challenging to obtain.


#4 –Stakeholder Management
Stakeholders are a critical component of any project, and it is important to carefully manage the interactions and information distribution with stakeholders. Failure to properly understand, manage, and engage stakeholders can result in poor adoption, lack of needed support, poor perception, and confusion. Stakeholders should be properly identified and categorized based on level of interest and influence over the project. Projects should be operated in a highly transparent manner allowing stakeholders visibility in the process, progress, and results, with expectations carefully aligned with the project’s objectives.


#5 - Breaking Down Projects Into Smaller Pieces
A business principle called the “Cone of Uncertainty” demonstrates that the level of risk in a plan increases exponentially as you move further out in time. While you may be fairly certain what development efforts lie before you over the next few days, as you start to look further beyond into months or even years, the degree of uncertainty caused by positive or negative influencing factors becomes great, increasing the level of risk that you won’t be operating on plan. So, why plan so far in advance? Break projects up into the smallest pieces possible to allow operation with the most predictable variables and provide timelines with more certainty and less risk.


#6 - Managing Risks
Risks are a part of every project that can yield significant positive or negative impact. It is critical to proactively manage risks by identifying leading risks by likelihood and impact, and determining a strategy of mitigate, transfer, avoid, or accept. Obviously, you’ll never be able to identify all possible risk scenarios, but, you’ll find projects progress much more effectively when you are actively looking for, and managing, those risks that become apparent. Risk management is an art form that is learned over time.


#7 – Scope Creep
For anyone that has worked with software development long, “scope creep” is a very familiar concept. Often changes appear to be quick to implement, and are made without following formalized scope management policies. These changes, however, add up to significant additional work over time. With no tracking, it can be easy to suddenly be behind schedule and over budget without realizing the cause was a series of small changes. A rigid change management system is imperative. Document the initial intended functionality and scope of the project, and when something arises that is different, identify that it is a scope change, and identify the impact to schedule and budget. It is common that stakeholders don’t realize the impact these changes might have, so, highlighting the change and its impact can help keep the project on track. It is also important to realize the scope creep does not only come from outside of the team. The development team itself, as well as project manager, are just as likely to see opportunities to improve the product and add these “quick” changes to scope.


#8 – Rigidity
At first glance, identifying rigidity as a failure point seems counter-intuitive from the previous point of the necessity of change management. The opposite is actually true – change in a project is not a bad thing and it should be embraced. It is critically important to understand and manage change, but changes to make the product better, better serve the customer, or create additional value should certainly be acted upon if possible. Often times projects fail to deliver the intended value to the business because they stick to the planning that was completed a year or more earlier during the planning phases. Remember, a project delivered on time, in budget, and within scope is still a failure if it doesn’t bring value to the organization. Be open to changes that improve the project or keep it aligned with the business.

While projects can fail for countless reasons, the above listed eight project failure reasons highlight some of the leading causes. The common theme through each of these items is the need for strong project management and leadership skills, adherence to a project methodology, and close alignment between business and technology. With a focus on these areas, projects stand a much greater chance of success.

The Waterfall to Agile Continuum

The two leading approaches to software development are the agile and waterfall methodologies. A common perception is that these are two distinct methodologies that organizations could select. In actual practice, however, organizations will seldom implement at either of these extremes. Waterfall and Agile can be thought of as the end points of a methodology continuum. An organization’s actual implementation of a methodology will lie somewhere along this continuum based on numerous factors and the specific goals and requirements for their development processes.

When listing the benefits of a strict agile approach, it’s easy to jump to the conclusion that agile is the far superior methodology. It does provide numerous benefits that waterfall lacks, which is why many organizations benefit from shifting along the continuum towards agile. However, waterfall certainly has a number of benefits as well and has produced successful results during its decades of dominance as a development methodology. This is where the concept of the continuum becomes so valuable – it allows organizations to not simply choose one approach, but to evaluate the importance of key benefits from each to find their ideal point.


Organizational Structure / Environment
The organization’s structure and environment play a key role in determining the point along the methodology continuum. A highly agile approach requires significant direct collaboration between team members from throughout the organization. If an organization has high barriers for communication between departments or teams due to structure, organization, geography, or any other factor, this could present a challenge for a highly agile approach.

Next, the management style used within the organization must be evaluated. A traditional command and control style is most common within the waterfall approach. A more agile approach promotes self-organization and control within teams that are held accountable for the results they produce.

Finally, what are the values the organization places as it encourages and rewards its team members? Are team members rewarded based on their individual contributions to the project – such as number of features they personally develop, volume of code, or code quality? As a highly agile approach promotes the teams, employee evaluation should be focused on how well individuals contribute and thrive within the team environment, evaluating aspects such as how well they mentor junior team members, how they encourage and promote their team to accomplish sprint goals, how they communicate ideas within their team, and the level of insight they provide during team discussions.


Level of Support
The level of support is a critical factor to consider as you are evaluating the change to a more agile methodology. A highly agile approach requires an organizational level change across teams and functional areas of the organization. Management at all levels must embrace the change in thinking, responsibilities, and structure. If a change is limited to only the development team, the team can adopt agile practices to gain certain benefits, but the resulting methodology will be limited in its movement towards the agile end on the continuum.


Level of Talent
A third factor to consider is the level and type of talent available to your teams. An agile approach often works best with a combination of junior and senior level talent. In this environment, senior level resources will tend to provide mentoring to junior level talent, allowing them to continuously improve their skillsets. Projects will always tend to have a combination of difficulty levels of the work to be performed, so the team will be able to collaboratively work together to complete all work utilizing the varying levels and types of expertise available.

Another factor to consider relative to the team is their perspective on development. In the “software factory” model of development, teams are used to receiving precise specifications and requirements, and focus on coding and technology to meet these requirements. There are some teams that thrive in this kind of environment, and these types of developers could become less effective if given more latitude and responsibility for business problem solving. Other teams, however, may produce far superior results if given the opportunity to apply their full skillset to analyzing, solving, and implementing solutions based on a broader understanding of the end user’s need. If the later is the case, then the team will succeed with a more agile approach.


Trust in Teams
Do you trust your teams? The immediate answer to this question is generally, “Yes, of course – we wouldn’t have hired them if we didn’t trust them!” An organization should take a very critical approach to evaluating the true level of trust they have for their teams. A highly agile approach gives significant empowerment to the teams, and trusts the teams will perform at their highest level capacity when given more freedom and latitude. Further, agile empowers teams to make key decisions throughout the process that will directly impact the success, or failure, of the project. If an organization does not feel that they can stand behind the decisions made by their teams, or that the teams will not perform at optimal levels when left to their own self-organization, than a highly agile approach would not be appropriate.

An important is that a change to a more agile approach doesn’t mean the organization must start trusting its teams. As a subtle difference, the organization should focus on forming teams that it can trust, not simply instilling trust in existing teams. Strong teams are the foundation of the success of an agile approach, so it is critical that the teams be seeded with trusted, high performing team members. If the organization trusts the people going into the teams, then the organization will be able to trust the teams.


Conclusion
Determining the methodology for your organization is a major decision. Moving towards a more agile approach can be a lengthy and difficult change, but can yield great rewards. An organization should start with a thorough and honest evaluation of itself, including considering the above factors. With this insight, the organization can create a methodology that is tailored to gain the benefits from both the agile and waterfall worlds. This customized methodology allows you to find the optimal spot along the continuum where your organization will realize the greatest results.

Going Mobile - An Introductory Guide Part 4 - Marketing Your Brand

In the “going mobile” series, we’ve evaluated the approaches of creating mobile websites and mobile apps . The final avenue to explore in the “Going Mobile” series is in marketing your brand through mobile channels. This option involves strategic placement of your business identity in front of consumers through mobile channels. Much like traditional media advertising strategy, organizations identify channels familiar to their target demographic and place their message in these channels. This may include simple ad placement on mobile sites and in mobile apps, or more transparent brand integration opportunities.

Mobile website and app ad placement works very similar to internet ad placement. The nature of the medium allows highly precise demographic targeting to focus advertising dollars. Depending on the channel, ads can be selectively displayed based on user age, geographic location, interests, and more. Mobile advertising clearinghouses simplify this process with the ability to place ads across multiple sites/apps focusing specifically on the target demographic. Rich media advertisements, those that are in some manner interactive, are proving to be highly effective at attractive attention and converting into click-through.

Brand integration mobile advertising strategies allow a more natural, often subliminal placement of your business information in mobile sites or apps. Your products, message, or image can be placed at key opportunities on mobile sites and apps. This might include mentions or reviews of products / services, products or images being visible in other contexts, etc. Often the goal of cross-promotional activities is not to result in immediate click-through transactions, but instead to increase awareness, image, or align with other products, people, or organizations for validity and association.

An exciting opportunity for mobile marketing exists with locations based apps. A number of social apps integrate geographic location check-in and location awareness. This presents a new unique opportunity to reach out to potential customers on their device at the precise opportunity to attract them to your location. For example, offering discounts or information to attract customers into your location when they check-in at a nearby location. Encouraging customer to check-in from your business can also help to encourage new customers through implied, unsolicited recommendations (seeing friends visiting a business could encourage people to visit your business as well).

Opportunities for mobile marketing go far beyond the few ideas presented in this posting. As mobile devices increasingly grow in their role as a communications tool, entertainment center, information gathering portal, and consumer transaction device, the opportunities will continue to expand. Businesses must carefully evaluate the various avenues for engaging the mobile market through marketing, mobile website development, and app development. Each implementation approach provides significant opportunities when fully integrated and supporting an organization’s overall business strategy. This represents an exciting method to engage, attract, and retain customers through the technology that consumers cling to in their daily lives.