Mobile QA Strategy

With the emergence of mobile app development, companies often rushed to be the first to market with their new apps, often overlooking steps critical to ensuring a successful application.  As this market is more established, businesses now realize that, just as any other media for reaching customers, a mobile app must be a well planned, properly designed, and quality piece of software in order to be successful.  

Quality Assurance (QA) testing is one of these previously overlooked steps that companies now realize is critically important.  QA activities take on a new level of complexity with mobile applications that can make this a daunting task.  With a wide variety of devices, manufacturers, operating systems, mobile browsers, screen sizes, screen resolutions, and screen proportion, there are literally thousands of unique combinations.  Each combination can result in a different user experience or functional impact, making the scope of testing needs tremendous.

An important first step related to the QA efforts that should happen early in, or prior to, the design and development phase, and before any formal QA testing begins, is to identify a matrix of the target devices and Operating Systems/browsers.  The business should determine a list of all of the different devices that they feel are important, and provide a priority ranking for each.  Similarly, they should provide a list of all mobile operating systems and browsers targeted and provide a priority ranking for each.  Data is available online that gives an overview of the market usage and prevalence of operating systems, browsers, and devices (see http://www.netmarketshare.com/).  Along with the general market information, the business should evaluate their target market, demographic usage patterns and requirements, and business needs in determining their device/platform prioritization.  As there is a direct cost associated with compatibility development, changes, and defect resolution specific to individual devices, this prioritized matrix allows the business to evaluate where money should be spent to enable devices, and which devices are of lower priority and not worth the investment.

Although the application is meant for mobile, the easiest testing method is on the desktop through a standards browser (for mobile website) or development tools (for mobile applications).  Basic functional testing to ensure the code is defect free, meets business requirements, and data entry is appropriately handled can be addressed on the desktop.  Providing feedback to developers is simplest when tested on the desktop, and the easiest to resolve.  This ensures the application is functional in its own state and allows future device specific testing to focus on defects and issues related specifically to the device and software.  Some browsers, such as Firefox, have a setting allowing control of the identified browser agent, which allows the server to provide mobile versions to the desktop browser.  

Emulators provide a convenient and easy way to provide pre-device testing.  As nearly all mobile manufacturers provide development device emulators, developers and QA resources should utilize these services for initial testing across the full spectrum of prioritized devices.  Emulators don't always exactly mimic the device, and certain usability testing can be difficult to test, however this can provide an ability to initially eliminate errors related to the device.  

The final step, which is critical to fully ensuring compatibility and usability, is to test on the actual devices.  The tester should perform tests across the full spectrum of categories (functionality, usability, data entry, security) directly on the devices using a combination of manual testing and automated testing directly against the device.  

Testing on each of the target devices can be a logistical challenge.  Especially if a large variety of devices has been identified, securing devices with mobile data connections across the spectrum could be very costly.  Depending on the nature and frequency of apps being created, outsourcing device testing could be a viable option.  There are three major categories of outsourcing: outsourced testing, device outsourcing, and crowd sourcing.  

Outsourced testing is, as it sounds, completely outsourcing the testing of a mobile application to a vendor specializing in mobile testing.  The testing organization is responsible for test case creation and full execution of testing on mobile devices.  Results of the testing are provided back to the development organization for correction.  

Device outsourcing retains the QA activities within the development organization, however utilizes devices owned/maintained by the outsourcing vendor.  Depending on the vendor, this could involve actually renting devices which are shipped to the development organization, utilizing devices within a lab setup provided by the outsourcing vendor, or remote access to the actual devices.  In the later scenario, you have full control of the actual devices via remote control, and can view the screen through both a screen share, and an actual camera feed of the device.

Crowd sourcing represents an interesting option for testing.  Vendors coordinate real-world users based on the types of devices and configurations they use for their personal mobile communications.  Development organizations can specify the exact category of users to test, and the vendor will provide the application to users to test.  This is generally not thorough testing – simple user interaction testing to provide user feedback and error logs from each type of device.   Although this testing does not provide a full QA implementation, it is an effective method to get initial field results.  

   

Although mobile testing represents significant additional challenges above traditional software testing, it is critically important to ensure thorough testing.  Mobile users have little tolerance for defects, and will not continue to use an application that does not function well or intuitively on their device.  The best chances for success in your mobile application involve creating a well planned and designed, fully tested application.  

Successful IT Project Management

Survey’s show that over 60% of IT projects fail. Failure can refer to numerous things; going over budget, beyond schedule, or not meeting the requirements of the business. Another survey (http://www.genecaresearchreports.com/) asked business and IT professionals their perspective on IT projects and found that 75% felt development projects would either always or usually fail, 78% said they felt the business is usually or always “out of sync” with project requirements, and 80% responded they felt at least 50% of the time during a project is spent on rework. These statistics represent a tremendous amount of wasted time, money, and resources.

What causes projects to fail? Often, this comes down to weak processes, structure, and project management. In this article, we’ll explore the top areas to focus efforts and evaluate how strong project management and adherence to a methodology can dramatically improve project success.

Before exploring opportunities for improvement, it is important to take a deeper look at what defines success. Traditionally, the constraints triangle has set the standard for defining project success with scope, schedule, and budget. Companies are now realizing that the traditional metrics don’t reflect all important success criteria. The project’s overall impact to the business, how effectively it solves the organization’s needs, and the general perception of the project define true project success.


#1 – Business vs. IT Projects
Think in a larger scope for your projects. An IT project is designed to implement a system with a very narrow scope. Focusing solely on IT requirements may ignore the true metrics for success within the overall organization, causing the project to fail in the eyes of the business. A business project, however, is designed to solve an organizational problem. This approach brings together the appropriate expertise from throughout the organization, gains top management buy in, and ties goals and metrics back to business needs. The broad perspective approach of a business project is an underlying theme within all areas of improving project management.


#2 – Proper Planning
Perhaps one of the most common, and most obvious types of project failures arises from poor planning. Before a project can begin, there must be careful planning to ensure the solution solves the business problem. This involves carefully understanding the needs, business drivers, success criteria, stakeholder impact, and numerous other factors related to the environment, organization, and situation. It is critical to have a clear view of where you are heading and the full problem.

A less obvious planning concern is around over-planning. There is a balance that must be identified between identifying enough details to ensure you are moving in the proper direction and taking proper steps, while not fully documenting and exploring every possible detail. Over-planning can result in budget and schedule overrun before the first piece of code is ever written.


#3 – Poor Team
A major cause of project failure is having a poor team. This could involve the most obvious issues – such as a team that doesn’t have the skillset or skill level to complete the work at the needed level of quality, or a team that doesn’t communicate or work well together. Thinking of the team in a broader sense as all who have influence over the project – those actually completing the work, the business leaders, the project manager, the executive stakeholders, etc. – shows there are a lot of areas for breakdowns and failures. All team members and stakeholders must be operating effectively, communicating appropriately, possessing the necessary skills, aligned with the business goals and needs, and motivated to reach the same goals in order for a team to be high performing. Organizing a high performing team is both an art and science, requires significant attention and work by all involved, and can be very challenging to obtain.


#4 –Stakeholder Management
Stakeholders are a critical component of any project, and it is important to carefully manage the interactions and information distribution with stakeholders. Failure to properly understand, manage, and engage stakeholders can result in poor adoption, lack of needed support, poor perception, and confusion. Stakeholders should be properly identified and categorized based on level of interest and influence over the project. Projects should be operated in a highly transparent manner allowing stakeholders visibility in the process, progress, and results, with expectations carefully aligned with the project’s objectives.


#5 - Breaking Down Projects Into Smaller Pieces
A business principle called the “Cone of Uncertainty” demonstrates that the level of risk in a plan increases exponentially as you move further out in time. While you may be fairly certain what development efforts lie before you over the next few days, as you start to look further beyond into months or even years, the degree of uncertainty caused by positive or negative influencing factors becomes great, increasing the level of risk that you won’t be operating on plan. So, why plan so far in advance? Break projects up into the smallest pieces possible to allow operation with the most predictable variables and provide timelines with more certainty and less risk.


#6 - Managing Risks
Risks are a part of every project that can yield significant positive or negative impact. It is critical to proactively manage risks by identifying leading risks by likelihood and impact, and determining a strategy of mitigate, transfer, avoid, or accept. Obviously, you’ll never be able to identify all possible risk scenarios, but, you’ll find projects progress much more effectively when you are actively looking for, and managing, those risks that become apparent. Risk management is an art form that is learned over time.


#7 – Scope Creep
For anyone that has worked with software development long, “scope creep” is a very familiar concept. Often changes appear to be quick to implement, and are made without following formalized scope management policies. These changes, however, add up to significant additional work over time. With no tracking, it can be easy to suddenly be behind schedule and over budget without realizing the cause was a series of small changes. A rigid change management system is imperative. Document the initial intended functionality and scope of the project, and when something arises that is different, identify that it is a scope change, and identify the impact to schedule and budget. It is common that stakeholders don’t realize the impact these changes might have, so, highlighting the change and its impact can help keep the project on track. It is also important to realize the scope creep does not only come from outside of the team. The development team itself, as well as project manager, are just as likely to see opportunities to improve the product and add these “quick” changes to scope.


#8 – Rigidity
At first glance, identifying rigidity as a failure point seems counter-intuitive from the previous point of the necessity of change management. The opposite is actually true – change in a project is not a bad thing and it should be embraced. It is critically important to understand and manage change, but changes to make the product better, better serve the customer, or create additional value should certainly be acted upon if possible. Often times projects fail to deliver the intended value to the business because they stick to the planning that was completed a year or more earlier during the planning phases. Remember, a project delivered on time, in budget, and within scope is still a failure if it doesn’t bring value to the organization. Be open to changes that improve the project or keep it aligned with the business.

While projects can fail for countless reasons, the above listed eight project failure reasons highlight some of the leading causes. The common theme through each of these items is the need for strong project management and leadership skills, adherence to a project methodology, and close alignment between business and technology. With a focus on these areas, projects stand a much greater chance of success.

The Waterfall to Agile Continuum

The two leading approaches to software development are the agile and waterfall methodologies. A common perception is that these are two distinct methodologies that organizations could select. In actual practice, however, organizations will seldom implement at either of these extremes. Waterfall and Agile can be thought of as the end points of a methodology continuum. An organization’s actual implementation of a methodology will lie somewhere along this continuum based on numerous factors and the specific goals and requirements for their development processes.

When listing the benefits of a strict agile approach, it’s easy to jump to the conclusion that agile is the far superior methodology. It does provide numerous benefits that waterfall lacks, which is why many organizations benefit from shifting along the continuum towards agile. However, waterfall certainly has a number of benefits as well and has produced successful results during its decades of dominance as a development methodology. This is where the concept of the continuum becomes so valuable – it allows organizations to not simply choose one approach, but to evaluate the importance of key benefits from each to find their ideal point.


Organizational Structure / Environment
The organization’s structure and environment play a key role in determining the point along the methodology continuum. A highly agile approach requires significant direct collaboration between team members from throughout the organization. If an organization has high barriers for communication between departments or teams due to structure, organization, geography, or any other factor, this could present a challenge for a highly agile approach.

Next, the management style used within the organization must be evaluated. A traditional command and control style is most common within the waterfall approach. A more agile approach promotes self-organization and control within teams that are held accountable for the results they produce.

Finally, what are the values the organization places as it encourages and rewards its team members? Are team members rewarded based on their individual contributions to the project – such as number of features they personally develop, volume of code, or code quality? As a highly agile approach promotes the teams, employee evaluation should be focused on how well individuals contribute and thrive within the team environment, evaluating aspects such as how well they mentor junior team members, how they encourage and promote their team to accomplish sprint goals, how they communicate ideas within their team, and the level of insight they provide during team discussions.


Level of Support
The level of support is a critical factor to consider as you are evaluating the change to a more agile methodology. A highly agile approach requires an organizational level change across teams and functional areas of the organization. Management at all levels must embrace the change in thinking, responsibilities, and structure. If a change is limited to only the development team, the team can adopt agile practices to gain certain benefits, but the resulting methodology will be limited in its movement towards the agile end on the continuum.


Level of Talent
A third factor to consider is the level and type of talent available to your teams. An agile approach often works best with a combination of junior and senior level talent. In this environment, senior level resources will tend to provide mentoring to junior level talent, allowing them to continuously improve their skillsets. Projects will always tend to have a combination of difficulty levels of the work to be performed, so the team will be able to collaboratively work together to complete all work utilizing the varying levels and types of expertise available.

Another factor to consider relative to the team is their perspective on development. In the “software factory” model of development, teams are used to receiving precise specifications and requirements, and focus on coding and technology to meet these requirements. There are some teams that thrive in this kind of environment, and these types of developers could become less effective if given more latitude and responsibility for business problem solving. Other teams, however, may produce far superior results if given the opportunity to apply their full skillset to analyzing, solving, and implementing solutions based on a broader understanding of the end user’s need. If the later is the case, then the team will succeed with a more agile approach.


Trust in Teams
Do you trust your teams? The immediate answer to this question is generally, “Yes, of course – we wouldn’t have hired them if we didn’t trust them!” An organization should take a very critical approach to evaluating the true level of trust they have for their teams. A highly agile approach gives significant empowerment to the teams, and trusts the teams will perform at their highest level capacity when given more freedom and latitude. Further, agile empowers teams to make key decisions throughout the process that will directly impact the success, or failure, of the project. If an organization does not feel that they can stand behind the decisions made by their teams, or that the teams will not perform at optimal levels when left to their own self-organization, than a highly agile approach would not be appropriate.

An important is that a change to a more agile approach doesn’t mean the organization must start trusting its teams. As a subtle difference, the organization should focus on forming teams that it can trust, not simply instilling trust in existing teams. Strong teams are the foundation of the success of an agile approach, so it is critical that the teams be seeded with trusted, high performing team members. If the organization trusts the people going into the teams, then the organization will be able to trust the teams.


Conclusion
Determining the methodology for your organization is a major decision. Moving towards a more agile approach can be a lengthy and difficult change, but can yield great rewards. An organization should start with a thorough and honest evaluation of itself, including considering the above factors. With this insight, the organization can create a methodology that is tailored to gain the benefits from both the agile and waterfall worlds. This customized methodology allows you to find the optimal spot along the continuum where your organization will realize the greatest results.

Going Mobile - An Introductory Guide Part 4 - Marketing Your Brand

In the “going mobile” series, we’ve evaluated the approaches of creating mobile websites and mobile apps . The final avenue to explore in the “Going Mobile” series is in marketing your brand through mobile channels. This option involves strategic placement of your business identity in front of consumers through mobile channels. Much like traditional media advertising strategy, organizations identify channels familiar to their target demographic and place their message in these channels. This may include simple ad placement on mobile sites and in mobile apps, or more transparent brand integration opportunities.

Mobile website and app ad placement works very similar to internet ad placement. The nature of the medium allows highly precise demographic targeting to focus advertising dollars. Depending on the channel, ads can be selectively displayed based on user age, geographic location, interests, and more. Mobile advertising clearinghouses simplify this process with the ability to place ads across multiple sites/apps focusing specifically on the target demographic. Rich media advertisements, those that are in some manner interactive, are proving to be highly effective at attractive attention and converting into click-through.

Brand integration mobile advertising strategies allow a more natural, often subliminal placement of your business information in mobile sites or apps. Your products, message, or image can be placed at key opportunities on mobile sites and apps. This might include mentions or reviews of products / services, products or images being visible in other contexts, etc. Often the goal of cross-promotional activities is not to result in immediate click-through transactions, but instead to increase awareness, image, or align with other products, people, or organizations for validity and association.

An exciting opportunity for mobile marketing exists with locations based apps. A number of social apps integrate geographic location check-in and location awareness. This presents a new unique opportunity to reach out to potential customers on their device at the precise opportunity to attract them to your location. For example, offering discounts or information to attract customers into your location when they check-in at a nearby location. Encouraging customer to check-in from your business can also help to encourage new customers through implied, unsolicited recommendations (seeing friends visiting a business could encourage people to visit your business as well).

Opportunities for mobile marketing go far beyond the few ideas presented in this posting. As mobile devices increasingly grow in their role as a communications tool, entertainment center, information gathering portal, and consumer transaction device, the opportunities will continue to expand. Businesses must carefully evaluate the various avenues for engaging the mobile market through marketing, mobile website development, and app development. Each implementation approach provides significant opportunities when fully integrated and supporting an organization’s overall business strategy. This represents an exciting method to engage, attract, and retain customers through the technology that consumers cling to in their daily lives.

Going Mobile - An Introductory Guide Part 3 - Creating a Mobile App

In the last post of the “going mobile” series, I discussed the importance of businesses setting up mobile websites. A second opportunity for entering the mobile space that business should evaluate is if a mobile app is right for their brand and a fit within their business strategy. A mobile app differs from a mobile website in that it is a piece of operational software that is installed on the device and available offline. Whereas a mobile website is accessible through the mobile browser, the app is accessed generally by an icon specific to the app on the device desktop.
Recent studies show the average smartphone user is now spending more time throughout the day using apps than browsing the web through the mobile browser. So what types of apps are people using most? Game play represents the largest category, followed by weather, social networking, navigation, music, and news.
So, should your organization rush out to create an app? Maybe not. Before you rush into building a new app, determine how this really fits into your strategy – keeping in mind that a strategy reflects the needs and objectives of both the business and the customers. While every business would love to have customers that eagerly await their press releases, latest news, and carefully review the details of their business on a regular basis, this may not really equate to the customer’s needs and desires. “If you build it, they will come” may work for a baseball field in Iowa, but doesn’t hold true to an app for your business. Convincing people to first download your app, and then to use it regularly can be a difficult sell depending on the type of app you have to offer.
Don’t write off creating an app just yet, though. Although an app doesn’t fit into the strategic model for every business, it is equally fair to say that apps do represent a very significant successful contributor to the strategy of other organizations. Given the volume of time spent by consumers using apps, and the value consumers perceive in apps (spending an estimated $3.8 Billion in 2011 in app stores), it is certainly a worthwhile consideration if an app should be part of your strategic roadmap.
Does your business posses some kind of information, service, or other offering that customers will want to access often or when they are offline? Does this need correspond to a benefit to the business – to better serve the customer, drive additional revenue, attract new customers, or compete against competitors to maintain customers? If so, than perhaps an app is the right approach. Get creative. Be innovative. Engage industry experts to help identify how a mobile app approach fits into your business strategy. It comes down to a straightforward value comparison for both the business and consumer. For the consumer – does the app provide a benefit (time saver, entertainment, satisfy a need, etc.) that is more valuable than the time (and, depending on your model, price) of downloading AND using the app (don’t forget – downloading the app is only half the battle, there has to be a value to encourage the user to continually use the app)? For the business, does the value towards strategic goals generated from offering the app (revenue, customer service/loyalty, etc.) exceed the development and maintenance costs of the app? If the answer to both of these is yes – than an app represents a great opportunity for your business to meet its goals.
Once you’ve identified that an app is a fit for your strategy, the next step is determining your technical approach. Depending on the needs of the app, you may select to develop it using cross platform development tools, or native development. A cross-platform development technique allows a single application to be created for accessibility across different device categories (smartphones, tablets, etc.), device manufacturers, and operating systems (iOS, Android, etc.). Cross platform yields a larger market segment across devices at a lower cost, but you heavily trade a more “natural” experience specific to the device and the deep level of device integration. For simpler apps that are highly general information based, this may be a great approach. Native development is created in a programming language more specific to a particular operating system (iOS, Android, etc.). Native development yields apps that are highly customized to the specific device with a natural feel and a clean integration with all of the features of the device, but results in multiple implementations and a higher cost. Identifying your specific organization, customer, budget, and app needs can help determine the appropriate approach technique.
While determining your approach, you must also consider what types of devices to target. Should you target iOS, Android, or both? Should you target smartphones, tablets, or both? These are critical decisions that can be identified only through an evaluation of your customers, your business goals, and the specifics of your app.
Although the discussion on apps has been centered around a business launching a public facing distributed app, it is also important to quickly note that apps can also be highly effective towards meeting organization objectives as internal apps. Internal apps represent an opportunity for empowering efficiency within a mobilized workforce, as well as providing additional customer service opportunities through increased availability to systems and information from the field. Organizations may find opportunities to use third party mobile apps already built for their internal systems, or create new custom apps, to gain a significant competitive advantage with access and efficiency away from the desktop.

Going Mobile - An Introductory Guide Part 2 - Mobile Websites

Looking back 10 years, and perhaps even less, many businesses were saying “We don’t need a website”. The profound impact the Internet has had on commerce in the modern culture has made a clear and compelling case that every business has a need for a website, and few would argue the value of at least a minimal web presence today. In much the same way, the explosive growth in mobile consumer patterns shows a need for businesses to cater to the browsing experience on a mobile device.

This second installment of the four part “Going Mobile” blog series highlights mobile websites - one of the most basic, and most important, approaches a business or organization can take in establishing a mobile presence. Once an organization has determined their business strategy by evaluating customer needs and business goals, the mobile website can be an effective tactic in implementing their strategy and connecting with customers in the mobile space.

A mobile website can simply be thought of as a website that is formatted in content, design, and experience to work with a mobile browser. While mobile browsers are able to browse standard Internet websites, the smaller screen size, nature of the device, and limited functionality of the browser yield many websites either inoperable or challenging to view and navigate. Traditionally, mobile customers have been accepting of these limitations. However, as the novelty of these devices wavers and they become the norm, the acceptance of a less than optimal mobile experience will quickly fade, becoming a liability for businesses that ignore this need. Just as businesses have come to understand their website as the virtual face of their company, a poorly functioning mobile presence provides an undesirable image of the company to mobile users.

An effective mobile website is more than simply a translation of the current Internet website to fit the screen size. The user experience design is absolutely critical for your site. Identify and understand the needs and objectives of both your organization and your customers as it pertains to your overall strategy in the context of a mobile device. Using this understanding, create the navigation, flow, aesthetics, and content for the site. Avoid long text, extensive scrolling, and deep background details in favor of more situational-aware concise and current information.

Going Mobile - An Introductory Guide

Going mobile – it’s a key component of the strategic road map of many organizations. The mobile market represents an enormous opportunity to serve existing customers, attract new customers, and explore new areas of business growth. Given the dramatic and ongoing growth in consumer mobile data usage and spending patterns, this is a market that cannot, and should not, be ignored. While some businesses are capitalizing on these opportunities, others may still be struggling to understand the landscape.

Not sure if the mobile market is a worthwhile investment? A quick look at a few market statistics shows the undeniable importance of mobile. There are 312 million people in the United States and 328 million mobile subscriptions – meaning there are more devices active on the mobile networks in this country than people. If we limit this down to just data enabled devices, there are still 278 million – with smartphones and tablets comprise over one third of this number. Data traffic on mobile providers networks increased by over 110 percent last year, and is anticipated to grow even more dramatically this year. Consumer spending and purchasing behaviors via mobile devices continue to dramatically increase each year. Mobile device consumerism is growing and represents a significant opportunity.

How do you get started? It’s critical to avoid a common pitfall of jumping right to the technology. Taking the next step into the mobile space is a business project – not a technical project. This is an important difference as it reflects a new method for connecting with customers, and not simply a new piece of software. As with any new business venture, it is critical to start with an understanding of the customer – their needs, wants, capabilities, common practices, etc. Next, understand the business’s objectives and goals. Once you understand your customers and your business, then you can formulate an overall strategy of how to best integrate the two. Finally, the strategy is translated into the tactical plan of how to accomplish these goals, including what technologies to use and how mobile can be engaged to serve the business and customers.

The most common approaches for “going mobile” are mobile websites, mobile apps, and mobile marketing. Each has distinct costs, opportunities, and uses. It is important to recognize there is no expectation that every business should engage in all three – these are highly distinct avenues and not appropriate for all businesses.

Over the coming weeks, we’ll explore these various approaches and opportunities for entering the mobile space. Each will cover the approach, appropriate uses, and benefits associated with the plan. Stay tuned for this four part series on “Going Mobile”.