
Newton Technology Journal June 1996
23
The change should be code-reviewed by another development engineer.
The developer making the change should write a release note about the
change, its impact on other areas of the system, and important tests to run.
SQA should verify and test the change and any related areas.
Bugs
Since you can’t test everything or use the product in every way that a real
customer might, you won’t find every bug, and you’ll almost certainly ship
with known bugs. Make sure you prioritize the bugs to fix based upon
their importance to the customer. If a bug happens in only obscure cases
it may not be worth fixing. Fix the bugs and usability problems that will
affect the majority of users, or which might cause bad press. Sometimes
you’ll guess wrong. Keep track of the bugs you defer and see if real
customers report them, then escalate them to fix in the next version.
• Customer Feedback
Improving product development is based on a feedback loop. There are
three main programs for incorporating customer feedback at different parts of
the product cycle: user testing, beta testing, and (uh oh) customer support.
User Testing
Bringing in target users to give feedback on your product early in the
development cycle can give you data about the intuitiveness of the design
and validate whether your feature set is the right one. SQA engineers can
also improve their tests by watching how real customers use the product.
Fresh perspectives are always valuable, because the development team
becomes too close to the product.
Beta Testing
Software used in real-world situations often provides some of the best
feedback and bugs. During the Newton 2.0 OS project, most people on
the Newton team carried around flash ROM MessagePads with the latest
software. A lot of the good bug reports came from people using them in
meetings. That was the difference between 1.x and 2.0 – you can actually
take notes in meetings.
Customer Support
Product development doesn’t end when you ship your product.
Customer support quantifies customer reaction, and based upon call
frequency you can prioritize bugs to fix in future revisions. Feature
requests provide a source of good ideas from people actually using the
product. The development of Newton 2.0 OS was largely driven from
customer requests and usability problems, along with inspired design
work from the software engineering team.
SPECIFICS OF NEWTON TESTING
Testing a Mac OS, Windows, or Newton application is all essentially the same.
Take your software development and SQA background and overlay the
technical details of the platform. When testing a Newton application, know
how the Newton works and how it’s architected. The best sources of
information are the user’s manual, Newton Programmer’s Guide (NPG),
and Newton User Interface Guidelines. Applications written for the Newton
plug into a layer of system services. So, besides testing the features of your
software in isolation, you need to test the areas where your application
intersects with these system services.
Some of the important things to test are:
• Make sure the recognition view flags are set for the right kind of input.
• Test data storage and filing on an unlocked and locked card and
internally to validate the soup code.
• Erase your application from extras to test its removeScript and optional
deletionScript. Make sure the application removes itself from various
registries to avoid the “grip of death.”
• Eject a card with application data on it. Make sure the screen is
refreshed.
• Test the differences when your application modifies a built-in prototype.
• If your application supports landscape, make sure all the dialogs fit.
• Test your application’s use of Find – especially if it does something
different like highlight the entry.
• Make your application the backdrop and see if it behaves correctly.
• If you link in custom dictionaries, test them in the various fields.
• Make sure soup change and other notifications get intercepted by your
application.
• Optimize your program’s use of frame and/or system memory.
• Make sure your application doesn’t create global functions unless
absolutely necessary.
• Test with the latest version of the system update. (The easiest way to
remove a system update is to remove all batteries and short the backup
battery terminals with a penny).
• Make sure your application uses screen-relative bounds to remain
compatible with different screen sizes.
• Make sure the user interface of your application conforms to the
guidelines.
• Delete your soup from the storage folder and see if your application
handles it gracefully.
• If your application supports both 1.x and 2.0, test on both platforms.
• Test how your application reacts to its own data on a read-only 1.x and
locked 2.0 formatted card in a 2.0 system.
• Use the Newton Keyboard with your application and make sure the tab
order for views is correct.
• Test any application interfaces exported for public use.
• Test all peripherals you support.
• If your application runs on 1.x and 2.0, make sure it uses
gestalt
to
test for the existence of Newton features.
• If your application modifies the behavior of a built-in application, then
research existing applications and see if anyone else is doing what you’re
doing and document any incompatibilities.
• Make sure your application uses documented APIs or DTS-approved
methods, for example, when accessing built-in application soup data.
This will avoid incompatibilities in the future.
Applying the background and Newton-specific information, the following
five steps provide the framework for testing a Newton application through its
development:
1. Create a very detailed, hierarchical feature list for the product based
upon documentation, prototypes, and communication with
development engineers. Every feature that a user can access should be
Kommentare zu diesen Handbüchern