Q: How can I deal with performance issues that are dependent on both H/W and S/W?

A: These dependencies are sorted out early on in the product definition process or while producing the Test Specification Document ("TSD"). Let's clarify the issue with an example: the locking time of a certain PLL (Phase Lock Loop) embedded into your product may be dependent on temperature and unit-to-unit variations as well as the efficiency of a certain software algorithm. Instead of blindly testing the acquisition time in a unit in conditions that may or may not be representative of the actual operation, it is much better to identify the physical and H/W dependencies and then test them separately from the S/W component.

A: Not really, or at least not directly. This new testing service that we offer refers strictly to the product functionality. What we test and certify is the compliance with your own Marketing requirements (which we might help to formulate in a rigorous format if necessary).

A: Product Testing is more akin to Software Quality Assurance, looked from the stand point of the product functionality as seen by your customers. The agency approvals for safety, network connectivity and radio interference issues are disciplines than although related, they require different kind of skills and tools than Product Testing. The agency approvals involve a complex separate set of legal, administrative and technical issues, often of international scope. We can help you through another specialized group of our associate consultants who are experts in all sorts of agency approvals.

A: Definitely, no. It is highly advisable to develop at least the preliminary version of the Test Specification Document ("TSD") while or even before starting to develop your product. This document can become one of the most valuable references during the design, especially for the software functionality and user interfaces. Having a good TSD can contribute to the efficient code development by making available to the H/W and S/W developers a description of the specific items that will be tested.

A: In the worst case, you can always compile a list of "Known Issues" with their respective workarounds and publish this information with your product release. This is always a better alternative than letting the customer discover the problems in the hard way. Overall it is always less costly to know the issues than to ignore them, even if you are hard pressed to release the product to the field.

A: Black Box testing: Also known as functional testing. A software testing technique whereby the tester concerns about external functionality and features of the product. For example, in a black box test on a software design the tester only knows the inputs and what the expected outcomes should be and not how the program arrives at those outputs. The tester does not ever examine the programming code and does not need any further knowledge of the program other than its specifications. In practice this approach is only partially possible because it quickly

B: White Box testing: Also known as 'glass box', 'structural', 'clear box' and 'open box' testing. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. Unlike black box testing, white box testing uses specific knowledge of programming code to examine outputs. The test is accurate only if the tester knows what the program is supposed to do.

A: Of course, it is not...! In fact, it can be a significant development cost saver. When we are engaged early on in the development process we can definitely save time by use our extensive product experience to produce a preliminary version of the Test Specification Document to be used as a one of the fundamental design documents for your product. A second way that we can save development cost is by acting as a "gate" to avoid premature exposure of your product to the field before it is ready for it. A good Test Plan consists of hundreds or even thousands of Test Cases that reproduce what your users will do with your product.

A: We will be delighted to help you in every product release but we can also train your own engineers to perform the rigorous tests as required by a high quality product. One of the main benefits that we bring to the table is our independent view of the issues. Even though it is very tempting to use your own development engineering manpower to perform the testing, it is psychologically hard for the developers (or even for the development managers) to be as thorough and impartial as a rigorous test might require. Not to mention that the Development and the Testing are two quite different disciplines.

A: We detail the test results as "pass/no pass", qualitatively or quantitatively as appropriate to the nature of each individual test. The report can include the failed passes or only the test results after the corrective actions are performed.

A: The name of "4-corner testing" usually refers to functionality tests performed at the four combinations given by max-min values of temperature and input voltage, which are usually the most important external variables. These may or may not be relevant to product testing and an early determination of functionality dependencies to these parameters can save significant testing time. The "8-corner testing" usually refers to the addition of a third parameter into consideration, which doubles the number of tests to be performed.

A: This refers to the fact that even for a simple product functionality it is virtually and practically impossible to test all the possible combinations of external parameters, input values, settings and internal states. The reason is that the combination of possible test cases would be for all practical purposes equivalent to infinity. Even though this makes the Product Testing a seemingly impossible task, in fact it stresses the point for applying the Product Testing discipline from years of experience. This experience is what allows to choose a reasonable number of test cases to obtain nearly full coverage of the product functionality.

A: Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and his/her overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, the development organization's management/accountants/testers /salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have his/her own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free software.

If you would like more information about these or any other topics on our Product Testing Services, please send e-mail with your Product Testing Assistance Request to or call (510) 792-1760 for a prompt response.