Testing that the API accepts the promised sequence of calls

Randall Maas 6/15/2010 10:33:53 AM

This entry is about a body of checks to verify that each API accepts the sequences of calls promised by the regex description. This is done by turning each element of the regular expression into its own test procedure. Each of the five regular expression constructs has its own format for the procedure:

Sequences of the constructs below
Matching an API call
|,Alternation.
?,Optional.
+,One or more. This is translated into an alternation. Note: the test method cheats since it can't generate an infinite number of calls to a routine. Instead it checks that the routine can be called once and twice in the test pattern.
*,0 or more. This is translated into an alternation. Note: the test method cheats since it can't generate an infinite number of calls to a routine. Instead it checks that the routine can be called once and twice in the test pattern.

Sequences, and Matching an API call

Matching an API calls is the best place to introduce the new concepts. Each element of a sequence is turned into a procedure like the data, and is woven into each test procedure. The code for an individual match node has the following structure:

void __fastcall TD1(PCall_t* next)
{
   PCall_t PNext={next,TD2};
   T_Procedure(&PNext);
}

Each of the regex's test procedure is named TDnumber. (These test procedures employ a different name format to distinguish between them and the procedures that test calling parameters.) The testing of the API regex will call the procedure calling parameter tests.

The next variable is a pointer to the next test procedure in the overall (nested) sequence to call. The first thing done, in a sequence procedure, is to prepend a pointer to the next test procedure in the sequence on the list. Then it calls the test procedure for the procedure under test (if the element of the sequence is a procedure name), or the test procedure for the regex node (if it is a nested regex).

For example if we are testing the sequence (A B) C:

The first procedure will be passed a null next.

C

Then it will call the procedure that tests the sequence A B .
This second procedure will prepend the test for B onto the list.
Then it will can the test for A, which in turn will resume by calling B and so on.

| Alternation

Alternation it is very similar to the calling parameter test procedure, with the management of the next call chain added in:

void __fastcall TD2(PCall_t* next)
{
   PCall_t PNext;
   void* BaseState = BrdState_save();
   TraceNode_t* OT = Trace_tail;

   // call the first alternative
   // first set up pointer to the next item after the alternative
   PNext.next=next;
   PNext.Proc=TD3;
   T_procedure ();
   Trace_freeTo(OT);
   BrdState_set(BaseState);

   // repeat for each of the other alternative

   BrdState_free(BaseState);
}

In other words, the procedure saves the state, tries an alternative, restores the state, tries another alternative, restores the state, and so on.

Quantification

The quantification nodes - ?,+,* - are all translated into alternation nodes, and the code is generated.

+ One or More. The construct Z+ is converted into (Z Z) | Z, and the code is generated as described above.

?, Optional, Zero or One.. The construct Z? is converted into Z |. The empty match in the alternation is a case where the generated code inserts a call to node after the Z?. The code is generated as described earlier (with this exception).

*, Many, Zero or More.. The construct Z* is converted into ZZ| Z |.

Next time

That is as far as I got with in developing the tool. It has take me longer to write up the notes than it took to write the software. (Originally it took a full week of work, according to my notes)

In the future I'll describe some of the ideas I had for testing, again related to regular expressions of the API, on checking to see if it is even possible to make a bad sequence of calls.