LDT/Developer Area/Testing ASTs

In LDT, we deal with Lua code. Our tool have to analyze code provided by users. In order to do so, we generate ASTs which are data structures representing code. When we change generation process, several problems occur:

• Are the changes effective?
• Do they cause regressions?
• Do they affect performances?

In order to detect failure, we will run Unit test as AST generation smoke tests.

What will we be testing

We are focused on model generation on Lua side. It is the most tricky part of our code. It is also where refactoring is the most dangerous. Testing this area first enable us to be more confident in the critical part of the code.

Create AST tests

Before performing any work on ASTs we might create some.

Create a reference AST

The idea is to load a Lua source file as plain text. Process it like we do for regular source files. The result of this operation is a plain Lua table described parsed source. Now we will serialize this table using Metalua and store it in a file. The content of this file is not going to change, we will use it as reference.

Valid new AST generation

When AST generation will be changed, we will compare newly generated ASTs with reference one. New ASTs will be generated from the same source as the reference one. Then if new and old ASTs are equivalent, AST generation is successful. It is all automatic and convenient to test large files as it is not necessary to describe them manually.

Dependencies

No worries, all testing requirements are already part of our target platform. Serialization will be performed with Metalua, unserialization is done with loadstring, which is a built in Lua function. Unit test framework will be JUnit. AST generation and table comparison will be done using Lua.

Aternatives

While loading strings from Metalua serailization we had some weird behaviors. If it does not meet our expectations we might try Data Dumper or Serpent, which have similar features.

Implementation

Details

Table comparison

We use to relay on a third lib to compare our tables. It may not be a good idea as we got specific table comparison needs. Here is table explaining what we should take for equal for table comparison.

Value type	Equality definition
nil	== operator
number	== operator
string	== operator
function	True, we can test functions as values. The following is valid {f = function () end} == {f=function(n) return n+1 end} .
table	Same attributes value and same attribute count.

Handle cycles

We deal with infinite structures, as some tables can refer at themselves. Testing such structure is infinite. I see two potential solution.

Detect cycles

When could bear in mind table we already analysed and then avoid them. What could be even better while comparing table, is check if both tables have cycles at same structural places.

-- Those tables would be equal
local o, t = {}, {}
o.same = o
t.same = t

Avoid cycles

Let our testing API know about some field names which will be ignored when comparing tables.

What is already done

All java classes which enable to load testing material for Lua are ready. There is also a script enabling to convert Lua source files in serialized API Model files. It is available at scripts/serializeapimodel.lua. There is also a function enabling to compare tables, but it does not behave as described above. It just valid that Java and Lua testing component work well together.

What missing

Mandatory

Ensure that Metalua or Data Dumper handle table self references the way we want it. Implement table comparison as described above.

Improvement

We could test the conversion of our Lua model to Java ones.

We could test if our model generation produces correct results.

We could also test the layout of the documentation embedded in our models.