Difference between revisions of "Lyo/LDPTestSuiteExploration"

Latest revision as of 12:40, 7 April 2014

Page will gather various notes about how a LDP test suite may be implemented, with looking not just at the general needs of LDP server implementations but also how it can be leveraged for OSLC needs.

Options being explored

• TestNG + REST-asured

Sample test:

	@Test(groups = { MUST }, dataProvider = MediaTypeDataProvider.NAME, dataProviderClass = MediaTypeDataProvider.class)
	public void testCreateResource(String mediaType) throws URISyntaxException {
		Model model = ModelFactory.createDefaultModel();
		Resource resource = model.createResource("", model.createResource("http://example.com/ns#Bug"));
		resource.addProperty(model.createProperty("http://example.com/ns#severity"), "High");
		resource.addProperty(DC.title, "A very serious bug.");
		resource.addProperty(DC.description, "Server won't start.");

		Response postResponse = RestAssured
		        .given().contentType(mediaType).body(model, new RdfObjectMapper())
		        .expect().statusCode(HttpStatus.SC_CREATED)
		        .when().post(new URI(containerUri));

		String location = postResponse.getHeader(LOCATION);
		assertNotNull(location);

		// Test it's a valid URI. Throws a URISyntaxException if not.
		new URI(location);
	}

• Frisby.js + jasmine-node + node.js

Sample test:

    frisby.create('MUST: POST new resource in text/turtle')
        .addHeader('Content-Type', 'text/turtle')
        .post(containerURI, null, {body: bugTurtle})
        .expectStatus(201)
        .after(function(err, res, body) {
            createdResourceURI = res.headers['location'];
            expect(createdResourceURI).toBeDefined();
        })
    .toss();

Pros and cons

TestNG

Pros:

• Tests can be organized into groups (for instance, MUST/SHOULD/MAY or by capability like "Container Tests" or "Paging")
  • Groups can be included or excluded for test runs
  • Groups can be used in reporting
• Tests can easily be parameterized (for instance, by media type)
• Flexible and extensible reporting of results
• Jena provides a common API for dealing with various RDF media types (Turtle, JSON-LD, RDF/XML)
• IDE integration

Cons:

• Tests are pass/fail. No native support for "warnings," for instance if a SHOULD or MAY test fails, although it is possible to skip tests either using annotations or at runtime with SkipException.

Frisby.js + jasmine-node

Pros:

• Lighter weight. More concise code
• Might be more interesting to community

Cons:

• Less flexible reporting
• Difficult to chain HTTP calls in a single test, for instance GET + PUT (Frisby.js limitation)
• Tests are pass/fail. No support for "warnings."

Still to explore:

• Debugging test
• RDF libraries: alternatives to rdfstore-js or N3.js?
• Generating HTML reports with ant and jasmine-node --junitreport

References

• W3C LDP Test cases
• W3C LDP Testing page