Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Lyo/LDPTestSuiteExploration"

< Lyo
(Pros and cons)
Line 50: Line 50:
 
''Pros:''
 
''Pros:''
  
* Tests can be organized into groups (e.g. MUST/SHOULD/MAY or by capability like "Container Tests" or "Paging")  
+
* Tests can be organized into groups (for instance, MUST/SHOULD/MAY or by capability like "Container Tests" or "Paging")  
 
** Groups can be included or excluded for test runs
 
** Groups can be included or excluded for test runs
 
** Groups can be used in reporting
 
** Groups can be used in reporting
* Tests can be parameterized
+
* Tests can easily be parameterized (for instance, by media type)
 
* Flexible and extensible reporting of results
 
* Flexible and extensible reporting of results
 
* Jena provides a common API for dealing with various RDF media types (Turtle, JSON-LD, RDF/XML)
 
* Jena provides a common API for dealing with various RDF media types (Turtle, JSON-LD, RDF/XML)
Line 62: Line 62:
 
* Tests are pass/fail. No native support for "warnings," for instance if a SHOULD or MAY test fails.
 
* Tests are pass/fail. No native support for "warnings," for instance if a SHOULD or MAY test fails.
  
'''Frisby.js'''
+
'''Frisby.js + jasmine-node'''
  
* ...
+
''Pros:''
  
 +
* Lighter weight
 +
* Might be more interesting to community
 +
 +
''Cons:''
 +
 +
* Less flexible reporting
 +
* Difficult to chain HTTP calls in a single test, for instance GET + PUT (Frisby.js limitation)
 +
* Tests are pass/fail
 +
 +
''Still to explore:''
 +
 +
* Debugging test
 +
* RDF libraries: alternatives to rdfstore-js or N3.js ?
 +
* Generating HTML reports with ant and jasmine-node JUnit report option
  
 
== References ==
 
== References ==
 
* [https://dvcs.w3.org/hg/ldpwg/raw-file/b3683634c29f/Test%20Cases/LDP%20Test%20Cases.html W3C LDP Test cases]
 
* [https://dvcs.w3.org/hg/ldpwg/raw-file/b3683634c29f/Test%20Cases/LDP%20Test%20Cases.html W3C LDP Test cases]
 
* [http://www.w3.org/2012/ldp/wiki/Testing W3C LDP Testing page]
 
* [http://www.w3.org/2012/ldp/wiki/Testing W3C LDP Testing page]

Revision as of 11:25, 1 April 2014

Page will gather various notes about how a LDP test suite may be implemented, with looking not just at the general needs of LDP server implementations but also how it can be leveraged for OSLC needs.

Options being explored

Sample test:

	@Test(groups = { MUST }, dataProvider = MediaTypeDataProvider.NAME, dataProviderClass = MediaTypeDataProvider.class)
	public void testCreateResource(String mediaType) throws URISyntaxException {
		Model model = ModelFactory.createDefaultModel();
		Resource resource = model.createResource("", model.createResource("http://example.com/ns#Bug"));
		resource.addProperty(model.createProperty("http://example.com/ns#severity"), "High");
		resource.addProperty(DC.title, "A very serious bug.");
		resource.addProperty(DC.description, "Server won't start.");

		Response postResponse = RestAssured
		        .given().contentType(mediaType).body(model, new RdfObjectMapper())
		        .expect().statusCode(HttpStatus.SC_CREATED)
		        .when().post(new URI(containerUri));

		String location = postResponse.getHeader(LOCATION);
		assertNotNull(location);

		// Test it's a valid URI. Throws a URISyntaxException if not.
		new URI(location);
	}

Sample test:

    frisby.create('MUST: POST new resource in text/turtle')
        .addHeader('Content-Type', 'text/turtle')
        .post(containerURI, null, {body: bugTurtle})
        .expectStatus(201)
        .after(function(err, res, body) {
            createdResourceURI = res.headers['location'];
            expect(createdResourceURI).toBeDefined();
        })
    .toss();

Pros and cons

TestNG

Pros:

  • Tests can be organized into groups (for instance, MUST/SHOULD/MAY or by capability like "Container Tests" or "Paging")
    • Groups can be included or excluded for test runs
    • Groups can be used in reporting
  • Tests can easily be parameterized (for instance, by media type)
  • Flexible and extensible reporting of results
  • Jena provides a common API for dealing with various RDF media types (Turtle, JSON-LD, RDF/XML)
  • IDE Integration

Cons:

  • Tests are pass/fail. No native support for "warnings," for instance if a SHOULD or MAY test fails.

Frisby.js + jasmine-node

Pros:

  • Lighter weight
  • Might be more interesting to community

Cons:

  • Less flexible reporting
  • Difficult to chain HTTP calls in a single test, for instance GET + PUT (Frisby.js limitation)
  • Tests are pass/fail

Still to explore:

  • Debugging test
  • RDF libraries: alternatives to rdfstore-js or N3.js ?
  • Generating HTML reports with ant and jasmine-node JUnit report option

References

Back to the top