Doc CIDashboard

How to use the CI-Dashboard#

In KnowWE, continuous integration facilitates the automated testing and building of knowledge bases. Since, we provide a distributed knowledge base development, it is important to install simple mechanisms to automatically control the quality of the knowledge base after every change or on demand.

Installation#

The continuous integration plugin is installed by

%%CIDashboard
  @name: <dashboard name>
  @trigger: <onSave/onDemand> <article for onSave trigger>
  @test: <testname> <test object> <parameters (if required)...>
%

Please use the annotations of the plugin as follows:

Annotation Mandatory Description
name yes Specify the name of the dashboard
trigger yes Controls the time, when the integration is executed: onSave executes a new integration after any save action in the wiki (recommended) and onDemand only executes the integration when the user explicitly starts it in the tool menu of the continuous integration plugin (only recommended for large knowledge bases and slower computers).
test optional Specify the name of the test and its parameters. To define more than one test, you may use the test annotation multiple times.

Available Tests#

In this installation the following tests are available. However, new tests can be added by new plugins.

Name of TestTest ObjectDescription
ArticleContainsArticleSynopsis: @test ArticleContains «Article» «SearchString»

Checks, whether the articles contain the text defined by the search string.

Parameter Details:

  • «SearchString»:
    Specifies the string or pattern (regex) that will be searched. Any found occurrence within the test object will be considered as test failure.
ArticleHasErrorsArticleSynopsis: @test ArticleHasErrors «Article»

Checks, that the specified articles reports no compile errors.

This test also allows to specify items to be ignored:

  • «allowed-message-regex» (optional):
    Specify regular expression of messages that are ignored by this test
ArticleHasMarkupErrorsArticleSynopsis: @test ArticleHasMarkupErrors «Article»

CI Test that tests whether an article consist of markup(s) with errors in it. This encompasses missing '%' at the beginning and end as well as nested markups

This test also allows to specify items to be ignored:

  • «article-title» (optional):
    Specify articles that are ignored by this test
ArticleHasWarningsArticleSynopsis: @test ArticleHasWarnings «Article»

Checks, that the specified articles reports no compile warnings.

This test also allows to specify items to be ignored:

  • «allowed-message-regex» (optional):
    Specify regular expression of messages that are ignored by this test
CompilerHasErrorsObjectSynopsis: @test CompilerHasErrors «Object»

Checks, that the specified compiler reports no compile errors.

This test also allows to specify items to be ignored:

  • «allowed-message-regex» (optional):
    Specify regular expression of messages that are ignored by this test
CompilerHasWarningsObjectSynopsis: @test CompilerHasWarnings «Object»

Checks, that the specified compiler reports no compile warnings.

This test also allows to specify items to be ignored:

  • «allowed-message-regex» (optional):
    Specify regular expression of messages that are ignored by this test
DashboardMetaCheckArticleSynopsis: @test DashboardMetaCheck «Article»

Checks all other Dashboards for failed tests.

DiaFlux:AbnormalityKnowledgeBaseSynopsis: @test DiaFlux:Abnormality «KnowledgeBase» «no-error-solution-name» [«timeout»]

Checks whether paths with only normal (not abnormal) answers always lead to one of the given no-error solutions, as expected. Also checks that paths to error solution have at least one abnormal answer in them. Reports all flow paths that violate this check.

Parameter Details:

  • «no-error-solution-name»:
    The name of the no-error solution.
  • «timeout» (optional):
    timeout in seconds when the abnormality test will be terminated.
DiaFlux:AmbiguousEdgeKnowledgeBaseSynopsis: @test DiaFlux:AmbiguousEdge «KnowledgeBase»

If multiple outgoing edges of a node use the same choice, the edges of this node become ambiguous. In this case the flow may divert into two parallel paths which is often not desired.

This test also allows to specify items to be ignored:

  • «Ignore solution leaves» (optional):
    Set to LEAF_SOLUTION_NODES to ignore those ambiguous edges, that point to leaf solution nodes (meaning solution nodes that don't have additional successors)
DiaFlux:CombinedFlowRuleAbstractionKnowledgeBaseSynopsis: @test DiaFlux:CombinedFlowRuleAbstraction «KnowledgeBase»

Checks whether there is an abstraction question, that is derived by an abstraction rule as well as by a DiaFLux action node - this may cause TMS conflicts.

DiaFlux:CombinedFlowRuleSolutionKnowledgeBaseSynopsis: @test DiaFlux:CombinedFlowRuleSolution «KnowledgeBase»

Checks, whether there is a solution for which states are assigned by a DiaFlux node AND a scoring rule as this might cause truth-maintenance problems.

DiaFlux:ConflictingAssignmentKnowledgeBaseSynopsis: @test DiaFlux:ConflictingAssignment «KnowledgeBase» [«timeout»]

Tries to find a path, where one question is first conditioned by a guard of an edge and later a new value is assigned to this question. The assignment of the new value will retract the path until the previous condition and consequently will yield a infinite reasoning cycle. For this reason, before setting a new value to a question, that was previously conditioned, a snapshot node has to be placed.

Parameter Details:

  • «timeout» (optional):
    timeout in seconds when the search process for conflicting assignments will be terminated.

This test also allows to specify items to be ignored:

  • «ignore objects» (optional):
    Knowledge objects with a name matching this regular expression are excluded from the conflicting assigenment search.
DiaFlux:CyclicFlowKnowledgeBaseSynopsis: @test DiaFlux:CyclicFlow «KnowledgeBase» [«timeout»]

This cyclic flowchart test examines all DiaFlux flowcharts of a given knowledge base for any cyclic dependency. A cyclic dependency is available if there is any node form that you can follow the diagnostic path towards the node itself. If there is any snapshot in between, or any node that repeatedly asks a specified question, the path is not assumed to be cyclic, because after re-answering a question you may follow a different path. Thus you are not 'captured' in the cycle when performing the interview.

Parameter Details:

  • «timeout» (optional):
    timeout in seconds when the search process for cycles will be terminated.

This test also allows to specify items to be ignored:

  • «flowchart»:
    The flowchart a node or edge that shall be ignored in. You must specify the id or name of the flowchart.
  • «node-or-edge»:
    Id of the node or edge that shall be ignored in the flowcharts.
DiaFlux:DuplicatePathsKnowledgeBaseSynopsis: @test DiaFlux:DuplicatePaths «KnowledgeBase» [«min-path-size»] [«timeout»]

This tests searches for duplicate sub-paths within the flowcharts of a knowledge base. A duplicate sub-path is a set of nodes and interconnection edges between the nodes of a flowchart, that is identically found at the same or an other flowchart.

Parameter Details:

  • «min-path-size» (optional):
    Defines the number of nodes a duplicate sub-path must at least consist of, to be considered as a duplicate path. Default value is '2'
  • «timeout» (optional):
    timeout in seconds when the search process for duplicate paths will be terminated.
DiaFlux:InformationGainKnowledgeBaseSynopsis: @test DiaFlux:InformationGain «KnowledgeBase» «min-gain-average» «min-gain-each» [«attach»] [«timeout»]

The information gain test checks every flowchart node that asks a question for the information gain achieved by the particular question. The information gain for each question node is measured by all solutions that can be reached by following the flowchart(s) from the question node and how the number of these solutions is reduced by answering this particular question. Each information gain is compared against the maximum information gain of the node, calculated depending on the number of outgoing edges. You can compare both the average information gain and the particular information gain of each asking node against the specified limits.

Parameter Details:

  • «min-gain-average»:
    specifies the minimum percentage of the possible information gain to be reached by all decisive questions in average, e.g. 0.1 or 10%.
  • «min-gain-each»:
    specifies the minimum percentage of the possible information gain to be reached by each decisive question, e.g. 0.1 or 10%
  • «attach» (optional):
    specifies if the detailed result table of the evaluation shall be attached or not. Default value is 'false'.
  • «timeout» (optional):
    timeout in seconds when the calculation of the average path length will be terminated.

This test also allows to specify items to be ignored:

  • «ignored-node» (optional):
    identifier of nodes to be ignore by this test. Identifier should look like: flow-id#node-id
DiaFlux:InterviewLengthKnowledgeBaseSynopsis: @test DiaFlux:InterviewLength «KnowledgeBase» «max-cost-average» «max-cost-each» [«attach»] [«timeout»]

The interview length test check the cost and number of questions asked during the interview until a solution is reached. You may specify the average interview cost that should not been exceeded in average and a maximum interview cost that should also not been exceeded by the longest interview path. If not specified otherwise using the property COST, the cost per question will default to 1.

Parameter Details:

  • «max-cost-average»:
    specifies the maximum interview cost that should not be exceeded in average.
  • «max-cost-each»:
    specifies the maximum interview cost that should not be exceeded for any particular solution.
  • «attach» (optional):
    specifies if the detailed result table of the evaluation shall be attached or not. Default value is 'false'.
  • «timeout» (optional):
    timeout in seconds when the calculation for interview costs will be terminated.
DiaFlux:MissingEdgeKnowledgeBaseSynopsis: @test DiaFlux:MissingEdge «KnowledgeBase»

Searches for all nodes in the DiaFlux models of a knowledge base, where not all possible values or outgoing edges are connect to other nodes. In this case the flow may stop unexpectedly in between.

This test also allows to specify items to be ignored:

  • «nodes-of-question» (optional):
    Nodes asking for the question with the given id should be ignored.
DiaFlux:NoAutostartKnowledgeBaseSynopsis: @test DiaFlux:NoAutostart «KnowledgeBase»

Tests whether exists an autostart flow in the DiaFlux model.

DiaFlux:StandardQuestionDerivedByAbstractionKnowledgeBaseSynopsis: @test DiaFlux:StandardQuestionDerivedByAbstraction «KnowledgeBase»

Checks whether there exists a standard question (not an abstraction question), that is derived by an abstraction rule or a DiaFlux action node.

DiaFlux:UnusedFlowKnowledgeBaseSynopsis: @test DiaFlux:UnusedFlow «KnowledgeBase»

This test checks whether the knowledge base contains DiaFlux models, that are neither Autostart models nor are called by other DiaFlux models

This test also allows to specify items to be ignored:

  • «flows»:
    A regular expression naming those DiaFlux models to be excluded from the tests.
DiaFlux:UnusedNodeKnowledgeBaseSynopsis: @test DiaFlux:UnusedNode «KnowledgeBase»

Looks for non-connected start or exit nodes in DiaFlux models.

EmptyQuestionnaireKnowledgeBaseSynopsis: @test EmptyQuestionnaire «KnowledgeBase»

Tests whether the knowledge base has questionnaires that do not contain any questions or other questionnaires.

This test also allows to specify items to be ignored:

  • «questionnaire»:
    A regular expression naming those d3web qcontainers to be excluded from the tests.
FunctionalPropertyObjectSynopsis: @test FunctionalProperty «Object»

Checks for all existing functional property whether they are indeed used as functional, i.e., for each subject node only one object exists.

IncompleteChoiceQuestionKnowledgeBaseSynopsis: @test IncompleteChoiceQuestion «KnowledgeBase»

Looks for choice questions that have no choice alternatives defined yet.

SingleAlternativeKnowledgeBaseSynopsis: @test SingleAlternative «KnowledgeBase»

A simple test checking the knowledge base for choice question with only one alternative. You can specify a list of choices that are accepted to be the single alternative of a question, e.g. "ok" or "continue".

This test also allows to specify items to be ignored:

  • «Accepted single alternatives» (optional):
    Specify a list of names or prompts for single alternatives, that are ok and should be ignored by this test.
SingleTermDefinitionArticleSynopsis: @test SingleTermDefinition «Article»

This test checks, if each term is defined only once.

This test also allows to specify items to be ignored:

  • «terms» (optional):
    Term names to ignore for this test.
SparqlAskObjectSynopsis: @test SparqlAsk «Object» «expected value» [«warning»]

Checks the truth value of given sparql ask query against an expected value.

Parameter Details:

  • «expected value»:
    expected boolean value to compare with
  • «warning» (optional):
    show warning instead of failure if this test fails
SparqlResultSizeObjectSynopsis: @test SparqlResultSize «Object» «comparator» «expected size» [«warning»]

Checks the size (amount of rows) of a given sparql query result against an expected value.

Parameter Details:

  • «comparator»:
    how to compare the result size against the expected size
  • «expected size»:
    expected size to compare with
  • «warning» (optional):
    show warning instead of failure if this test fails
SparqlResultTableObjectSynopsis: @test SparqlResultTable «Object» [«comparator»] [«warning»]

Test a expected sparql result set against the result set of the actually executed sparql query.

Parameter Details:

  • «comparator» (optional):
    how to compare the result data against the expected table data: equal/atleast
  • «warning» (optional):
    show warning instead of failure if this test fails
TermnameConventionArticleSynopsis: @test TermnameConvention «Article» «Naming convention pattern»

Tests whether the objects names used on this article comply to the specified naming convention pattern.

Parameter Details:

  • «Naming convention pattern»:
    A regular expression for the desired naming convention. All terms are checked against this expression.
TestCaseTestCaseSynopsis: @test TestCase «TestCase» «KnowledgeBase» [«ValueOutOfRange»]

This test executes test cases. It compares the expected findings defined in the test cases with the findings actually derived by the knowledge base.

Parameter Details:

  • «KnowledgeBase»:
    Specifies the knowledge base with which the test case is to be tested.
  • «ValueOutOfRange» (optional):
    Specifies whether values out of the defined ranges should be skipped or not. By default they are not ignored.
URIPatternTestObjectSynopsis: @test URIPatternTest «Object»

Checks for all Terms (URIs) whether they follow best practice URL patterns.

This test also allows to specify items to be ignored:

  • «Ignore-Pattern»:
    Specifies a regular expression of all terms to be ignored. Please note that the term is similar to the abbreviated uri usedin turtle markups, but uses '#' as a separator.

Example#

%%CIDashboard
  @name: MyArticle CI Test
  @trigger: onSave MyArticle
  @test: EmptyQuestionnaireTest MyArticle
  @test: IsolatedInterfaceNodeInDiaFluxModel MyArticle
  @test: NoAutostartFlowCITest MyArticle
  @test: StandardQuestionDerivedByAbstraction MyArticle
  @test: CombinedDiaFluxRuleAbstraction MyArticle
  @test: StandardQuestionDerivedByAbstraction MyArticle
  @test: IncompleteChoiceQuestion MyArticle
%

Looks like:

Screenshot of a CIDashboard

Related Doc Pages#