Newer
Older
[](./README.md)
[](README.de.md)
Providing combined offline validation of eforms-EU and eForms-DE schematron (.sch) rules. Additionally schema (.xsd) validation is included and some eForms-EU rule errors are left out via blacklist.
General process of validation:
1. eForms-EU schematron of matching version are validated
2. Triggered errors from blacklisted rules are left out
3. eForms-DE schematron of matching version are validated
4. Combined validation result (valid / not-valid) including eforms-EU and eforms-DE errors and warnings is returned
5. It is a possibility to change the default validation phase in the national schematron rules with application property eforms-validator.api.de_schematron_phase
The current Blacklist for each DE version can be found here: [ted-excluded-rules.txt]
(src/main/resources/schematron/de/native-validation/{eformsVersion}/ted-excluded-rules.txt)
- [Java 11](https://openjdk.java.net/projects/jdk/11/)
## Supported Versions
- eForms-DE
Hardware requirements are heavily dependent on how many schematron versions are loaded
concurrently. The below-mentioned numbers are only valid for only eforms-de 1.0.1.
## Building
The project is built by gradle and a gradle wrapper is part of this project, you do not need to install it locally on your machine.
### Build Thin Jar
This includes the application itself and all direct dependencies of the application. To run it locally, Java still needs to be installed in your environment. The build command is the following:
The fat jar includes the application itself and **all** needed dependencies. Java does not need
to be installed in your local environment to run it. The build command is the following:
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
```
import com.nortal.efafhb.eforms.validator.validation.ValidatorService;
@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
@Produces(MediaType.APPLICATION_JSON)
public Response validate(@MultipartForm @Valid ValidatorRequestDTO validatorRequestDTO) {
ValidationRequestDTO validationRequestDTO = convertToValidationRequestDTO(validatorRequestDTO);
return Response.ok(validatorService.validate(validationRequestDTO)).build();
}
```
DTO usage
```
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Pattern;
import javax.ws.rs.FormParam;
import javax.ws.rs.core.MediaType;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.jboss.resteasy.annotations.providers.multipart.PartType;
@Builder
@Getter
@Setter
@NoArgsConstructor
public class ValidatorRequestDTO {
@NotNull(message = "Version must not be null")
@FormParam("eformsVersion")
private @Pattern(regexp = "^[\\d]\\.[\\d]$", message = "BAD_EFORM_VERSION_FORMAT") String version;
@NotNull(message = "Sdk type must not be null.")
@FormParam("sdkType")
private String sdkType;
@NotNull(message = "EForms must not be null")
@FormParam("eforms")
@PartType(MediaType.APPLICATION_OCTET_STREAM)
private byte[] eforms;
}
```
Additionally to service or dependency, this eForms Validator also provides a RESTful API endpoint. It accepts input in the form of a multipart form data with necessary parameters and returns validation results in JSON format.
Create the Quarkus runnable `.jar` archive under `target/quarkus-app/app-runner.jar`
```shell
./mvnw clean package -DskipTests
```
Start the service as Quarkus application
```shell
java -jar target/quarkus-app/app-runner.jar
Endpoint URL: `/v1/eforms-validation`
Method: `POST`
Consumes: `multipart/form-data`
Produces: `application/json`
#### Request Parameters
The request should be a multipart form data containing the following parameters:
- `sdkType`: The type of the SDK used for eForms (String).
- `eformsVersion`: The version of the eForms Standard used (String).
- `eforms`: The eForms data to be validated (XML file) (String($binary)).
#### Response
Upon successful validation, the service will respond with a JSON object containing the validation results. The response will have an HTTP status code of 200 (OK). The format of the response will be specified by the `ValidatorService` used.
"valid": false,
"validatedEformsVersion": "eforms-de-1.0.1",
"warnings": [],
"errors": [
{
"type": "SCHEMATRON",
"description": null,
"rule": "[SR-DE-1 ]The value eforms-de-1.1 of cbc:CustomizationID must be equal to the current version (eforms-de-1.0) of the eForms-DE Standard.",
"ruleContent": "text() = concat('eforms-de-', '1.0')",
"path": "/can:ContractAwardNotice/cbc:CustomizationID"
}
]
## Validation Mode (XSLT vs. XPath)
Validation of schematron rules is provided based on APIs of dependency `ph-schematron-pure` (based on XPath expressions) and `ph-schematron-xslt`
(based on static XSLT files pre-generated from Schematron rules).
XPath validation is the default mode, XSLT validation can be enabled by setting property `eforms-validator.api.enable_xslt_validation_mode`
to `true` in [application.properties](./src/main/resources/application.properties)
or by specifying runtime environment variable `APP_API_ENABLE_XSLT_VALIDATION_MODE=true`.
## Pre-generating XSLT files
XSLT validation is based on static `.xslt` files. These files are expected to exist - and will be looked up - in the applications classpath during application startup when XSLT validation was enabled.
The `.xslt` files are generated from the Schematron files (`.sch`) - which can be found under the Schematron resource base directory `src/main/resources/schematron` - using the `ph-schematron-maven-plugin`.
The plugin is configured to write the `.xslt` file to the source directory where the Schematron files are located.
(Please review plugin configuration inside [pom.xml](./pom.xml) for details)
## XSLT Generation for newly added schematron files
When introducing a new version and generating the `.xslt` file, ensure to review rules with element `cbc:ElectronicMail`
and update the regex to include `+([A-Za-z]{2,18})$` instead of `+([A-Za-z]{2,})$` for correct `.xslt` generation.
Previously this case was present in files `validation-stage-3b.sch` in eu native-validation.
A generated `.xslt` file and its source Schematron files need to be _in-sync_ at all times, so re-generating the `.xslt` is crucial when a `.sch` file
is added, removed or edited.
A Bash script [generate_schematron_md5.sh](./generate_schematron_md5.sh) is utilized to create a plain-text file named `schematron.md5` which is basically
a flat record of the MD5 checksum of each `.sch` file from the Schematron resource directory.
Running [generate_schematron_md5.sh](./generate_schematron_md5.sh) script does not always work for everybody.
If you encounter any issues, try running the script from WSL (Windows Subsystem for Linux). Java must be installed on the system.
If you run in WSL and get the error `./mvnw: /bin/sh^M: bad interpreter: No such file or directory`, run `sed -i 's/\r//' mvnw` and then again `./check_schematron_xslt.sh`
```
NOTE: `schematron.md5` must _never_ be removed or updated manually!
```
Another Bash script ([check_schematron_xslt.sh](./check_schematron_xslt.sh)) will compare the MD5 checksums with the previously recorded MD5 checkums. For each differing or missing MD5 checksum the base directory containing the presumably offending `.sch` is calculated and the re-generation of the `.xslt` file is automatically executed by invoking the `ph-schematron-maven-plugin` with its parameters adjusted accordingly.
After all (relevant) `.xslt` were updated, the `schematron.md5` record is updated automatically
to reflect the latest state.
*Note:* `check_schematron_xslt.sh` is also executed during a CI Pipeline run. (Please review ci configuration inside [.gitlab-ci.yml](./.gitlab-ci.yml) for details). When detecting different or missing MD5 checksums the script will exit with an error code and the commands required to invoke to create a valid state are printed to _stdout_.
## Resolving Issues with *.sch Files and Line Endings
If you encounter issues where the *.sch files on your local system differ from those in the repository,
this may be caused by line-ending discrepancies during code pull operations. Here's how you can resolve this:
1. Check Git Configuration for Line Endings: Use the following command to list your current Git configuration:
```shell
git config --get core.autocrlf
```
Ensure that the core.autocrlf setting is appropriately configured to prevent unwanted line-ending changes.
In this case, setting it to `false` ensures no line feed conversion during pull.
```shell
git config core.autocrlf false
```
2. Re-clone the Repository: Once the configuration is updated, it may be necessary to re-clone the project into a new directory and pull the latest changes to avoid inconsistencies:
```shell
git clone <repository-url>
```
## Git-Hooks (pre-commit)
Execute the following command to install a custom `pre-commit` Git hook to your `.git/hooks` directory.
````bash
./git_hooks/install_git_hooks.sh
````
The `pre-commit` git hook makes sure that the MD5 record file `schematron.md5` and the Bash
scripts working with it are not deleted by accident through a commit and will also
execute `check_schematron_xslt.sh` before pushing changes to the remote repository.
If you run the application in dev mode, you edit the source code and
any changes made will immediately be reflected in the running instance,
means you don't have to restart it.
You can run the application in dev mode that enables live coding using:
## Unit-Tests
Please write Unit-Tests using *Junit 5*.
API-Tests are asserted using *RestAssured*
If you annotate your tests with *@QuarkusTest*, Gradle will launch the application in Dev-Mode
once you trigger the tests, either via your IDE or from CLI.
Consult the [Quarkus-Guide](https://quarkus.io/guides/getting-started-testing) for detailed information regarding testing quarkus applications.
You can trigger the unit test suite using:
```shell script
```
## Code Coverage
Code coverage is measured with [Jacoco](https://github.com/jacoco/jacoco) and sent to [Sonar](https://www.sonarqube.org/).
To keep track of code coverage while developing you can spin up a local sonar instance.
```shell
docker-compose -f docker-compose.yaml up -d
```
After you ran any tests, run a sonar check using:
```shell script
```
Check the sonar report launching `http://localhost:9000`.
Lookup credentials to access the sonar dashboard can be found in `pom.xml`.
The Gitlab pipeline ist defined in `.gitlab-ci.yml`.
We welcome contributions to improve and enhance the functionality of eForms Validator - Core. If you encounter any issues or have suggestions for improvement, please feel free to create a pull request or raise an issue.
eForms Validator - Core is licensed under the Apache License, Version 2.0. You are free to use, modify, and distribute the code, subject to the terms of the license.