User Manual
WIP
Go to the overview page of all products - Lucy - Products
Search if your product has already been created. If yes, then open the detail view of the product, otherwise create a new product via the "Create a new Product" button and fill all necessary fields as needed.
-
Go to your Lucy -Products and choose the product where you want to upload the BOM
-
Click on the green "Upload" button, choose the CSV you have downloaded and then click on the blue "Upload" button. The upload status will change to "Processing". Wait and check until it is back to "Successful" to see the new libraries from the upload.
Upload Status:
- Successful: Lucy finished processing the upload.
- Processing: Lucy is still processing the upload. Wait until is finished (if it takes longer than 10 minutes, then an error may have occurred).
- Failure: Lucy could not read and process the upload. Check log files for more information.
You can use the "Reload" button to check if the upload status and libraries has changed.
Download the "Publish" OSS list, License Text Archive and the "Delivery" Source Code Archive from the "Reports" tab.
License Text Archive
- Unpack the License Text Archive and check if the HTML is displayed correctly and the links work.
- Check if the information in the HTML are correct. Like the name of the product, version and disclaimer.
License Text Archive and "Publish" OSS List
- License column should not contain "Unknown" or "Non-Licensed" licenses.
- License column should not contain toxic licenses (e.g. Strong Copyleft or Forbidden licenses).
Source Code Archive
- In the source code archive, you can see the libraries that have a license set with the "Share Source" requirement.
After the product has been finally evaluated, the archives and files must be transferred to our servers for historisation.
For the source code archive the "Download" option can be used:
Edit the product and check the "Delivered" field:
Create a new version of the same product in Lucy. It is no need to copy other fields from the previous version, only to set the new version of the next release.
Open the detail view of a library. If it contains wrong information, click on "Edit" and change the respective fields. Save your changes with the "Save" button.
Library with "Unknown" license
- Search for the library and check if the license could not be correctly identified from the "Original License" field or it is a complete new license.
- If it could not be correctly identified then "Edit" this library and set the correct license manually.
- The "License" and "License To Publish" field should be set with the correct license(s).
- If it is a new license then go to the "License" entity and create a new license with the correct information.
- "Edit" the library and set the new license for the "License" and "License To Publish" field.
Library with "Non-Licensed" license
- Search manually for the correct license (e.g., Google, Github, Fossology).
- Create a new license if necessary and set the license for the "License" and "License To Publish" field.
License
- Are three different license fields for a library:
- License: These are the licenses of the whole library. Mostly defined by the copyright holder.Here we have and specify licenses with "OR" and "AND" links.
- License To Publish: These are the licenses you want to publish in your reports. In case of "OR" links the license with the lowest risk is selected. Only "AND" linked licenses can be specified.
- License Of Files: These are licenses found through a deep scan with different tools (e.g. Fossology, FossID) or manually in the source code archive.
- Some files in the library can be under a different license.
Auto-completion of License field
- The "License" field is automatically completed by the "Original License" after an upload.
- Licenses that could not be detected automatically are indicated as "Unknown" licenses.
- Different license names must be mapped using the License Naming Mapping list and adjust the list, if licenses cannot be identified correctly.
- The "License To Publish" field is automatically completed from the "license" field if no license has been set for it yet.
- The license with the lowest risk is always selected for "OR" links.
License URL
- The "License URL" field is intended for the link to the license text. This should also be about the specific license text of a library.
- If the link to a specific version is not available, then the link to the general version or to the previous or next version can be entered (e.g. github master branch).
- Here it must be ensured that the license has not changed in different versions. Multiple links can be entered separated with a ",".
- The license URL will be used by the "License Text" field.
Auto-completion of License URL field
- If the Library Type is set and the field itself is empty, then it will automatically try to find the correct URL after saving. If no URL can be found, then "No URL found" is set.
- Auto-completion is working for the following Library Types: github, golang, maven, npm, nuget, pypi.
License Text
- The "License Text" field contains the complete license text, which must be published, of the library.
- The license texts of different licenses are appended under each other. Multiple "=" characters should be used as separator.
- The license text should be formatted using HTML. The "Preview" field shows the HTML code rendered.
Auto-completion of License Text field
- If the license URL is set and the field itself is empty, then the license text is automatically downloaded from the URL after saving.
- Here you must pay attention which license URL is set, because the complete HTML page is always downloaded.
- For Github URLs it is sufficient if the link points to a specific file (e.g. License.txt), because for Github there is already the support that only the content of the file is downloaded.
- For other platforms, such as Bitbucket or Gitlab, the URL to the "raw" content should be stored in the best case. Otherwise, the downloaded content should be manually adjusted afterwards.
Source Code URL
- The "Source Code URL" field is the link to the downloadable source code archive. You should make sure that the URL is the link to the archive with the uncompiled files.
- The URL must be the direct download of the archive, so when you click on the link, the download should start directly.
- This is necessary because this link is also used for creating the "Source Code Archives" for products, otherwise the correct archive will not be downloaded.
Auto-completion of Source Code URL field
- If the Library Type is set and the field itself is empty, then it will automatically try to find the correct URL after saving. If no URL can be found, then "No URL found" is set.
- Auto-completion is working for the following Library Types: github, golang, maven, npm, nuget, pypi.
Comment
- The comment field can be used for any additional information about the library.
- This can be information about modifications, delivery info or explanations about certain decisions etc.
When uploading files, a specific format is required depending on the entity and file type.
Files can be uploaded for a Product, Library or License entity.
The upload page can be reached via "Upload" -> "Create a new Upload".
Product / Library Upload
- When uploading for a product, the product ID for which the upload should apply must be set.
- Libraries that are not yet stored in the database are created and stored with the information from the file. For already existing libraries, if the file has information for fields of a library that are empty, then they will be filled.
- Libraries are identified by the unique combination of GroupId, ArtifactId and Version.
- An upload never overwrites already existing information!
- The upload for the library entity is identical to that for a product except that no product ID needs to be defined.
- Libraries are created and empty fields are completed.
The upload supports different file formats:
- CSV - Artifact Id and Version are mandatory fields
- XML (created by a CycloneDX plugin)
- JSON (created by Lucy)
License Upload
- When uploading licenses, licenses that are not present in the database are created or licenses with missing information are completed.
- If missing information of a license should be completed, then the ShortIdentifier must be identical with the value as stored in the database.
- An upload never overwrites existing information, except for Requirements.
- A license always contains exactly the requirements that were defined in the file.
The upload supports different file formats:
- CSV
- JSON (created by Lucy)
Lucy can start scans of libraries via Fossology's API. For this, the source code (URL) must be stored in the library.
After starting the scan in Lucy, a progress bar can be seen, which automatically updates the status of the scan in Fossology.
After successful completion, you can navigate directly to the result via the link in Link.
Configuration
- In order to enable and use Fossology in Lucy, the docker-compose.yml must be adapted.
- The following keys are provided for the Fossology integration:
API Token
- In order for Lucy to communicate with Fossology, a token is created in Fossology and stored in Lucy. In Lucy, the token is inserted in the docker-compose.yml file.
- The key for which the token must be inserted is called: APPLICATION_FOSSOLOGY_TOKEN=
- In Fossology, you can only create a token for a validity period of one month. This means it must be updated manually on a regular basis!
Guide
- Login to Fossology as the user "lucy".
- Go to "Edit User Account".
- Create a new token. The name of the token cannot be the same as a expired token.
- Set the expiration date to the last possible date and scope to "Read/Write access".
- After the creation of the token copy the value and edit the docker-compose.yml of Lucy.
- Execute the following commands:
Logging
- Lucy logs all possible events that happen in the background.
- To keep the logs clear, only the most important messages are logged in productive mode.
- In the admin area under Logs the log level of the different loggers can be configured to get more or less messages.
Location
- The log files are in the directory: /path/to/lucy/logs
- Lucy creates a new log file for each day. The logs are historised for a maximum of 30 days. Older logs are automatically deleted.
- Two different log files are created. One is the main log file: logFile.timestamp.log and the other is the log file for the source code archive in the logs/archiveLogs subdirectory: archivelogFile.timestamp.log
- The path can be changed in the docker-compose.yml file:
Lucy sends emails in the following situations:
* Creating a new account. The email contains a link to activate the account.* Forgotten password. The email contains a link to a page where you can change your password.
For sending emails to work, a mail server must be configured. The following parameters must be adjusted in the docker-compose.yml file:
Liquibase
- Exception while starting the application: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by ...
- If the application is abruptly stopped, then it can happen that the lock remains stuck.
- Solution: Connect to the database and execute the following command:
