add readme and exe files for windows
This commit is contained in:
665
README.md
Normal file
665
README.md
Normal file
@@ -0,0 +1,665 @@
|
|||||||
|
|
||||||
|
# Embedded System Test Automation Framework
|
||||||
|
|
||||||
|
This is a C++ application designed for testing embedded systems using UART communication. The system allows you to load and execute test cases that interact with the UART interface according to predefined protocols.
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
- [Overview](#overview)
|
||||||
|
- [Key Features](#key-features)
|
||||||
|
- [Supported Embedded Toolchains](#supported-embedded-toolchains)
|
||||||
|
- [Project Structure](#project-structure)
|
||||||
|
- [Requirements](#requirements)
|
||||||
|
- [Development Environment](#development-environment)
|
||||||
|
- [Build and Deployment](#build-and-deployment)
|
||||||
|
- [Configuration](#configuration)
|
||||||
|
- [Test Stand Settings](#test-stand-settings)
|
||||||
|
- [Test Plan](#test-plan)
|
||||||
|
- [Running the Test Stand](#running-the-test-stand)
|
||||||
|
- [Test Case Syntax](#test-case-syntax-json)
|
||||||
|
- [Processing Text Data Received via UART](#processing-text-data-received-via-uart)
|
||||||
|
- [Processing Binary Data Received via UART](#processing-binary-data-received-via-uart)
|
||||||
|
- [Reporting](#reporting)
|
||||||
|
- [Notes (summary)](#notes)
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Embedded Systems Test Automation Framework is a console-based C++ application for automated testing of embedded systems through UART communication.
|
||||||
|
|
||||||
|
The framework supports:
|
||||||
|
|
||||||
|
* execution of predefined test plans;
|
||||||
|
* transmission of binary commands to the target device;
|
||||||
|
* reception and validation of UART responses (if text format) or parsing of binary C structures directly from the target memory;
|
||||||
|
* generation of text logs and HTML reports;
|
||||||
|
|
||||||
|
The framework is configured entirely through JSON files and can be easily integrated into automated verification environments.
|
||||||
|
|
||||||
|
## Key Features
|
||||||
|
|
||||||
|
- **Binary Command Support**: Send raw `.bin` payloads directly to the microcontroller.
|
||||||
|
- **UART I/O Communication**: UART I/O for interacting with microcontrollers.
|
||||||
|
- **Sequence Validation in text format**: Validates incoming messages by matching specific patterns:
|
||||||
|
- **Start Message**: Initial handshake/trigger.
|
||||||
|
- **Sequence**: Ordered sequence of expected intermediate messages.
|
||||||
|
- **End Message**: Completion criteria for the test case.
|
||||||
|
- **JSON-Driven Configuration**: Easily define test cases, UART parameters, and test plans in JSON format.
|
||||||
|
- **Flexible Execution**:
|
||||||
|
- **Grouping**: Organize tests into suites (e.g., --group=smoke, --group=full).
|
||||||
|
- **Repeats**: Built-in support for stress testing by repeating cases (using --repeat=N).
|
||||||
|
- **Advanced Logging**: Generates detailed text logs and visual HTML reports for every run.
|
||||||
|
|
||||||
|
## Supported Embedded Toolchains
|
||||||
|
|
||||||
|
The framework itself is compiler-independent.
|
||||||
|
|
||||||
|
However, automatic binary structure parsing is currently implemented for firmware generated by:
|
||||||
|
|
||||||
|
* IAR Embedded Workbench
|
||||||
|
* ELF/DWARF compatible toolchains
|
||||||
|
|
||||||
|
The parser extracts:
|
||||||
|
|
||||||
|
* variable addresses;
|
||||||
|
* structure sizes;
|
||||||
|
* nested structures;
|
||||||
|
* array dimensions;
|
||||||
|
* primitive data types.
|
||||||
|
|
||||||
|
The generated metadata are stored as JSON files inside:
|
||||||
|
|
||||||
|
```text
|
||||||
|
./iar_files/
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```text
|
||||||
|
├── cmd
|
||||||
|
│ ├── cmd_abort.bin
|
||||||
|
│ ├── set_interface_params.bin
|
||||||
|
│ ├── set_settings_NAND_micron_MT29F16G08AJADAWP.bin
|
||||||
|
├── compiler
|
||||||
|
│ ├── Makefile
|
||||||
|
│ ├── Makefile.Debug
|
||||||
|
│ ├── Makefile.Release
|
||||||
|
│ ├── data.bin
|
||||||
|
├── config.json
|
||||||
|
├── configs
|
||||||
|
│ ├── cmd
|
||||||
|
│ ├── test_cases
|
||||||
|
│ │ ├── test_UART_cmdSetDataInterface.json
|
||||||
|
│ │ ├── test_abort.json
|
||||||
|
│ │ ├── test_readSnapshot_HK.json
|
||||||
|
│ │ ├── test_read_NANDctrlOper.json
|
||||||
|
│ │ ├── test_set_micron_MT29F16G08AJADAWP.json
|
||||||
|
│ └── test_plan.json
|
||||||
|
├── iar_files
|
||||||
|
│ ├── ENTRY_LIST_map.json
|
||||||
|
│ ├── NANDctrlOper_t.json
|
||||||
|
│ └── Snapshot_HK_t.json
|
||||||
|
├── logs
|
||||||
|
│ ├── 12_05_2026_15_48_16.log
|
||||||
|
│ ├── report.html
|
||||||
|
│ ├── test_cases.log
|
||||||
|
│ └── test_stand.log
|
||||||
|
├── python
|
||||||
|
│ └── iar_parser
|
||||||
|
│ ├── elf_parser.py
|
||||||
|
│ ├── map_parser.py
|
||||||
|
│ ├── requirements.txt
|
||||||
|
├── reports
|
||||||
|
├── src
|
||||||
|
│ ├── core
|
||||||
|
│ │ ├── cli_parser.cpp
|
||||||
|
│ │ ├── cli_parser.h
|
||||||
|
...
|
||||||
|
│ │ ├── uart.cpp
|
||||||
|
│ │ └── uart.h
|
||||||
|
│ ├── embedded_test_stand.pro
|
||||||
|
│ ├── embedded_test_stand.pro.user
|
||||||
|
│ ├── gtest_dependency.pri
|
||||||
|
│ ├── main.cpp
|
||||||
|
│ └── tests
|
||||||
|
│ ├── test_runner.cpp
|
||||||
|
│ ├── uart_fixture.cpp
|
||||||
|
│ └── uart_fixture.h
|
||||||
|
└── src.rar
|
||||||
|
```
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
### Development Environment
|
||||||
|
The project was developed and tested on Windows 10/11 using the following software:
|
||||||
|
|
||||||
|
| Component | Version |
|
||||||
|
|----------|---------|
|
||||||
|
| Qt | 5.15.2 |
|
||||||
|
| Qt Creator | 9.x or newer |
|
||||||
|
| Compiler | MinGW 8.1.0 (64-bit) |
|
||||||
|
| Debugger | `gdb.exe` |
|
||||||
|
| Build system | qmake |
|
||||||
|
| qmake | Qt 5.15.2 |
|
||||||
|
| mingw32-make | MinGW 8.1.0 |
|
||||||
|
| Python | 3.9+ |
|
||||||
|
| GoogleTest | Latest stable version |
|
||||||
|
|
||||||
|
#### Qt Modules
|
||||||
|
|
||||||
|
The following Qt modules are required:
|
||||||
|
|
||||||
|
* QtCore
|
||||||
|
* QtSerialPort
|
||||||
|
|
||||||
|
#### Python Dependencies
|
||||||
|
|
||||||
|
Python scripts located in:
|
||||||
|
|
||||||
|
```text
|
||||||
|
python/iar_parser/
|
||||||
|
```
|
||||||
|
|
||||||
|
To install the required dependencies, run the following command:
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
#### GoogleTest
|
||||||
|
|
||||||
|
The project uses GoogleTest sources directly:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GOOGLETEST_DIR=C:\path\to\googletest
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build and Deployment
|
||||||
|
|
||||||
|
The project is built using **qmake**.
|
||||||
|
|
||||||
|
Generate Makefiles:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
qmake embedded_test_stand.pro
|
||||||
|
```
|
||||||
|
|
||||||
|
Build:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mingw32-make -j8
|
||||||
|
```
|
||||||
|
|
||||||
|
Clean:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mingw32-make clean -j8
|
||||||
|
```
|
||||||
|
|
||||||
|
The build directory used during development:
|
||||||
|
|
||||||
|
```text
|
||||||
|
./compiler
|
||||||
|
```
|
||||||
|
|
||||||
|
After successful compilation, Qt runtime libraries should be deployed using:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
windeployqt embedded_test_stand.exe
|
||||||
|
```
|
||||||
|
|
||||||
|
In Qt Creator the following deployment step is used:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Command:
|
||||||
|
windeployqt
|
||||||
|
|
||||||
|
Arguments:
|
||||||
|
%{buildDir}/release/embedded_test_stand.exe
|
||||||
|
|
||||||
|
Working directory:
|
||||||
|
%{buildDir}/compiler
|
||||||
|
```
|
||||||
|
|
||||||
|
This command automatically copies:
|
||||||
|
|
||||||
|
* QtCore.dll
|
||||||
|
* Qt5SerialPort.dll
|
||||||
|
* platform plugins
|
||||||
|
* required MinGW runtime libraries
|
||||||
|
* other Qt dependencies
|
||||||
|
|
||||||
|
to the executable directory.
|
||||||
|
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Test Stand Settings
|
||||||
|
See `./config.json` file:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"UART": {
|
||||||
|
"RS485_address": 1,
|
||||||
|
"COM_port": "COM14",
|
||||||
|
"COM_baudRate": 921600,
|
||||||
|
"COM_bits": 8,
|
||||||
|
"COM_parity": 3,
|
||||||
|
"COM_stopBits": 1,
|
||||||
|
"COM_flowControl": false
|
||||||
|
},
|
||||||
|
"Path": {
|
||||||
|
"MK_elf_file": "C:/Danila/work/sputnik_test/src/sputnik/Debug/c.out",
|
||||||
|
"MK_map_file": "C:/Danila/work/sputnik_test/src/sputnik/Debug/sputnik.map",
|
||||||
|
"test_plan_path": "C:/Danila/work/embedded_test_stand/configs/test_plan.json",
|
||||||
|
"test_cases_path": "C:/Danila/work/embedded_test_stand/configs/test_cases/",
|
||||||
|
"test_log_path": "C:/Danila/work/embedded_test_stand/logs/",
|
||||||
|
"stand_report_html_path": "C:/Danila/work/embedded_test_stand/logs/report.html",
|
||||||
|
"map_parser": "C:/Danila/work/embedded_test_stand/python/iar_parser/map_parser.py",
|
||||||
|
"elf_parser": "C:/Danila/work/embedded_test_stand/python/iar_parser/elf_parser.py",
|
||||||
|
"binary_cmd_path": "C:/Danila/work/embedded_test_stand/cmd/"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
- **`UART`** - Define UART hardware parameters
|
||||||
|
- **`Path`** - Define path settings
|
||||||
|
- **`MK_elf_file`** - the result of compiling the microcontroller software in the form of en elf-file (needed if using `"protocol_ID": 2` in `test case json file`)
|
||||||
|
- **`MK_map_file`** - the result of compiling the microcontroller software in the form of map-file
|
||||||
|
- **`test_plan_path`** - the path with test plan json file
|
||||||
|
- **`test_cases_path`** - the path with test cases json files
|
||||||
|
- **`test_log_path`** - the path to which log files will be written
|
||||||
|
- **`stand_report_html_path`** - the path and filename to which html-report will be written
|
||||||
|
- **`map_parser`** - the path and filename where python-file is located. This python-script parse map-file (from compiler for microcontroller software)
|
||||||
|
- **`elf_parser`** - the path and filename where python-file is located. This python-script parse elf-file (from compiler for microcontroller software)
|
||||||
|
- **`binary_cmd_path`** - the path with binary data will be transmitted to microcontroller through UART
|
||||||
|
|
||||||
|
|
||||||
|
### Test Plan
|
||||||
|
See `./configs/test_plan.json` file. This file define test groups and their associated test cases, for example:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"groups": {
|
||||||
|
"smoke": [
|
||||||
|
"test_UART_cmdSetDataInterface.json",
|
||||||
|
"test_readSnapshot_HK.json",
|
||||||
|
"#test_erase_all.json"
|
||||||
|
],
|
||||||
|
|
||||||
|
"full": [
|
||||||
|
"### set NAND settings ###",
|
||||||
|
"test_UART_cmdSetDataInterface.json",
|
||||||
|
"test_start_March_FTE_by_blocks.json"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
- **`"smoke", "full"`** - user defined groups containing sets of test cases
|
||||||
|
- **`"#test_abort.json"`** - ignored test
|
||||||
|
- **`"test_readSnapshot_HK.json"`** - example of user test case
|
||||||
|
- **`"### set NAND settings ###"`** - comments
|
||||||
|
|
||||||
|
## Running the Test Stand
|
||||||
|
Before starting the application make sure that:
|
||||||
|
|
||||||
|
* the target embedded device is powered on;
|
||||||
|
* the UART interface is connected correctly;
|
||||||
|
* the COM port specified in `config.json` exists and is not used by another application;
|
||||||
|
* all configuration files and binary command files are present;
|
||||||
|
* ELF and MAP files of the embedded firmware are available if binary structure parsing is used (`protocol_ID = 2`).
|
||||||
|
|
||||||
|
|
||||||
|
Execute the built binary with command-line arguments:
|
||||||
|
```bash
|
||||||
|
./embedded_test_stand.exe --delay 10 --group full
|
||||||
|
```
|
||||||
|
or
|
||||||
|
```bash
|
||||||
|
./embedded_test_stand.exe --delay 10 --group smoke --case test_UART_cmdSetDataInterface
|
||||||
|
```
|
||||||
|
**Available Flags**:
|
||||||
|
- `--group <GROUP_NAME>`: Specify a test group (e.g., `smoke`, `full`).
|
||||||
|
- `--case <TEST_NAME>`: Run a specific test case (overrides group).
|
||||||
|
- `--repeat <N>`: Repeat the test case `N` times.
|
||||||
|
- `--delay`: <N> delay `N` msec between each test cases
|
||||||
|
|
||||||
|
expects the following files to be present:
|
||||||
|
|
||||||
|
```text
|
||||||
|
config.json
|
||||||
|
configs/
|
||||||
|
├── test_plan.json
|
||||||
|
└── test_cases/
|
||||||
|
├── test_readSnapshot_HK.json
|
||||||
|
└── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
The application loads:
|
||||||
|
|
||||||
|
* global settings from `config.json`;
|
||||||
|
* a list of test cases from `test_plan.json`;
|
||||||
|
* individual test descriptions from `configs/test_cases/*.json`;
|
||||||
|
* UART command binaries specified in the `"binary"` field of each test case.
|
||||||
|
|
||||||
|
After startup, the application:
|
||||||
|
|
||||||
|
1. Opens the UART port.
|
||||||
|
2. Sends binary commands to the embedded target.
|
||||||
|
3. Receives UART responses.
|
||||||
|
4. Parses received binary structures or text.
|
||||||
|
5. Validates structure fields according to the `struct_check` rules.
|
||||||
|
6. Generates console and HTML test reports.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test Case Syntax
|
||||||
|
The general view of the test case depends on the type of processed data, received via UART:
|
||||||
|
- binary data (see below `"protocol_ID": 2`)
|
||||||
|
- text data (see below `"protocol_ID": 1`)
|
||||||
|
|
||||||
|
### Processing text data, received via UART
|
||||||
|
Each test case file (for example, `test_start_March_FTE_by_blocks.json`) defines the expected UART sequence:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"meta": {
|
||||||
|
"name": "start_March-FTE_by_blocks",
|
||||||
|
"description": "start March-FTE by specified blocks",
|
||||||
|
"parameters": {
|
||||||
|
"_comment": "protocol_ID = 1 for text format from UART, see protocol_ID_en",
|
||||||
|
"protocol_ID": 1,
|
||||||
|
"version": 1,
|
||||||
|
"cmd_from_binary_file": true,
|
||||||
|
"cmd_code": 21,
|
||||||
|
"mode_March_FTE": 2,
|
||||||
|
"begin_block": 20,
|
||||||
|
"end_block": 25
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"binary": "start_March-FTE_by_blocks_20_25.bin",
|
||||||
|
|
||||||
|
"expectations": {
|
||||||
|
"start": {
|
||||||
|
"type": "json",
|
||||||
|
"fields": {
|
||||||
|
"msgType": 2,
|
||||||
|
"cmd": 21
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"sequence": [
|
||||||
|
{
|
||||||
|
"type": "regex",
|
||||||
|
"value": "start march-FTE in block \\d+(, LUN 0, Target 0)?"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
|
||||||
|
"end": {
|
||||||
|
"type": "contains",
|
||||||
|
"value": "NAND Flash test stand ready to receive new commands"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"timeout_msec": 35000
|
||||||
|
}
|
||||||
|
```
|
||||||
|
- **`"timeout_msec"`** - UART response timeout. Messages may arrive in parts over a specified time interval. After the timeout expires, data reception stops, and the message parts are combined into a final string as they arrive, which is then analyzed in its entirely
|
||||||
|
- **`"meta"`**: `"parameters"` - user defined test case parameters
|
||||||
|
- `"protocol_ID"` - defines the type of data received from the UART that needs to be processed
|
||||||
|
- `"version"` - reserved for future realization
|
||||||
|
- `"cmd_from_binary_file"` - always `true`, because command data file for transmit via UART is taken form the binary file now, see `"binary"` field below
|
||||||
|
- `"cmd_code"` - command code (user information) \
|
||||||
|
**Additionally, the user can add other fields here in order to distinguish test cases**
|
||||||
|
- **`"binary"`** - The binary data file name, whose contents will be transmitted via UART. Corresponding file path is taken from `"binary_cmd_path"` field, see `./config.json`
|
||||||
|
- **`"expectations"`** - Validates incoming messages by matching specific patterns \
|
||||||
|
- **`"start"`** Find the initial specified substring at the beginning of the message from UART.
|
||||||
|
- **`"sequence"`** Find ordered specified sequence of expected intermediate messages from UART.
|
||||||
|
- **`"end"`** Find specified terminate substring of massage from UART.
|
||||||
|
- *`"type"`** - specifies the type of message to compare. Possible meanings: `"contains"`, `"regex"`, `"json"` \
|
||||||
|
`"contains"` - find the substring, specified in the "value" field in the entire message received via UART \
|
||||||
|
`"json"` - find the substring in json-format, specified in the "value" field in the entire message received via UART \
|
||||||
|
`"regex"` - find the substring by using regular expression, specified in "value" field in the entire message received via UART
|
||||||
|
|
||||||
|
|
||||||
|
An example message from UART for passing this test case:
|
||||||
|
```
|
||||||
|
{"msgType": 2,"cmd": 21}
|
||||||
|
|
||||||
|
NAND Flash manufacture: MICRON
|
||||||
|
block Endurance (when using internal ECC): 100000 cycles
|
||||||
|
**************************************************************
|
||||||
|
|
||||||
|
start test March-FTE in NAND Flash
|
||||||
|
algorithm test:
|
||||||
|
1. ERASE BLOCK - erasing one block
|
||||||
|
2. READ(~D)n - read all pages of one block 3 times to check for data equality to the 0xFF pattern, incrementing address
|
||||||
|
3. PROGRAM(D) - programming pages of one block with pattern data, incrementing address
|
||||||
|
4. READ(D)n - read all pages of one block 3 times to check for data equality to written pattern data, incrementing address
|
||||||
|
5. READ(D) - repeat read all pages of one block, incrementing address
|
||||||
|
6. ERASE BLOCK - erasing one block
|
||||||
|
7. READ(~D)n - read all pages of one block 3 times to check for data equality to the 0xFF pattern, incrementing address
|
||||||
|
8. PROGRAM(D) - programming pages of one block with pattern data, DECREMENTING address
|
||||||
|
9. READ(D)n - read all pages of one block 3 times to check for data equality to written pattern data, DECREMENTING address
|
||||||
|
10. READ(D) - repeat read all pages of one block, DECREMENTING address
|
||||||
|
|
||||||
|
March-FTE test will check the following blocks: from: 20 to: 25, LUN: 0, target: 0
|
||||||
|
**************************************************************
|
||||||
|
start march-FTE in block 20, LUN 0, Target 0
|
||||||
|
start march-FTE in block 24, LUN 0, Target 0
|
||||||
|
TEST FINISH!
|
||||||
|
NAND Flash test stand ready to receive new commands
|
||||||
|
```
|
||||||
|
|
||||||
|
### Processing binary data, received via UART
|
||||||
|
Each test case file (for example, `test_readSnapshot_HK.json`) defines the expected structure, received via UART:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"meta": {
|
||||||
|
"name": "read_Snapshot_HK",
|
||||||
|
"description": "read Snapshot_HK structure",
|
||||||
|
"parameters": {
|
||||||
|
"_comment": "protocol_ID = 2 for binary format data from UART, reading C structure from memory, see protocol_ID_en",
|
||||||
|
"protocol_ID": 2,
|
||||||
|
"version": 1,
|
||||||
|
"cmd_code": 27,
|
||||||
|
"struct_name": "Snapshot_HK",
|
||||||
|
"struct_type": "Snapshot_HK_t",
|
||||||
|
"cmd_from_binary_file": true,
|
||||||
|
"addr_user_set": true,
|
||||||
|
"address": "0x08125600",
|
||||||
|
"size_user_set": true,
|
||||||
|
"size_bytes": 222
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"binary": "data.bin",
|
||||||
|
|
||||||
|
"struct_check" : [ {
|
||||||
|
"field": "size",
|
||||||
|
"type": "uint16",
|
||||||
|
"op": "==",
|
||||||
|
"expect": 222
|
||||||
|
}, {
|
||||||
|
"field": "counter",
|
||||||
|
"type": "uint32",
|
||||||
|
"op": ">",
|
||||||
|
"expect": 0
|
||||||
|
}, {
|
||||||
|
"field": "uart.errorLastOperation",
|
||||||
|
"type": "uint8",
|
||||||
|
"op": "==",
|
||||||
|
"expect": 0
|
||||||
|
}, {
|
||||||
|
"field": "rmap.error",
|
||||||
|
"array": true,
|
||||||
|
"type": "uint16",
|
||||||
|
"op": "==",
|
||||||
|
"exclude": [0, 22],
|
||||||
|
"expect": 0
|
||||||
|
} ],
|
||||||
|
|
||||||
|
"timeout_msec": 100
|
||||||
|
}
|
||||||
|
```
|
||||||
|
- **`"timeout_msec"`** - UART response timeout. Messages may arrive in parts over a specified time interval. After the timeout expires, data reception stops, and the message parts are combined into a final string as they arrive, which is then analyzed in its entirely
|
||||||
|
- **`"meta"`**: `"parameters"` - user defined test case parameters
|
||||||
|
- `"protocol_ID"` - defines the type of data received from the UART that needs to be processed
|
||||||
|
- `"version"` - reserved for future realization
|
||||||
|
- `"cmd_code"` - command code (user information) \
|
||||||
|
- `"struct_name"` - a variable having the type of the structure being checked as in the source embedded code (for example: "Snapshot_HK")
|
||||||
|
- `"struct_type"`: - the name of the type of the structure being checked as in the source embedded code (for example: "Snapshot_HK_t")
|
||||||
|
- `"cmd_from_binary_file"` - always `true`, because command data file for transmit via UART is taken form the binary file now, see `"binary"` field below
|
||||||
|
- `"addr_user_set"` - if `true`, means checked variable address is taken from "address" field (see below). If `false` means checked variable address is taken from parsed compiler map-file (by using python-script) from `./iar_files/ENTRY_LIST_map.json`
|
||||||
|
- `"address"` - checked variable address in hex format, for example: "0x08125600"
|
||||||
|
- `"size_user_set"` - if `true`, means checked structure size is taken from "size_bytes" filed (see below). If `false` means checked structure size is taken from parsed compiler map-file (by using python-script) from `./iar_files/ENTRY_LIST_map.json`
|
||||||
|
- `"size_bytes"` - specify checked structure size in bytes
|
||||||
|
**Additionally, the user can add other fields here in order to distinguish test cases**
|
||||||
|
- **`"binary"`** - The binary data file name, whose contents will be transmitted via UART. Corresponding file path is taken from `"binary_cmd_path"` field, see `./config.json`
|
||||||
|
- **`"struct_check"`** - this section defines a set of validation rules applied to the binary structure received from the target device via UART.
|
||||||
|
Each entry describes:
|
||||||
|
|
||||||
|
* **`field`** – structure field name. Nested fields are supported using dot notation, for example:
|
||||||
|
|
||||||
|
* `"uart.errorLastOperation"`
|
||||||
|
* `"rmap.error"`
|
||||||
|
|
||||||
|
* **`type`** – expected C data type of the field:
|
||||||
|
|
||||||
|
* `uint8`
|
||||||
|
* `uint16`
|
||||||
|
* `uint32`
|
||||||
|
* `uint64`
|
||||||
|
* `int8`
|
||||||
|
* `int16`
|
||||||
|
* `int32`
|
||||||
|
* `int64`
|
||||||
|
* `float`
|
||||||
|
* `double`
|
||||||
|
|
||||||
|
* **`op`** – comparison operator used for validation:
|
||||||
|
|
||||||
|
* `"=="` equal to
|
||||||
|
* `"!="` not equal to
|
||||||
|
* `">"` greater than
|
||||||
|
* `">="` greater than or equal to
|
||||||
|
* `"<"` less than
|
||||||
|
* `"<="` less than or equal to
|
||||||
|
|
||||||
|
* **`expect`** – expected value used in comparison.
|
||||||
|
|
||||||
|
* **`array`** *(optional)* – if `true`, the specified field is treated as an array and every element is checked against the condition.
|
||||||
|
|
||||||
|
* **`exclude`** *(optional)* – list of array indices excluded from validation. This is useful when some array elements are reserved, contain implementation-specific values, or are expected to change dynamically.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"field": "rmap.error",
|
||||||
|
"array": true,
|
||||||
|
"type": "uint16",
|
||||||
|
"op": "==",
|
||||||
|
"exclude": [0, 22],
|
||||||
|
"expect": 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This rule means:
|
||||||
|
|
||||||
|
* the field `rmap.error` is an array of `uint16`;
|
||||||
|
* all array elements must be equal to `0`;
|
||||||
|
* elements with indices `0` and `22` are skipped during validation.
|
||||||
|
|
||||||
|
The validation result for each rule is reported independently. If at least one check fails, the test case is considered failed and a detailed diagnostic message is printed, including:
|
||||||
|
|
||||||
|
* field name;
|
||||||
|
* actual value;
|
||||||
|
* expected value;
|
||||||
|
* comparison operator;
|
||||||
|
* array index (for array checks).
|
||||||
|
|
||||||
|
The test-case checks the following structure received via UART:
|
||||||
|
```
|
||||||
|
typedef __packed struct
|
||||||
|
{
|
||||||
|
uint8_t errorLastOperation; //RMAP_errFlag_en
|
||||||
|
uint16_t error[COUNT_ERR_MAX_ELEMENTS_RMAP];
|
||||||
|
} RMAP_HK_t;
|
||||||
|
|
||||||
|
typedef __packed struct
|
||||||
|
{
|
||||||
|
volatile uint8_t errorLastOperation; // UART_operErrCode_enum
|
||||||
|
volatile uint16_t error[COUNT_ERR_MAX_ELEMENTS];
|
||||||
|
} UART_HK_t;
|
||||||
|
|
||||||
|
extern UART_HK_t UART_HK;
|
||||||
|
|
||||||
|
typedef __packed struct
|
||||||
|
{
|
||||||
|
uint16_t size;
|
||||||
|
uint32_t counter;
|
||||||
|
UART_HK_t uart;
|
||||||
|
RMAP_HK_t rmap;
|
||||||
|
} Snapshot_HK_t;
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Reporting
|
||||||
|
|
||||||
|
- **`./logs/test_stand.log`**: contains a report of errors from the Embedded System Test Automation Framework
|
||||||
|
- **`./logs/test_cases.log`**: contains a report with detailed execution steps.
|
||||||
|
- **`./logs/report.html`**: contains a summary report. It is created after execution and includes:
|
||||||
|
- Test case pass/fail status.
|
||||||
|
- Timestamps and error messages.
|
||||||
|
- UART communication statistics.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes (summary)
|
||||||
|
- The application uses asynchronous UART communication for real-time testing.
|
||||||
|
|
||||||
|
- Incoming data from target may arrive in multiple fragments. The framework accumulates all received packets until the timeout expires, specified in the test case by `"timeout_msec"`
|
||||||
|
After that, the complete UART message is assembled and processed.
|
||||||
|
|
||||||
|
- Test cases are fully described in JSON files and can be extended without modifying the C++ source code.
|
||||||
|
|
||||||
|
- Binary UART commands are stored as external `.bin` files. The file name is specified in the `"binary"` field of the test case, while the search directory is configured in `config.json` using `"binary_cmd_path"`.
|
||||||
|
|
||||||
|
- When `"protocol_ID": 2` is used, the framework can automatically decode C structures received from the target. Structure descriptions are extracted from:
|
||||||
|
- IAR map file (`*.map`);
|
||||||
|
- ELF file (`*.out`, `*.elf`);
|
||||||
|
- JSON files generated by the Python parsers located in `./python/iar_parser`.
|
||||||
|
|
||||||
|
- The files inside `./iar_files/` are generated automatically by:
|
||||||
|
- `map_parser.py` – parses IAR map files and extracts symbols, addresses, and sizes;
|
||||||
|
- `elf_parser.py` – parses ELF files and exports C structure layouts.
|
||||||
|
|
||||||
|
- The framework supports:
|
||||||
|
- nested C structures;
|
||||||
|
- packed structures;
|
||||||
|
- arrays with element exclusion;
|
||||||
|
- validation using comparison operators (`==`, `!=`, `>`, `<`, `>=`, `<=`);
|
||||||
|
- structure field access using dot notation (for example: `"uart.errorLastOperation"`).
|
||||||
|
|
||||||
|
- The application generates:
|
||||||
|
- console output;
|
||||||
|
- detailed log files in `./logs`;
|
||||||
|
- an HTML report summarizing the entire test session.
|
||||||
|
|
||||||
|
- The framework is a **console application** (`QT += serialport`, `CONFIG += console`) and does not require GUI components at runtime.
|
||||||
|
|
||||||
|
- The project was developed and tested on **Windows 10/11** using:
|
||||||
|
- Qt 5.15.2;
|
||||||
|
- MinGW 8.1.0 (64-bit);
|
||||||
|
- GoogleTest;
|
||||||
|
- qmake build system.
|
||||||
|
|
||||||
|
- Before running the application, make sure:
|
||||||
|
- the UART device is connected;
|
||||||
|
- the selected COM port exists and is not occupied by another application;
|
||||||
|
- paths in `config.json` point to valid files;
|
||||||
|
- the required binary command files exist in the configured `binary_cmd_path`.
|
||||||
|
|
||||||
|
- After compilation on Windows, it is recommended to run `windeployqt` to copy all required Qt runtime libraries and plugins to the executable directory.
|
||||||
|
|
||||||
|
- The framework can be easily extended with additional UART protocols, new validation rules, custom report generators, and support for other embedded compilers besides IAR.
|
||||||
|
|
||||||
|
---
|
||||||
BIN
exe_windows_10_11/Qt5Core.dll
Normal file
BIN
exe_windows_10_11/Qt5Core.dll
Normal file
Binary file not shown.
BIN
exe_windows_10_11/Qt5SerialPort.dll
Normal file
BIN
exe_windows_10_11/Qt5SerialPort.dll
Normal file
Binary file not shown.
BIN
exe_windows_10_11/libgcc_s_seh-1.dll
Normal file
BIN
exe_windows_10_11/libgcc_s_seh-1.dll
Normal file
Binary file not shown.
BIN
exe_windows_10_11/libstdc++-6.dll
Normal file
BIN
exe_windows_10_11/libstdc++-6.dll
Normal file
Binary file not shown.
BIN
exe_windows_10_11/libwinpthread-1.dll
Normal file
BIN
exe_windows_10_11/libwinpthread-1.dll
Normal file
Binary file not shown.
Reference in New Issue
Block a user