add readme and exe files for windows

2026-06-17 15:04:11 +03:00
parent d5a6fe4d7d
commit 1534d825a4
6 changed files with 665 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,665 @@
+
+# Embedded System Test Automation Framework
+
+This is a C++ application designed for testing embedded systems using UART communication. The system allows you to load and execute test cases that interact with the UART interface according to predefined protocols.
+
+## Table of Contents
+
+- [Overview](#overview)
+- [Key Features](#key-features)
+- [Supported Embedded Toolchains](#supported-embedded-toolchains)
+- [Project Structure](#project-structure)
+- [Requirements](#requirements)
+  - [Development Environment](#development-environment)
+  - [Build and Deployment](#build-and-deployment)
+- [Configuration](#configuration)
+  - [Test Stand Settings](#test-stand-settings)
+  - [Test Plan](#test-plan)
+- [Running the Test Stand](#running-the-test-stand)
+- [Test Case Syntax](#test-case-syntax-json)
+  - [Processing Text Data Received via UART](#processing-text-data-received-via-uart)
+  - [Processing Binary Data Received via UART](#processing-binary-data-received-via-uart)
+- [Reporting](#reporting)
+- [Notes (summary)](#notes)
+
+## Overview
+
+Embedded Systems Test Automation Framework is a console-based C++ application for automated testing of embedded systems through UART communication.
+
+The framework supports:
+
+* execution of predefined test plans;
+* transmission of binary commands to the target device;
+* reception and validation of UART responses (if text format) or parsing of binary C structures directly from the target memory;
+* generation of text logs and HTML reports;
+
+The framework is configured entirely through JSON files and can be easily integrated into automated verification environments.
+
+## Key Features
+
+- **Binary Command Support**: Send raw `.bin` payloads directly to the microcontroller.
+- **UART I/O Communication**: UART I/O for interacting with microcontrollers.
+- **Sequence Validation in text format**: Validates incoming messages by matching specific patterns:
+  - **Start Message**: Initial handshake/trigger.
+  - **Sequence**: Ordered sequence of expected intermediate messages.
+  - **End Message**: Completion criteria for the test case.
+- **JSON-Driven Configuration**: Easily define test cases, UART parameters, and test plans in JSON format.
+- **Flexible Execution**:
+  - **Grouping**: Organize tests into suites (e.g., --group=smoke, --group=full).
+  - **Repeats**: Built-in support for stress testing by repeating cases (using --repeat=N).
+- **Advanced Logging**: Generates detailed text logs and visual HTML reports for every run.
+
+## Supported Embedded Toolchains
+
+The framework itself is compiler-independent.
+
+However, automatic binary structure parsing is currently implemented for firmware generated by:
+
+* IAR Embedded Workbench
+* ELF/DWARF compatible toolchains
+
+The parser extracts:
+
+* variable addresses;
+* structure sizes;
+* nested structures;
+* array dimensions;
+* primitive data types.
+
+The generated metadata are stored as JSON files inside:
+
+```text
+./iar_files/
+```
+
+
+## Project Structure
+
+```text
+├── cmd
+│   ├── cmd_abort.bin
+│   ├── set_interface_params.bin
+│   ├── set_settings_NAND_micron_MT29F16G08AJADAWP.bin
+├── compiler
+│   ├── Makefile
+│   ├── Makefile.Debug
+│   ├── Makefile.Release
+│   ├── data.bin
+├── config.json
+├── configs
+│   ├── cmd
+│   ├── test_cases
+│   │   ├── test_UART_cmdSetDataInterface.json
+│   │   ├── test_abort.json
+│   │   ├── test_readSnapshot_HK.json
+│   │   ├── test_read_NANDctrlOper.json
+│   │   ├── test_set_micron_MT29F16G08AJADAWP.json
+│   └── test_plan.json
+├── iar_files
+│   ├── ENTRY_LIST_map.json
+│   ├── NANDctrlOper_t.json
+│   └── Snapshot_HK_t.json
+├── logs
+│   ├── 12_05_2026_15_48_16.log
+│   ├── report.html
+│   ├── test_cases.log
+│   └── test_stand.log
+├── python
+│   └── iar_parser
+│       ├── elf_parser.py
+│       ├── map_parser.py
+│       ├── requirements.txt
+├── reports
+├── src
+│   ├── core
+│   │   ├── cli_parser.cpp
+│   │   ├── cli_parser.h
+...
+│   │   ├── uart.cpp
+│   │   └── uart.h
+│   ├── embedded_test_stand.pro
+│   ├── embedded_test_stand.pro.user
+│   ├── gtest_dependency.pri
+│   ├── main.cpp
+│   └── tests
+│       ├── test_runner.cpp
+│       ├── uart_fixture.cpp
+│       └── uart_fixture.h
+└── src.rar
+```
+
+## Requirements
+
+### Development Environment
+The project was developed and tested on Windows 10/11 using the following software:
+
+| Component | Version |
+|----------|---------|
+| Qt | 5.15.2 |
+| Qt Creator | 9.x or newer |
+| Compiler | MinGW 8.1.0 (64-bit) |
+| Debugger     | `gdb.exe`            |
+| Build system | qmake                |
+| qmake | Qt 5.15.2 |
+| mingw32-make | MinGW 8.1.0 |
+| Python | 3.9+ |
+| GoogleTest | Latest stable version |
+
+#### Qt Modules
+
+The following Qt modules are required:
+
+* QtCore
+* QtSerialPort
+
+#### Python Dependencies
+
+Python scripts located in:
+
+```text
+python/iar_parser/
+```
+
+To install the required dependencies, run the following command:
+```bash
+pip install -r requirements.txt
+```
+
+#### GoogleTest
+
+The project uses GoogleTest sources directly:
+
+```text
+GOOGLETEST_DIR=C:\path\to\googletest
+```
+
+### Build and Deployment
+
+The project is built using **qmake**.
+
+Generate Makefiles:
+
+```bash
+qmake embedded_test_stand.pro
+```
+
+Build:
+
+```bash
+mingw32-make -j8
+```
+
+Clean:
+
+```bash
+mingw32-make clean -j8
+```
+
+The build directory used during development:
+
+```text
+./compiler
+```
+
+After successful compilation, Qt runtime libraries should be deployed using:
+
+```bash
+windeployqt embedded_test_stand.exe
+```
+
+In Qt Creator the following deployment step is used:
+
+```text
+Command:
+windeployqt
+
+Arguments:
+%{buildDir}/release/embedded_test_stand.exe
+
+Working directory:
+%{buildDir}/compiler
+```
+
+This command automatically copies:
+
+* QtCore.dll
+* Qt5SerialPort.dll
+* platform plugins
+* required MinGW runtime libraries
+* other Qt dependencies
+
+to the executable directory.
+
+
+## Configuration
+
+### Test Stand Settings  
+See `./config.json` file:
+```json
+{
+  "UART": {
+    "RS485_address": 1, 
+    "COM_port": "COM14",
+    "COM_baudRate": 921600,
+    "COM_bits": 8,
+    "COM_parity": 3,
+    "COM_stopBits": 1,
+    "COM_flowControl": false
+  },
+  "Path": {
+    "MK_elf_file": "C:/Danila/work/sputnik_test/src/sputnik/Debug/c.out",
+    "MK_map_file": "C:/Danila/work/sputnik_test/src/sputnik/Debug/sputnik.map",
+    "test_plan_path": "C:/Danila/work/embedded_test_stand/configs/test_plan.json",
+    "test_cases_path": "C:/Danila/work/embedded_test_stand/configs/test_cases/",
+    "test_log_path": "C:/Danila/work/embedded_test_stand/logs/",
+    "stand_report_html_path": "C:/Danila/work/embedded_test_stand/logs/report.html",
+    "map_parser": "C:/Danila/work/embedded_test_stand/python/iar_parser/map_parser.py",
+    "elf_parser": "C:/Danila/work/embedded_test_stand/python/iar_parser/elf_parser.py",
+    "binary_cmd_path": "C:/Danila/work/embedded_test_stand/cmd/"
+  }
+}
+```
+- **`UART`** - Define UART hardware parameters
+- **`Path`** - Define path settings
+  - **`MK_elf_file`** - the result of compiling the microcontroller software in the form of en elf-file (needed if using `"protocol_ID": 2` in `test case json file`)
+  - **`MK_map_file`** - the result of compiling the microcontroller software in the form of map-file
+  - **`test_plan_path`** - the path with test plan json file
+  - **`test_cases_path`** - the path with test cases json files
+  - **`test_log_path`** - the path to which log files will be written
+  - **`stand_report_html_path`** - the path and filename to which html-report will be written
+  - **`map_parser`** - the path and filename where python-file is located. This python-script parse map-file (from compiler for microcontroller software)
+  - **`elf_parser`** - the path and filename where python-file is located. This python-script parse elf-file (from compiler for microcontroller software)
+  - **`binary_cmd_path`** - the path with binary data will be transmitted to microcontroller through UART
+
+
+### Test Plan 
+See `./configs/test_plan.json` file. This file define test groups and their associated test cases, for example:  
+```json
+{
+  "groups": {
+    "smoke": [
+      "test_UART_cmdSetDataInterface.json",
+      "test_readSnapshot_HK.json",
+      "#test_erase_all.json"
+    ],
+
+    "full": [
+      "### set NAND settings ###",
+      "test_UART_cmdSetDataInterface.json",
+      "test_start_March_FTE_by_blocks.json"
+    ]
+  }
+}
+``` 
+- **`"smoke", "full"`** - user defined groups containing sets of test cases
+- **`"#test_abort.json"`** - ignored test
+- **`"test_readSnapshot_HK.json"`** - example of user test case 
+- **`"### set NAND settings                       ###"`** - comments
+
+## Running the Test Stand
+Before starting the application make sure that:
+
+* the target embedded device is powered on;
+* the UART interface is connected correctly;
+* the COM port specified in `config.json` exists and is not used by another application;
+* all configuration files and binary command files are present;
+* ELF and MAP files of the embedded firmware are available if binary structure parsing is used (`protocol_ID = 2`).
+
+
+Execute the built binary with command-line arguments:  
+```bash
+./embedded_test_stand.exe --delay 10 --group full
+```
+or
+```bash
+./embedded_test_stand.exe --delay 10 --group smoke --case test_UART_cmdSetDataInterface
+```
+**Available Flags**:  
+- `--group <GROUP_NAME>`: Specify a test group (e.g., `smoke`, `full`).  
+- `--case <TEST_NAME>`: Run a specific test case (overrides group).  
+- `--repeat <N>`: Repeat the test case `N` times.   
+- `--delay`: <N> delay `N` msec between each test cases 
+
+expects the following files to be present:
+
+```text
+config.json
+configs/
+├── test_plan.json
+└── test_cases/
+    ├── test_readSnapshot_HK.json
+    └── ...
+```
+
+The application loads:
+
+* global settings from `config.json`;
+* a list of test cases from `test_plan.json`;
+* individual test descriptions from `configs/test_cases/*.json`;
+* UART command binaries specified in the `"binary"` field of each test case.
+
+After startup, the application:
+
+1. Opens the UART port.
+2. Sends binary commands to the embedded target.
+3. Receives UART responses.
+4. Parses received binary structures or text.
+5. Validates structure fields according to the `struct_check` rules.
+6. Generates console and HTML test reports.
+
+---
+
+## Test Case Syntax
+The general view of the test case depends on the type of processed data, received via UART:
+- binary data (see below `"protocol_ID": 2`)
+- text data (see below `"protocol_ID": 1`)
+
+### Processing text data, received via UART
+Each test case file (for example, `test_start_March_FTE_by_blocks.json`) defines the expected UART sequence:  
+```json
+{
+  "meta": {
+    "name": "start_March-FTE_by_blocks",
+    "description": "start March-FTE by specified blocks",
+    "parameters": {
+      "_comment": "protocol_ID = 1 for text format from UART, see protocol_ID_en",
+      "protocol_ID": 1,
+      "version": 1,
+      "cmd_from_binary_file": true,
+      "cmd_code": 21,
+      "mode_March_FTE": 2,
+      "begin_block": 20,
+      "end_block": 25
+    }
+  },
+
+  "binary": "start_March-FTE_by_blocks_20_25.bin",
+
+  "expectations": {
+    "start": {
+      "type": "json",
+      "fields": {
+        "msgType": 2,
+        "cmd": 21
+      }
+    },
+
+    "sequence": [
+      {
+        "type": "regex",
+        "value": "start march-FTE in block \\d+(, LUN 0, Target 0)?"
+      }
+    ],
+
+    "end": {
+      "type": "contains",
+      "value": "NAND Flash test stand ready to receive new commands"
+    }
+  },
+
+  "timeout_msec": 35000
+}
+```  
+- **`"timeout_msec"`** - UART response timeout. Messages may arrive in parts over a specified time interval. After the timeout expires, data reception stops, and the message parts are combined into a final string as they arrive, which is then analyzed in its entirely
+- **`"meta"`**: `"parameters"` - user defined test case parameters
+  - `"protocol_ID"` - defines the type of data received from the UART that needs to be processed
+  - `"version"` - reserved for future realization
+  - `"cmd_from_binary_file"` - always `true`, because command data file for transmit via UART is taken form the binary file now, see `"binary"` field below
+  - `"cmd_code"` - command code (user information) \
+  **Additionally, the user can add other fields here in order to distinguish test cases**
+- **`"binary"`** - The binary data file name, whose contents will be transmitted via UART. Corresponding file path is taken from `"binary_cmd_path"` field, see `./config.json`
+- **`"expectations"`** - Validates incoming messages by matching specific patterns \
+  - **`"start"`** Find the initial specified substring at the beginning of the message from UART.
+  - **`"sequence"`** Find ordered specified sequence of expected intermediate messages from UART.
+  - **`"end"`** Find specified terminate substring of massage from UART. 
+    - *`"type"`** - specifies the type of message to compare. Possible meanings: `"contains"`, `"regex"`, `"json"` \
+    `"contains"` - find the substring, specified in the "value" field in the entire message received via UART \
+    `"json"` - find the substring in json-format, specified in the "value" field in the entire message received via UART \
+    `"regex"` - find the substring by using regular expression, specified in "value" field in the entire message received via UART 
+
+
+An example message from UART for passing this test case:
+```
+{"msgType": 2,"cmd": 21}
+
+ NAND Flash manufacture: MICRON
+ block Endurance (when using internal ECC): 100000 cycles
+**************************************************************
+
+ start test March-FTE in NAND Flash 
+ algorithm test:
+ 1. ERASE BLOCK - erasing one block
+ 2. READ(~D)n - read all pages of one block 3 times to check for data equality to the 0xFF pattern, incrementing address
+ 3. PROGRAM(D) - programming pages of one block with pattern data, incrementing address
+ 4. READ(D)n - read all pages of one block 3 times to check for data equality to written pattern data, incrementing address
+ 5. READ(D) - repeat read all pages of one block, incrementing address
+ 6. ERASE BLOCK - erasing one block
+ 7. READ(~D)n - read all pages of one block 3 times to check for data equality to the 0xFF pattern, incrementing address
+ 8. PROGRAM(D) - programming pages of one block with pattern data, DECREMENTING address
+ 9. READ(D)n - read all pages of one block 3 times to check for data equality to written pattern data, DECREMENTING address
+ 10. READ(D) - repeat read all pages of one block, DECREMENTING address
+
+ March-FTE test will check the following blocks: from: 20 to: 25, LUN: 0, target: 0
+ **************************************************************
+start march-FTE in block 20, LUN 0, Target 0
+start march-FTE in block 24, LUN 0, Target 0
+TEST FINISH!
+NAND Flash test stand ready to receive new commands
+```
+
+### Processing binary data, received via UART
+Each test case file (for example, `test_readSnapshot_HK.json`) defines the expected structure, received via UART:  
+```json
+{
+  "meta": {
+    "name": "read_Snapshot_HK",
+    "description": "read Snapshot_HK structure",
+    "parameters": {
+      "_comment": "protocol_ID = 2 for binary format data from UART, reading C structure from memory, see protocol_ID_en",
+      "protocol_ID": 2,
+      "version": 1,
+      "cmd_code": 27,
+      "struct_name": "Snapshot_HK",
+      "struct_type": "Snapshot_HK_t",
+      "cmd_from_binary_file": true,
+      "addr_user_set": true,
+      "address": "0x08125600",
+      "size_user_set": true,
+      "size_bytes": 222
+    }
+  },
+
+  "binary": "data.bin",
+  
+  "struct_check" : [ {
+        "field": "size",
+		"type": "uint16",
+		"op": "==",
+		"expect": 222
+    }, {
+        "field": "counter",
+		"type": "uint32",
+		"op": ">",
+		"expect": 0
+    }, {
+        "field": "uart.errorLastOperation",
+		"type": "uint8",
+		"op": "==",
+		"expect": 0
+    }, {
+        "field": "rmap.error",
+		"array": true,
+		"type": "uint16",
+		"op": "==",
+		"exclude": [0, 22],
+		"expect": 0
+    } ],
+
+  "timeout_msec": 100
+}
+```  
+- **`"timeout_msec"`** - UART response timeout. Messages may arrive in parts over a specified time interval. After the timeout expires, data reception stops, and the message parts are combined into a final string as they arrive, which is then analyzed in its entirely
+- **`"meta"`**: `"parameters"` - user defined test case parameters
+  - `"protocol_ID"` - defines the type of data received from the UART that needs to be processed
+  - `"version"` - reserved for future realization
+  - `"cmd_code"` - command code (user information) \
+  - `"struct_name"` - a variable having the type of the structure being checked as in the source embedded code (for example: "Snapshot_HK")
+  - `"struct_type"`: - the name of the type of the structure being checked as in the source embedded code (for example: "Snapshot_HK_t")
+  - `"cmd_from_binary_file"` - always `true`, because command data file for transmit via UART is taken form the binary file now, see `"binary"` field below
+  - `"addr_user_set"` - if `true`, means checked variable address is taken from "address" field (see below). If `false` means checked variable address is taken from parsed compiler map-file (by using python-script) from `./iar_files/ENTRY_LIST_map.json`
+  - `"address"` -  checked variable address in hex format, for example: "0x08125600"
+  - `"size_user_set"` - if `true`, means checked structure size is taken from "size_bytes" filed (see below). If `false` means checked structure size is taken from parsed compiler map-file (by using python-script) from `./iar_files/ENTRY_LIST_map.json`
+  - `"size_bytes"` - specify checked structure size in bytes
+  **Additionally, the user can add other fields here in order to distinguish test cases**
+- **`"binary"`** - The binary data file name, whose contents will be transmitted via UART. Corresponding file path is taken from `"binary_cmd_path"` field, see `./config.json`
+- **`"struct_check"`** - this section defines a set of validation rules applied to the binary structure received from the target device via UART.
+Each entry describes:
+
+  * **`field`** – structure field name. Nested fields are supported using dot notation, for example:
+
+    * `"uart.errorLastOperation"`
+    * `"rmap.error"`
+
+  * **`type`** – expected C data type of the field:
+
+    * `uint8`
+    * `uint16`
+    * `uint32`
+    * `uint64`
+    * `int8`
+    * `int16`
+    * `int32`
+    * `int64`
+    * `float`
+    * `double`
+
+  * **`op`** – comparison operator used for validation:
+
+    * `"=="` equal to
+    * `"!="` not equal to
+    * `">"` greater than
+    * `">="` greater than or equal to
+    * `"<"` less than
+    * `"<="` less than or equal to
+
+  * **`expect`** – expected value used in comparison.
+
+  * **`array`** *(optional)* – if `true`, the specified field is treated as an array and every element is checked against the condition.
+
+  * **`exclude`** *(optional)* – list of array indices excluded from validation. This is useful when some array elements are reserved, contain implementation-specific values, or are expected to change dynamically.
+
+For example:
+
+```json
+{
+    "field": "rmap.error",
+    "array": true,
+    "type": "uint16",
+    "op": "==",
+    "exclude": [0, 22],
+    "expect": 0
+}
+```
+
+This rule means:
+
+* the field `rmap.error` is an array of `uint16`;
+* all array elements must be equal to `0`;
+* elements with indices `0` and `22` are skipped during validation.
+
+The validation result for each rule is reported independently. If at least one check fails, the test case is considered failed and a detailed diagnostic message is printed, including:
+
+* field name;
+* actual value;
+* expected value;
+* comparison operator;
+* array index (for array checks).
+
+The test-case checks the following structure received via UART:
+```
+typedef __packed struct
+{
+	uint8_t errorLastOperation; //RMAP_errFlag_en
+	uint16_t error[COUNT_ERR_MAX_ELEMENTS_RMAP];
+} RMAP_HK_t;
+
+typedef __packed struct
+{
+	volatile uint8_t errorLastOperation; // UART_operErrCode_enum
+	volatile uint16_t error[COUNT_ERR_MAX_ELEMENTS];
+} UART_HK_t;
+
+extern UART_HK_t UART_HK;
+
+typedef __packed struct
+{
+	uint16_t size;
+	uint32_t counter;
+	UART_HK_t uart;
+	RMAP_HK_t rmap;
+} Snapshot_HK_t;
+```
+
+---
+
+## Reporting  
+
+- **`./logs/test_stand.log`**: contains a report of errors from the Embedded System Test Automation Framework
+- **`./logs/test_cases.log`**: contains a report with detailed execution steps.  
+- **`./logs/report.html`**: contains a summary report. It is created after execution and includes:  
+  - Test case pass/fail status.  
+  - Timestamps and error messages.  
+  - UART communication statistics.  
+
+---
+
+## Notes (summary) 
+- The application uses asynchronous UART communication for real-time testing.
+
+- Incoming data from target may arrive in multiple fragments. The framework accumulates all received packets until the timeout expires, specified in the test case by `"timeout_msec"`
+After that, the complete UART message is assembled and processed.
+
+- Test cases are fully described in JSON files and can be extended without modifying the C++ source code.
+
+- Binary UART commands are stored as external `.bin` files. The file name is specified in the `"binary"` field of the test case, while the search directory is configured in `config.json` using `"binary_cmd_path"`.
+
+- When `"protocol_ID": 2` is used, the framework can automatically decode C structures received from the target. Structure descriptions are extracted from:
+  - IAR map file (`*.map`);
+  - ELF file (`*.out`, `*.elf`);
+  - JSON files generated by the Python parsers located in `./python/iar_parser`.
+
+- The files inside `./iar_files/` are generated automatically by:
+  - `map_parser.py` – parses IAR map files and extracts symbols, addresses, and sizes;
+  - `elf_parser.py` – parses ELF files and exports C structure layouts.
+
+- The framework supports:
+  - nested C structures;
+  - packed structures;
+  - arrays with element exclusion;
+  - validation using comparison operators (`==`, `!=`, `>`, `<`, `>=`, `<=`);
+  - structure field access using dot notation (for example: `"uart.errorLastOperation"`).
+
+- The application generates:
+  - console output;
+  - detailed log files in `./logs`;
+  - an HTML report summarizing the entire test session.
+
+- The framework is a **console application** (`QT += serialport`, `CONFIG += console`) and does not require GUI components at runtime.
+
+- The project was developed and tested on **Windows 10/11** using:
+  - Qt 5.15.2;
+  - MinGW 8.1.0 (64-bit);
+  - GoogleTest;
+  - qmake build system.
+
+- Before running the application, make sure:
+  - the UART device is connected;
+  - the selected COM port exists and is not occupied by another application;
+  - paths in `config.json` point to valid files;
+  - the required binary command files exist in the configured `binary_cmd_path`.
+
+- After compilation on Windows, it is recommended to run `windeployqt` to copy all required Qt runtime libraries and plugins to the executable directory.
+
+- The framework can be easily extended with additional UART protocols, new validation rules, custom report generators, and support for other embedded compilers besides IAR.
+
+---