Mavlink-inspector: MATLAB DataFlash and MAVLink log diagnostics for fixed-wing airplanes

@Georacer wrote:

Image may be NSFW.
Clik here to view.

Hello everyone,

I have begun working on a DataFlash diagnostic toolbox for fixed-wing aircraft, the mavlink-inspector.

---

Background

Log analysis has always been interesting to me. Being able to watch someone's flight at the ease of your desk and be able to resolve the causes of a crash or point out tuning deficiencies sounds exciting and borderline magical. Leveraging the value of stored information to identify the weak points of a real-world system is invaluable.

Log visualization tools such as MAVExplorer provides handy plotting of log message data series, facilitating log analysis.

Tools such as ardupilot's LogAnalyzer and Dronekit's Log Analyzer are also a step to the direction of diagnosis and have truly helped many users pick out sensor gone bad and power failures.

However, these tools are mostly targeted for multirotors and provide static tests on a couple of variables at a time. On the contrary, I am mostly interested on fixed-wing airframes and want to delve a little deeper on diagnostic tests which span as much message information as possible.
As a result, I have started the mavlink-inspector software project, to purse that goal.

mavlink-inspector is a collection of MATLAB scripts and functions which allow for DataFlash (currently) and MAVLink (in the future) log parsing and diagnosing. MAVLink is a popular telemetry protocol but the information is passes along can be used for much more than simple visualization. Still, I wanted to give myself an easier time by starting with ArduPlane DataFlash logs, which carry more diagnostic information.

Some people might have negative feelings against the MATLAB software suite, and from a programming perspective I agree with them. But as a controls engineer and am very familiar with it and, by all accounts, is the fastest algorithm prototyping tool I know.

---

Overview

The mavlink-inspector code can be cloned into any directory and used with a MATLAB Linux installation. Git is also required, to ensure that test results are up-to-date with the current checking code.

Currently, Mission Planner is used to extract the transcript of a Dataflash .BIN log into a cleartext .LOG file with the same structure. From there on, a log parser script converts it into separate arrays for each message type which appears in the log file.
With the log contents in MATLAB format, it is now possible to utilize the full spectrum of MATLAB's functionality.

The toolbox is based off of a few basic class definitions:

• The Evidence class, which is used to store the deciding information of each check, for later reference (inspired by Dronekit-LA)
• The Result class, which carries the result of each check and has one or more Evidence objects as its members
• The Checker class, which is inherited by individual diagnostic checks which provide the diagnostic functionality themselves and has a Result member as its member.

A test object can be given the log data and parse them, to perform its dedicated test. It can then be queried for the result printout or even stored.

---

So far, overview checks have been created for an ArduPlane DataFlash log, checking, among others,

• the firmware properties of the related log
• the date when the log was recorded
• the duration of the log
• the log size
• the number of different log messages received

A typical output of the overview checks can be seen below:

   gitBuild: Short git hash of the current branch
   The git hash of the mavlink-inspector test suite is ecc3ce0bb1572ee10ec1ec831372ac4e43d69f93

   isUpdatedMat: Check if the .mat file is up to date
   The stored .mat file is up-to-date with the current log2mat function.

   logName: Name of the current log
   Examining log 68.BIN

   parseDate: Date when this result was extracted
   Current date and time is 07-Nov-2016 22:32:47 UTC

   logDate: Date when the log was recorded
   The log was recorded on 05-Oct-2016 08:33:08 UTC

   logSize: Name of the current log
   Log file size is 60304 kilobytes

   logDuration: Duration between first and last timestamp
   The duration of the log file, based on CPU timestamps is 1472.45 seconds

   fwStats: Firmware related statistics
   Platform: ArduPlane
   Version: V3.5.2
   Git hash: dc9d87fd

   msgStats: Statistics on parsed messages
   Log contains 47 different message types

---

ardupilot Log Analyzer port

The checking functions of ardupilot's Log Analyzer, which are applicable to fixed-wing aircraft have also been ported. These functions include:

• TestBrownout
• TestCompass
• TestDupeLogData
• TestEmpty
• TestGPSGlitch
• TestIMUMatch
• TestVCC

An example output of this family of checks can be seen below:

   TestBrownout: Test for a log that has been truncated in flight - Ported from ArduPilot LogAnalyzer
   No brownout detected | Warning: This check is discouraged - barometer drift may affect results

   TestEmpty: Test for empty or near-empty logs - Ported from ArduPilot LogAnalyzer
   PASSED: Throttle maximum value is 68

   TestGPSGlitch: Test for GPS glitch reporting or bad GPS data (satellite count, hdop) - Ported from ArduPilot LogAnalyzer
   PASSED: Min Satellites: 8 | Max HDop: 1.21

   TestVCC: Test for VCC within recommendations, or abrupt end to log in flight - Ported from ArduPilot LogAnalyzer
   WARNING: Vcc min/max difference is 0.43V with threshold 0.3V

   TestCompass: Test for compass offsets and throttle interference - Ported from ArduPilot LogAnalyzer
   PASSED: Compass checks successful

   TestDupeLogData: Test for duplicated data in log, which has been happening on PX4/Pixhawk - Ported from ArduPilot LogAnalyzer
   PASSED: No duplicate data found

   TestIMUMatch: Test compatibility between IMU1 and IMU2 - Ported from ArduPilot LogAnalyzer
   PASSED: Mismatch: 0.605451, WARN: 0.75, FAIL: 1.5

---

Documentation

A few words, documenting the current release can be found at the related Github Page.

---

What is to come

Dronekit-LA functionality is next to be ported onto mavlink-inspector.

After the baseline tests from Log Analyzer and Dronekit-LA have been ported, new functionality will be added. The main focus will be on:

1. Writing a pure .BIN to MATLAB log converter and leave Mission Planner out of the loop. This targets in isolating the workflow into Linux, whilst right now it spans Windows as well.
2. Create dedicated plotting functions for all available checks and their results/evidence
3. Go on with more involved checks, such as thruster diagnostics

---

In all, I hope this toolbox will prove useful to anyone interested in taking the fixed-wing UAV log analysis into a MATLB context.

Thanks for reading!

Posts: 1

Participants: 1

Read full topic