How to use barreltest and test barrel code on the command-line?

I am looking for guidance on how to test barrel code from the command-line. From the documentation:

Barrels can be compiled and tested via the command line.

barrelbuild
Calls the Monkey C Barrel compiler. Much like monkeyc the barrelbuild call takes code from multiple files to be included in the library (.barrel file). The -f argument is required as the manifest file path is specified in one of the Barrel’s Jungle files. For more info, run barrelbuild without any arguments to see a full list of argument definitions.
> barrelbuild -o <output.barrel> -f <barrel.jungle>
barreltest
Executes Run No Evil tests on the .barrel file. Requires the -d argument with a valid device id.

> barreltest -o <output.barrel> -f <barrel.jungle> -d <device_id>

This suggests that barreltest will run the annotated unit tests within the barrel code. However, barreltest succeeds silently and does not output any test results to the console.

From playing around a bit, I've found that I am able to test the barrel code alongside my main app code by following these steps:

  1. Build the barrel with barrelbuild
  2. Link to the .barrel file within my main app's jungle and manifest files.
  3. Build a PRG file for unit testing.
  4. Run Run No Evil on my main app from the command-line.
    1. The unit test results from my main app and the barrel are output normally to the console.

The above steps fail if I instead link my main app to the output file of barreltest. For example:

$ barreltest -o /path/to/main/app/barrels/Barrel-test.barrel -f /path/to/Barrel/monkey.jungle -d <device> -y <key_path>

$ monkeyc -o tmp.prg -f monkey.jungle -d<device> -y <key_path> --unit-test
ERROR: The Monkey Barrel jungle file could not be extracted from Barrel-test.barrel.
com.garmin.monkeybrains.Packager$PackageException: [207] The Monkey Barrel jungle file could not be extracted from LogMonkey-test.barrel.
at com.garmin.monkeybrains.ProjectBuilder.getMonkeyBarrelJungles(ProjectBuilder.java:830)
at com.garmin.monkeybrains.ProjectBuilder.createBarrelContexts(ProjectBuilder.java:292)
at com.garmin.monkeybrains.Monkeybrains.runPRGCompiler(Monkeybrains.java:1234)
at com.garmin.monkeybrains.Monkeybrains.compileApplication(Monkeybrains.java:1024)
at com.garmin.monkeybrains.Monkeybrains.run(Monkeybrains.java:2317)
at com.garmin.monkeybrains.Monkeybrains.simpleMain(Monkeybrains.java:254)
at com.garmin.monkeybrains.Monkeybrains.simpleMain(Monkeybrains.java:236)
at com.garmin.monkeybrains.Monkeybrains.main(Monkeybrains.java:283)
usage: monkeyc [-a <arg>] [-d <arg>] [-e] [--Eno-invalid-symbol] [-f <arg>] [-g]
[-h] [-i <arg>] [-m <arg>] [-o <arg>] [-p <arg>] [-r] [-s <arg>] [-t] [-u
<arg>] [-v] [-w] [-x <arg>] [-y <arg>] [-z <arg>]
-a,--apidb <arg> API import file
-d,--device <arg> Target device
-e,--package-app Create an application package.
--Eno-invalid-symbolDo not error when a symbol is found to be invalid
-f,--jungles <arg> Jungle files
-g,--debug Print debug output
-h,--help Prints help information
-i,--import-dbg <arg> Import api.debug.xml
-m,--manifest <arg> Manifest file
-o,--output <arg> Output file to create
-p,--project-info <arg>projectInfo.xml file to use when compiling
-r,--release Strip debug information
-s,--sdk-version <arg> SDK version to target
-t,--unit-test Enables compilation of unit tests
-u,--devices <arg> devices.xml file to use when compiling
-v,--version Prints the compiler version
-w,--warn Show compiler warnings
-x,--excludes <arg> Add annotations to the exclude list
-y,--private-key <arg> Private key to sign builds with
-z,--rez <arg> Resource file

Other notes:

  • The output of barrelbuild appears to be a zip of the barrel code.
  • The output of barreltest is some binary.

OS: linux/debian

SDK: 3.1.8

  • Yes, a .barrel file is simply a .zip of the barrel project.  When you compile an app that uses a barrel, the .zip is extracted, and rebuilt for each target.

    To be honest, I've found the best way to test a barrel is to write a simple app that uses the barrel.

  • Thanks for the advice. That's the conclusion I've come to as well. Though I'm still curious as to the purpose of barreltest in this case.

  • I'm not really familiar with the barreltest tool, but the output it is generating is a .PRG file. I'm pretty sure the flow is supposed to be...

    connectiq-sdk-mac-3.1.8 vitek$ ./bin/barreltest -a bin/api.db -y ~/developer_key.der -f ~/Desktop/eclipse-3.2.0/eclipse-workspace/TravisBarrel/monkey.jungle -o fenix5_Test.prg -d fenix5
    connectiq-sdk-mac-3.1.8 vitek$ ./bin/ConnectIQ.app/Contents/MacOS/simulator &
    connectiq-sdk-mac-3.1.8 vitek$ ./bin/monkeydo fenix5_Test.prg fenix5 -t
    
    Error: Unexpected Type Error
    Details: Failed to start CIQ Application
    Stack: 
    
    connectiq-sdk-mac-3.1.8-2020-03-04-e5981d10b vitek$

    In Eclipse you should be able to create a Run No Evil Test run configuration for a barrel and then run it to run the Run No Evil tests in the barrel. I just gave this a whirl and it appears to be broken because I'm getting errors:

    Beginning execution of Run No Evil framework.
    Building project for 'fenix5'... SUCCESS.
    Running tests as 'fenix5'.
    Executing tests for entire project.
    /Users/vitek/Projects/connectiq-sdks/connectiq-sdk-mac-3.1.8-2020-03-04-e5981d10b/bin/monkeydo /Users/vitek/Desktop/eclipse-3.2.0/eclipse-workspace/TravisBarrel/bin/test_fenix5_TravisBarrel.prg fenix5 -t 
    
    Error: Unexpected Type Error
    Details: Failed to start CIQ Application
    Stack: 

    I'm not sure when this was last used/tested, but it appears to have been broken for at least a few releases.

  • When I run the above commands using a 3.0.6 SDK I see the following output:

    connectiq-sdk-mac-3.0.6-2018-11-27-33ec0d6 vitek$ ./bin/monkeydo fenix5_Test.prg fenix5 -t
    ------------------------------------------------------------------------------
    Executing test TravisBarrel.testAnnotatedConstantExists...
    PASS
    ------------------------------------------------------------------------------
    Executing test TravisBarrel.testAnnotatedConstantValue...
    PASS
    
    ==============================================================================
    connectiq-sdk-mac-3.0.6-2018-11-27-33ec0d6 vitek$ 
    

    It looks like it is just broken and we need to fix it.