|
|
Line 1,180: |
Line 1,180: |
| <br> | | <br> |
| | | |
− | | OK | + | | [[Image:Check.png]] |
| |- | | |- |
| | | | | |
Line 1,202: |
Line 1,202: |
| <br> | | <br> |
| | | |
− | | OK | + | | [[Image:Check.png]] |
| |- | | |- |
| | | | | |
Line 1,224: |
Line 1,224: |
| <br> | | <br> |
| | | |
− | | OK | + | | [[Image:Check.png]] |
| |- | | |- |
| | | | | |
Line 1,246: |
Line 1,246: |
| <br> | | <br> |
| | | |
− | | OK | + | | [[Image:Check.png]] |
| |} | | |} |
| | | |
Revision as of 09:11, 31 July 2013
: test done, success
: test done, failure
: test not done yet, or in progress
: test not consistent for this release
|
Results and Status
Device(s) used for tests
|
RaspberryPi for GPIO tests, regular PC,
|
Status ( ok tests / total tests)
|
9 / 15 suites
|
Comments
|
|
Final status with date
|
|
Miscellaneous [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_1
|
Check optional dependency of agent.srvcon on agent.netman
|
Disable agent.netman (agent.config.network.activate = false). Reboot the device and verify the correct initialization of the Mihini
|
|
|
Config Store (agent.config ) [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_2.1
|
Data Reading
|
Test reading whole config, command arg = agent.config.get(""). Verify execution.
|
|
|
Mihini_2.2
|
Data Reading
|
Test reading sub part of config, command arg = agent.config.server. Verify execution.
|
|
|
Mihini_2.3
|
Data Reading
|
1- Modify some config entry, read it using ReadNode command, 2- Call agent.config.default(), read entry again to check default value is read.
|
|
|
Mihini_2.5
|
Test default()
|
Call default (agent.config.default()). Verify that default values are restored.
|
|
|
Mihini_2.6
|
Test diff()
|
Change some values in agent.config. Verify that the differences are correctly reported in the Lua shell.
|
|
|
Server Connection (agent.srvcon) [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_3.0.1
|
Connection policy: Cron
|
agent.config.server.autoconnect = { cron="mm hh dd MM YY"}. Reboot and verify connection according to cron entry.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.0.2
|
Connection policy: On reboot
|
agent.config.server.autoconnect = { onboot = 300}. Reboot and verify connection ~300 seconds after reboot.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.0.3
|
Connection policy: Periodic
|
agent.config.server.autoconnect = { period = 2 }. Reboot and verify a periodic connection every 2 minutes.
|
|
![Check.png](/images/2/2a/Check.png)
|
Commands (agent.devman.cmds) [done]
This chapter must be done using platform jobs: retrieve data, apply settings, send command.
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_3.1.1
|
Connect
|
Send job to device using platform server. Verify execution.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.1.2
|
ReadNode
|
Send read job on Mihini asset to device using platform server. Use empty path (""). Verify execution.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.1.3
|
ReadNode
|
Send job to device using platform server. Use existing path. Verify execution.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.1.4
|
ReadNode
|
Send job to device using platform server. Use non-existing path. Verify execution.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.1.5
|
Reboot
|
Send command to device using platform server. Verify execution.
|
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.1.6
|
UnknownCommand
|
Send command to device. Verify failure as command does not exist.
|
|
![Check.png](/images/2/2a/Check.png)
|
Data Writing [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_3.2.1
|
Data writing on agent.config
|
Executes from platform server a data writing on @sys.config. Verify execution.
|
1- Using datamodel from platform, modify Mihini > Configuration > Shell > HistorySize. 2- On device :":agent.srvcon.connect()" 3- On device: ":devicetree.get("config.shell.historysize")"
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_3.2.2
|
Data writing on asset datatree
|
Create a new asset (donkey) Executes from platform server a data writing on donkey.test. Verify execution.
|
testPolicies.test_SeveralJobs()
|
![Check.png](/images/2/2a/Check.png)
|
Network Manager (agent.netman) [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_4.1
|
Check that agent.netman is optional
|
De-activate agent.netman (agent.config.network.activate = false) activate Fake Netman init signal (agent.config.network.initsignal="SOME_BEARER", with "SOME_BEARER" present in agent.config.mediation.pollingperiod) Obviously, the target must provide somehow a pre-existent network connection
|
|
|
Mihini_4.2
|
Connectivity with ETH or GPRS
|
Set agent.config.network.activate = true, agent.config.network.bearerpriority = {"ETH","GPRS"}, agent.config.network.maxconnectiontime = 30 Set agent.config.network.bearer.GPRS.apn="sim_apn" and agent.config.network.bearer.ETH.mode = "dhcp" Reboot the device with an ethernet cable plugged. Verify that the device is connected on the net with the ethernet bearer
|
|
|
Mihini_4.3
|
|
Unplugged the ethernet cable, and execute a agent.srvcon.connecttoserver() command. Verify that the device is connected with the GPRS bearer. Verify that after 30s a new selection of bearer is triggered.
|
|
|
Mihini_4.4
|
|
Set agent.config.network.bearer.retry = 2, agent.config.network.bearer.retryperiod = 10, verify that agent.config.network.bearerpriority is {"ETH", "GPRS"} Reboot the device without any ethernet cable plugged. Verify that agent.netman tries to connect three times to the ethernet bearer (first attempt + 2 retries). Verify that agent.netman waits 10 seconds between each attempt.
|
|
|
Mihini_4.5
|
|
Re-execute previous test with agent.config.network.bearer.ETH.retry = 3 and agent.config.network.bearer.ETH.retryperiod = 50. Verify that the Mihini uses the specific bearer configuration instead of the "general" network configuration.
|
|
|
Mihini_4.6
|
|
Set agent.config.network.activate = true, agent.config.network.bearerpriority = {"ETH","GPRS"}, agent.config.network.maxconnectiontime = 30 Set agent.config.network.bearer.GPRS.apn = "sim_apn" and agent.config.network.bearer.ETH.mode = "dhcp". Reboot the device with an ethernet cable plugged. Verify that the device is connected with the ethernet bearer.
|
|
|
Mihini_4.7
|
Connectivity with selected bearer
|
Execute agent.netman.connect('GPRS'). Verify that the device is connected with the GPRS bearer.
|
|
|
Mihini_4.8
|
|
Execute agent.netman.connect('ETH'). Verify that the device is connected with the ETH bearer.
|
|
|
Software Update
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_5.1
|
Update job success by AWTDA
|
add the SoftwareUpdate Command, configure the device so that initial conditions are met, connect to Server
|
|
|
Mihini_5.2
|
Local update
|
put update package in drop folder, check update is done, package removed from drop folder, no update done on next boot
|
put file in update/drop/updatepackage.tar :agent.update.localupdate() ls update/drop/
|
|
Mihini_5.5
|
Update job errors
|
test each possible error cause: 1- job already in progress, 2- in manifest file, create a sw_list incompatibility (e.g. Mihini="<2.0"), 3- too many retries for a component
|
|
|
Mihini_5.6
|
Update job retries
|
network errors, Mihini reboot during update (from platform)
|
testMihiniBigSize.zip :agent.appcon.list() ls apps/
|
|
Mihini_5.7
|
Data Reading
|
Send ReadNode Command on some Update table, like {cmd = ReadNode, path=@sys, args=update.swlist}
|
|
|
Mihini_5.8
|
Data Writing
|
check data writing on update data is forbidden: 1- from the platform 2- from the asset/treemgr API.
|
1- on platform Mihini > Update > SWlist > Components > 3 > Version 2- :devicetree.set("update.swlist.components.3.version", 33)
|
|
Mihini_5.9
|
Update Script
|
Update job with update package containing software component which Package field = @sys.update.**
|
SW_update_4_atsys_update.tar
|
|
Mihini_5.10
|
Software Update Lua API
|
Update job with update package targeting an asset registered using "airvantage.asset"
|
test_Update() put file in update/drop/updatepackage.tar :agent.update.localupdate() :agent.update.getstatus()
|
|
Application Container [done]
(Linux Only)
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_6.1
|
Application monitor daemon on Linux: setup
|
Run appmon_daemon before starting RA. Be sure appmon_daemon has sufficient rights to do setuid/setgid actions:
- either run it as root
- or force uid/gid param to be current user
|
|
|
Mihini_6.2
|
Application monitor app restart delay
|
test application with run function that reports error "immediately", check restart delay
|
|
|
Mihini_6.3
|
Application Container and Application monitor communication
|
launch appmon, RA, add std apps in RA.AppCon, kill RA, check apps are still running, restart RA. Check apps haven't been started a second time, add some new app to check communication is still ok
|
|
|
Time[done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_7.1
|
Time unit test suite
|
Activate time but with no periodic synchronization. Require "tests.time", unittest.run(). Note: time unitest #3 need 64 bits system to pass.
|
|
|
Mihini_7.2
|
On boot only synchronization
|
Set agent.config.time.ntppolling = 0. Verify that a NTP sync is done on boot.
|
|
|
Mihini_7.3
|
On boot and periodic synchronization
|
Set agent.config.time.ntppolling = <cron entry>. Verify that a NTP sync is done on boot at <cron entry>. example: "*/3 * * * *" for synchronization every 3 minutes .
|
|
|
Mihini_7.4
|
On boot and periodic synchronization
|
Set agent.config.time.ntppooling = <number>. Verify that a NTP sync is done on boot and every <number> minutes.
|
|
|
Serial Framework (modbus)
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_8.1
|
All API functions in serial (RTU) mode
|
Run test.lua file calling all the API functions in an infinite loop. Verify execution.
|
project serial_modbus_test file main.lua
|
|
Mihini_8.2
|
All API functions in TCP mode
|
Run test.lua file calling all the API functions in an infinite loop. Verify execution.
|
|
|
Messaging (not supported in 0.9) [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_10.1
|
Send SMS
|
Activate SMS management (agent.config.modem = true, agent.config.modem.sms = true). From platform, send SMS using messaging SendSMS command.
|
|
NA
|
Mihini_10.2
|
SMS fallback
|
Activate SMS management (agent.config.modem = true, agent.config.modem.sms = true). Activate SMS fallback (agent.config.network.smsfallback = <phonenumber>. Verify SMS fallback if network bearers (ETH and/or GPRS) are down.
|
|
NA
|
Mihini_10.3
|
SMS notification
|
Activate SMS management (agent.config.modem = true, agent.config.modem.sms = true). From platform, send AWTDA notification (ie Connect command through SMS acting like as a WakeUp). Verify execution.
|
|
NA
|
Migration Helper [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_11.1
|
Test migration helper default script
|
Compile a brand new runtime, start it,
- Check the message "GENERAL-INFO: Migration executed successfully" appears
- Verify that no migration are attempted on the following boots
|
|
|
Mihini_11.2
|
Test migration helper without migration code
|
- Remove migration script (if any):
can runtime/lua/agent/migration.lua or runtime/lua/agent/migration.so
- Executes "persist.save("MihiniVersion", Mihini_11.2)" to trigger a migration attempt on next boot.
- Verify that the next boot reports no migration script found.
- Verify that no migration are attempted on the following boots.
|
|
![(coche)](/images/2/2a/Check.png)
|
Mihini_11.3
|
Test migration helper with migration code written in Lua
|
- persist.save("MihiniVersion", Mihini_11.3)
- put file runtime/lua/agent/migration.lua
- reboot agent
- check logs
|
|
![(coche)](/images/2/2a/Check.png)
|
Mihini_11.4
|
Test migration helper with migration code written in C
|
- persist.save("MihiniVersion", Mihini_11.4)
- compile C MigrationScript using Migration doc examples, see section "Migration script templates"
- Stop agent
- Execute in shell `export SWI_LOG_VERBOSITY="MIGRATIONSCRIPT:info"`
(in order to logs from migration script)
- Start agent
- check logs
|
|
![(coche)](/images/2/2a/Check.png)
|
M2M Server[done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_13.0.1
|
Racon Lua sample on Linux
|
Build and run agent. Run Lua sample in standalone Lua VM. Verify execution.
|
|
OK
|
M2M Server APIs [done]
Test ID
|
Test Purpose
|
Test process
|
Automatic test
|
Result
|
Mihini_13.1.1
|
Racon unit test suite
|
Require "tests.racon", unittest.run().
|
require "tests.airvantageTest" unittest.run()
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_13.1.2
|
Racon data sending
|
Create an asset (av=require "racon"; myasset = av.newasset("myasset"), av.init() myasset:start()). Use the newly created asset to push data. Verify data sending.
|
test_PolicyAll()
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_13.1.3
|
Racon Get-Set-Notify API
|
Create an asset. Use the newly created asset to set data in the Mihini data tree. Verify execution.
|
testJobs.test_SeveralJobs()
|
|
Mihini_13.1.4
|
Racon Get-Set-Notify API
|
Create an asset. Use the newly created asset to get data from the Mihini data tree. Verify execution.
|
testJobs.test_SeveralJobs()
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_13.1.5
|
Racon Get-Set-Notify API
|
Create an asset. Use the newly created asset to register a callback on the Mihini data tree modifications. Verify execution.
|
testJobs.test_asset_devicetree()
|
![Check.png](/images/2/2a/Check.png)
|
Mihini_13.1.8
|
Racon Update API
|
Create an asset. Use the newly created to register a callback on update. Initialize an update of the asset. Verify execution.
|
testJobs.test_SeveralJobs()
|
![Check.png](/images/2/2a/Check.png)
|
Device Status Access (N/A: AAF only) [done]
Test ID
|
Test Purpose
|
Test Process
|
Automatic test
|
Result
|
Mihini_14.4.1
|
Poll & Notify
|
From an application in the Mihini, Poll the device status. From an application in the Mihini, Register on the RSSI.(:devicetree.get('system.net.rssi')
|
|
NA
|
Mihini_14.4.2
|
Radio Information
|
Display all data defined in the AWTDA Dev Tree and check they are relevant. Check the availability of Radio Module data: - Hardware Version: system.cellular.hw_info.hw_ver - Firmware Version: system.cellular.sw_info.fw_ver
|
|
NA
|
Mihini_14.4.3
|
Router Information
|
Display all data defined in the AWTDA Dev Tree and check they are relevant. Check the availability of Router data: - Hardware Version: system.hw_info.hw_ver - Firmware Version: system.sw_info.fw_ver
|
|
NA
|
Mihini_14.4.4
|
Device Versioning
|
Display all data defined in the AWTDA Dev Tree and check they are relevant. From an application in the Mihini, Read the the Mihini version.
(:require "agent.update.common".data.swlist)
|
|
NA
|
Mihini Configuration Access
Test ID
|
Test Purpose
|
Test Process
|
Automatic test
|
Result
|
Mihini_14.5.1
|
Set, Poll & Notify
|
Change some RA data using the TreeMgr Then read this value and ensure that the modification have been taken into account Place a notification hook on a variable and modify it. Check that your hook handle has been triggered
|
test_devicetree_RSSI()
|
|
Factory Reset
Test ID
|
Test Purpose
|
Test Process
|
Automatic test
|
Result
|
Mihini_14.6.1
|
Factory Reset Integration
|
Install an one or more applications in the user space. Change some data in the treemgr. Perform a Reset to factory defaults using the UI and check that all applications and data have been removed.
|
|
|
Mihini Updates
Test ID
|
Test Purpose
|
Test Process
|
Automatic test
|
Result
|
Mihini_14.7.1
|
Mihini updated as part of system update
|
Install one or more applications in the user space. Then perform an update of the Mihini component. Checks the installed applications are still here.
|
|
|
Mihini_14.7.2
|
Packaged apps & libraries
|
Prerequisite: Install an application (here app1.tar) in its version 1.0 - Test: Install an application requiring a newest version and check that the installation is rejected. - Test : Install testMihini application (SW_update_1_depends.tar by example) , then uninstall it by updating with the uninstall_testMihini archive. Check that Mihini is still running (no crash) and testMihini application has been removed.
|
SW_update_1_depends.tar SW_update_1_depends_fail.tar agent.update.localupdate(path,true) with path=/tmp/RA/
|
|
Mihini_14.7.3
|
Can update existing app w/o uninstall
|
- Install an application in version 1.0 - Check the version of the application(:require"agent.update.common".data.swlist) - Perform an update to a new version - Check the new version of the application
|
|
|
Mihini_14.7.4
|
Fresh-install option
|
Install an application containing several files with the parameter "purge=true" Make an update to a new version with new files and new versions of old files. Check that the new files have been taken into account.
|
SW_update_2_old.tar SW_update_2_new.tar agent.update.localupdate()
|
|
Mihini_14.7.5
|
FIO - Base OS version checking
|
Try to install an application requiring a specific version of the OS - with a wrong one: Rejected - with a good one: Accepted
|
|
|
Mihini_14.7.6
|
FIO - API Version checking
|
199 : [Waikiki] Populate the Software Update component/feature list to provide ALEOS, Mihini, etc. as features with corresponding versions. Check in the TreeMgr that those versions are available and up-to-date
|
|
|
Mihini_14.7.7
|
FIO - Component/Library/App dependency version checking
|
Try to install an application requiring a library not present in the update package nor the installed list apps. The installation shall be rejected. Try to install an application requiring a library present in the update package. The installation shall be accepted.
|
SW_update_4.tar
|
|
Mihini_14.7.8
|
Default exec app per package
|
Check that the 'autostart=true' parameter works
|
SW_update_5.tar
|
|
Mihini_14.7.9
|
Global file resource sandboxing
|
Check that the RA runs with specific user rights Check that an application runs with application specific user rights (different from Mihini's user)
|
|
|
Mihini_14.7.10
|
Check application rights use an app that can:
- create file and folder in working dir
- rm files and folders created in working dir
- use persist module to create Lua persisted files.
- etc
|
Use app in attachement, try those test cycles:
- install it, check install status, check app traces, update the app, check install status, check app traces, uninstall it, check uinstall status
- install it, check install status, check app traces, reboot the device, check app traces, update the app, check install status, check app traces, uninstall it, check uinstall status
- install it, check install status, check app traces, reboot the device, check app traces, uninstall it, check uninstall status
|
test_uasuser_rights_persist_installation.tar test_uasuser_rights_persist_update.tar test_uasuser_rights_persist_uninstallation.tar agent.update.localupdate(path,true) agent.update.getstatus() :agent.appcon.list() :devicetree.get("update.swlist.components.3.version")
|
|
Mihini_14.7.11
|
Check many Mihini apps can be installed without rebooting the device
|
choose an app, install it and update it at least 10 times, see if everything is ok at least 10 cycle of install/uninstall for same app install 10 different apps (can be the same application code but 10 different app ids)
|
|
|
Security: M3DA authentication and encryption[done]
Test ID
|
Test purpose
|
Test Process
|
Automatic test
|
Result
|
Mihini_14.9.1
|
none/none
|
Test a simple connection from LUA shell and check the status. No errors should be raised
|
agent.config.server.authentication = nil
agent.config.server.encryption = nil
require 'agent.provisioning'. registration_password 'xy' (where x is first 4 numbers of S/N and y first 4 numbers of ENS)
|
OK
|
Mihini_14.9.2
|
MD5/AES128
|
idem
|
agent.config.server.authentication = 'hmac-md5'
agent.config.server.encryption = 'aes-cbc-128
require 'agent.provisioning' .registration_password 'xy
|
OK
|
Mihini_14.9.3
|
SHA1/AES128
|
idem
|
default config for authentication and encryption
<code>require 'agent.provisioning' .registration_password 'xy'
|
N/A (only HMAC-MD5 and AES-CBC-128 are suppotred)
|
Mihini_14.9.4
|
SHA1/AESCTR256
|
idem
|
agent.config.server.authentication = 'hmac-md5'
agent.config.server.encryption = 'aes-ctr-256' <code>require 'agent.provisioning' .registration_password 'xy' </code>
|
N/A (only HMAC-MD5 and AES-CBC-128 are suppotred)
|
Mihini_14.9.8
|
change password
|
change password on Server side and Mihini. Check that connection is still OK
|
|
0.8 features
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
AppCon - Provide a watchdog library for user apps
|
TBD: time before killing app TBC: diff between kick and kill Make an application with functions doing some very strong jobs (no wait) that will cause the app to not respond for several seconds Test the following cases: - check watchdog is enabled and start the job => app shall be killed/restarted - use API the disable temporarly the watchdog before doing the job => app shall not be killed/restarted - use API to kick the watchdog and start the job => app shall not be killed/restarted. - use API the disable temporarly the watchdog before doing the job and job last more than the watchdog delay => app shall be killed/restarted
|
NA - Removed from 0.8 and 0.9 scope.
|
|
RA C API Implementation - Airvantage Asset Update API
|
Use an update package with an extra parameter in the manifest file and check that extra param is correctly sent to app updater
|
in progres : minh
|
|
|
Airvantage Lua API - make asset update result to be sent asynchronously
|
Create an asset in which we attached a callback function. The latter have to return "async" and then that is a part of this asset which calls sendUpdateResult() to send a asynchronous response to RA
|
in progres : minh
|
|
|
Update - Package Download Resume
|
A C application receiving notification on DWL shall interrupt and resume DL. The resume DL feature shall restart the DL from the last received byte and not from the beginning.
|
in progres : minh
|
|
|
Add timeout management in EMP libs
|
make an asset command blocking for more than 60 seconds install and start it use the RA to call synchronously the command and check that after 60 seconds the RA returns an error.
|
in progres : minh
|
|
|
Update - Download: support user notification and request during download
|
- Start downloading an update file
- "Pause" the download
- Reboot Mihini
- "Resume" the download of update file
|
|
|
Update - check space left before archive extraction
|
Create a tar.gz file which unzipped file is greater than free space disk and give it to updater. The updater shall return an error: file too long
|
|
|
Handle the case Mihini reboot while apps were running
|
Use the Airvantage sample in both lua and C to: - start RA with appmon deamon - install application - make the RA crash (force stop)
|
|
|
Update - Add elastic retries in update package download mecanism
|
TBC: Disconnect the device from the network and start a remote application installation. The download will fail and the first retry shall occur after 1minute, the second shall occur only after 30 minutes and the last one after one hour.
|
|
|
Message dispatch: no ack is sent when msg from srv targets unregistred asset - M3DA
|
Send a command to an unknown asset. Check that the message has been acknowledged.
|
|
|
Airvantage C API: EMP SendData cmd handler is still using hessian to deserialize data
|
Test that a C program is able to receive data
|
|
Security [done]
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
|
use defaultconfig.lua to configure auth and cipher parameters
|
|
|
MD5 / AES-CBC-128
|
Check that with this configuration of authentication method and encryption :
- device is able to be authenticated by the server and authenticate the server (keys have been generated and stored in the keystore)
- device is able to execute commands sent from server
- device is able to receive data from the server
- server is able to receive data from the device
|
![Check.png](/images/2/2a/Check.png)
|
|
MD5 / NONE
|
Check that with this configuration of authentication method and encryption :
- device is able to be authenticated by the server and authenticate the server (keys have been generated and stored in the keystore)
- device is able to execute commands sent from server
- device is able to receive data from the server
- server is able to receive data from the device
|
![Check.png](/images/2/2a/Check.png)
|
|
NONE / NONE
|
Check that with this configuration of authentication method and encryption :
- device is able to connect to the server
- device is able to execute commands sent from server
- device is able to receive data from the server
- server is able to receive data from the device
|
![Check.png](/images/2/2a/Check.png)
|
|
Reset To Factory
|
- Configure the device into one of the security configuration above and ensure that device/server are communicating correctly
- Perform a reset to factory
- Check that device and server are still communicating correctly (keys have been conserved)
|
![Check.png](/images/2/2a/Check.png)
|
REST API
Device Tree [done]
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
GET Node
|
- Perform a GET on "devicetree/config"
- Check the returned configuration table match with the one returned by devicetree.get(config)
|
|
|
|
- Perform a GET on "devicetree/config/server/url"
- Check the returned configuration table match with the one returned by devicetree.get(config.server.url)
|
|
|
|
- Perform a GET on "devicetree/config.server.url"
- Check the returned configuration table match with the one returned by devicetree.get(config.server.url)
|
|
|
|
- Perform a GET on "devicetree/config.server"
- Check the returned configuration table match with the one returned by devicetree.get(config.server)
|
|
|
|
- Create a node in config table :
config[ "1234" ] = { my-leaf = "valid leaf" ;}
- Perform a GET on "devicetree/config/1234/my-leaf"
- Check "valid leaf" is returned
|
![Check.png](/images/2/2a/Check.png)
|
|
|
- Create a node in config table :
config[ "@test" ] = { my-leaf = "valid leaf" ;}
- Perform a GET on "devicetree/config/@test/my-leaf"
- Check "valid leaf" is returned
|
![Check.png](/images/2/2a/Check.png)
|
|
|
- Perform a GET on "update/config"
- Check an error is returned
|
![Check.png](/images/2/2a/Check.png)
|
|
PUT Node
|
- Perform a PUT on "devicetree/config/server" with JSON payload
{url : "somewhere.com" , port : 8088 }
- Check success and persistence over a reboot
|
![Check.png](/images/2/2a/Check.png)
|
|
|
- Perform a PUT on "devicetree/config/server/url" with JSON payload
"somewhere.com"
- Check success and persistence over a reboot
|
![Check.png](/images/2/2a/Check.png)
|
|
|
- Perform a PUT on "devicetree/ALEOS" with JSON payload
{ avms : { url : "toto" ; interval : 20000 } }
- Check success and persistence over a reboot
|
![Check.png](/images/2/2a/Check.png)
|
|
|
- Perform a PUT on "devicetree/config/notexistingnode/leaf" with JSON payload
"valid leaf"
- Check success and persistence over a reboot
|
|
|
|
- Perform a PUT on "devicetree/config/1234/my-leaf" with JSON payload
"valid leaf"
- Check success and persistence over reboot
|
|
|
|
- Perform a PUT on "devicetree/config/server/url" without JSON payload
- Check an error is returned
|
|
|
POST Node
|
- Perform a POST on "devicetree/config/server/url" with JSON payload
{url : "somewhere.com" , port : 8088 }
- Check an error is returned
|
|
|
UPDATE Node
|
- Perform an UPDATE on "devicetree/config/server/url" with JSON payload
{url : "somewhere.com" , port : 8088 }
- Check an error is returned
|
|
Application Container
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
List applications
|
Prerequisite:
- Install some application in application container
Tests steps:
- Perform a GET on application
- Check that returned list match with installed application list
|
[LBA] The test was: "Perform a GET on application/ " The Mihini REST spec was also mentionning "application/"
However when I do a GET on "application/", it failed with "status: 404 No handler found for application/"
On the contrary, "GET on application" works correctly.
I read appcon code, feature export to REST confirm it is done to support "application" without trailing /.
Reading some forum topics and speaking about it here and there, I think supporting only "GET on application" is OK for now.
We may want to support both for next release, but the question can be extended to any REST URL . TBD.
For now, I've edited this test and Mihini REST spec to conform to current behavior, and kept the test result as succesful.
|
|
|
Prerequisite:
- Install some application in application container
Tests steps:
- Perform a POST on application
- Check error is returned
|
|
|
Status
|
Prerequisite:
- Install an application named "testapp" in application container
Tests steps:
- Perform a GET on application/testapp
- Check that status is returned correctly
|
![(coche)](/images/2/2a/Check.png)
[RPE]
The output is correct but I am not completly sure about the variable "appname" (with or without Rest), it contains "2" here... There is a bunch of code in agent/agent/appcon.lua which converts a string coming from the appmon daemon to a table. Please check if this is a bug or not
[LBA]
the field "appname" comes from appmon_daemon, and it would be better if it was named "appid" indeed. So I don't think this is a bug, likely a badly named variable. As this variable is already used in ApplicationContainer and that the issue is only seen when displaying the application status to user, I don't think it's worth renaming it for now. We can open an issue for 0.10 I guess, so I set this test as successful.
|
|
|
Prerequisite:
- noapp is a not an installed application
Tests steps:
- Perform a GET on application/noapp
- Check that an error is returned
|
|
|
Start
|
Prerequisite:
- Install an application named "testapp" in application container
- Ensure application is stopped
Tests steps:
- Perform a PUT on application/testapp/start
- Check that application is now running
|
IN PROGRESS LBA
The Body be set "{}" (empty JSON payload), but the body can't be empty like stated in the Rest doc.
|
|
|
Prerequisite:
- noapp is a not an installed application
Tests steps:
- Perform a PUT on application/noapp/start
- Check that an error is returned
|
IN PROGRESS LBA
|
|
Stop
|
Prerequisite:
- Install an application named "testapp" in application container
- Ensure application is running
Tests steps:
- Perform a PUT on application/testapp/stop
- Check that application is now stopped
|
IN PROGRESS LBA
The Body be set "{}" (empty JSON payload), but the body can't be empty like stated in the Rest doc.
|
|
|
Prerequisite:
- noapp is a not an installed application
Tests steps:
- Perform a PUT on application/noapp/stop
- Check that an error is returned
|
IN PROGRESS LBA
|
|
Configure
|
Prerequisite:
- Install an application named "testapp" in application container
- Ensure application is running
Tests steps:
- Perform a PUT on application/testapp/configure with JSON payload ? {| border="1" |- |
autostart : true |}
- Check that autostart parameter has been modified
|
IN PROGRESS LBA
|
|
|
Prerequisite:
- Install an application named "testapp" in application container
- Ensure application is running
Tests steps:
- Perform a PUT on application/testapp/configure without JSON payload
- Check that an error is returned
|
IN PROGRESS LBA
|
Update
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
Local update
|
- Perform a POST on update/ with HTTP payload containing the binary application (tar)
- Check that application has successfully been installed
|
|
|
|
- Perform a POST on update/ without payload
- Check that an error is returned
|
|
|
|
- Perform a PUT on update/ with HTTP payload containing the binary application (tar)
- Check that an error is returned
|
|
|
|
- Perform a UPDATE on update/ with HTTP payload containing the binary application (tar)
- Check that an error is returned
|
|
|
|
- Perform a GET on update/ with HTTP payload containing the binary application (tar)
- Check that an error is returned
|
|
|
Status
|
- Perform a GET on update/ without payload
- Check that returned status matches with expected status
|
|
GPIO
For all the GPIO tests, here are the prerequisites:
- Use a device with system conforming to GPIO lib requirements (see download.eclipse.org/mihini/api/lua/gpio.html )
RaspberryPI with Raspbian system is a good candidate.
- Build GPIO targets:
GPIO targets are not built by default, do it by typing:
make gpio agent_treemgr_gpio
- Start mihini agent / application with root user rights to be able to access to the GPIO.
Here is a hardware setup to test the GPIO:
TBD
GPIO Lib
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
GPIO read
|
- Connect push button to some device GPIO
- Call gpio.configure on the GPIO with config:
{direction="in", edge="none", activelow="0"}
- Call gpio.read and check returned values are consistent with button state:
button pushed -> 1 button released -> 0
|
|
|
|
GPIO write
|
- Connect LED to somedevice GPIO
- Call gpio.configure on the GPIO with config:
{direction="out", edge="none", activelow="0"}
- Call gpio.write(id, 0), then gpio.write(id, 1)
|
|
|
|
GPIO register / notify
|
- Connect LED to some device GPIO and connect push button to another device GPIO
- Call gpio.configure on the GPIO connected to the button with config:
{direction="in", edge="both", activelow="0"}
- Call gpio.register on the GPIO connected to the button, for the hook function, you can use:
function myhook(id, value) print(string.format("GPIO %d has changed! New value =%s", id, value)) gpio.write(GPIO_led, value) end
- Check LED mimics button actions
|
|
|
|
GPIO automatic configuration
|
- Clear GPIO configuration (e.g. by rebooting the device!)
- Redo 3 first tests without calling configure explicitely
- Check each gpio automatic config after doing test action (read/write/register) by calling gpio.getconfig(id)
|
|
|
|
GPIO listing APIs:
|
- call gpio.enabledlist() after doing at least one of the previous test without rebooting the device
- check GPIO used in tests are listed
- call gpio.availablelist()
- check all device GPIO are listed
|
|
|
GPIO devicetree handler
Call devicetree.init() before doing any devicetree test.
Test ID
|
Test Purpose
|
Test Process
|
Result
|
|
GPIO device tree read
|
- Connect push button to some device GPIO
- Configure GPIO :
devicetree.set("system.gpio.settings.id", {direction="in", edge="none", activelow=0})
- Call devicetree.get("sytem.gpio.id")
|
|
|
|
GPIO device tree write
|
- Connect LED to some device GPIO
- Configure GPIO :
devicetree.set("system.gpio.settings.id", {direction="out", edge="none", activelow=0})
- Call devicetree.set("system.gpio.id", 0) then devicetree.set("system.gpio.id", 1)
|
|
|
|
GPIO device tree register / notify
|
- Connect LED to some device GPIO and connect push button to another device GPIO
- Call devicetree.set("system.gpio.settings.id", {direction="in", edge="both", activelow=0}) on GPIO connected to the button
- Call devicetree.register on the GPIO connected to the button (path: system.gpio.$ID_BUTTON) , for the hook function, you can use:
local function hook(notif) devicetree.set("system.gpio.$ID_LED", notif["system.gpio.$ID_BUTTON"]) end
- Check LED mimics button actions
|
|
|
|
GPIO device tree listing
|
- Do a get on "system.gpio" and "system.gpio.settings", check only enabled GPIO are returned
- Do a get on system.gpio.available, checks all system GPIO are returned as a single leaf value.
|
|
|
Automated tests [done]
All the following tests shall have been successfully run on the release tag.
Test ID
|
Environement
|
Target x86
|
Target x86_64
|
bysant
|
lua framework
|
|
|
luatobin
|
lua framework
|
|
|
posixsignal
|
lua framework
|
|
|
rpc
|
lua framework
|
|
|
sched
|
lua framework
|
|
|
persist
|
lua framework
|
|
|
timer
|
lua framework
|
|
|
bysant
|
agent
|
|
|
luatobin
|
agent
|
|
|
posixsignal
|
agent
|
|
|
rpc
|
agent
|
|
|
sched
|
agent
|
|
|
persist
|
agent
|
|
|
crypto
|
agent
|
|
|
treemgr
|
agent
|
|
|
config
|
agent
|
|
|
update
|
agent
|
|
|
stagedb
|
agent
|
|
|
monitoring
|
agent
|
|
|
asset_tree.lua
|
racon
|
|
|
airvantage.lua
|
racon
|
|
|
devicetree.lua
|
racon
|
|
|
sms.lua
|
racon
|
|
|
system.lua
|
racon
|
|
|
emp.lua
|
racon
|
|
|
dset_test (natif)
|
racon
|
|
|
emp_test (natif)
|
racon
|
|
|
av_test (natif)
|
racon
|
|
|
sms_test (natif)
|
racon
|
|
|
sys_test (natif)
|
racon
|
|
|
updatetest (natif)
|
racon
|
|
|
dt_test (natif)
|
racon
|
|
|