CIstat preview version released

CIstat is a Python package supporting to fetch, analyze and present automation test data from common CI services as CircleCI, Travis, Bamboo and Jenkins (preview version has implemented CircleCI to verify the idea)

CIstat_preview

Installation

CIstat with preview version could be installed with pip CVS interface from github directly.

Install cistat Pypi dist with dev branch latest code from Github:

pip install https://github.com/maxwu/cistat/archive/dev.zip

Install cistat from master (release) branch on Github:

pip install https://github.com/maxwu/cistat/archive/master.zip

Usage

Sample Charts

Pie Chart

report.get_class_rpt().plot_piechart_casenum(project, "Case Num per Class")

The Pie Chart for case numbers per class level is:

Pie Chart on Case Number per Class

Bar Chart

report.plot_barchart_rate(project, "Pass Rate per case")

The Bar Chart of Pass Rate for cistat project is:

Bar Chart on Pass Rate

Bubble Chart

The ROI chart is more considerable for nightly build result analysis in a testing project within XUnit test framework to control the resources as case, docker/cpu, feature integration.

In the sample height of bubble represents the pass rate and the size of bubble shows the test efforts (Currently it is simply the case number; It will be updated to a formula on test time and case number). The lower the bubble positions, the bigger it shall grow to get more resource and invest to float up to 1.0 the stable status.

report.get_class_rpt().plot_scatter_roi(project, "Test ROI per Class")

The Bubble Chart to present case ROI per class as below:

Bubble Chart on Test ROI

Console Entry Point

Quick step to run above sample: cistat-cli. Since circleci.com has throttling limit on RESTful API, it is recommended to configure an API-Key in environmental variable or local config.yaml.

This console command is installed with Pypi dist. cistat-cli is the console entry point planted by PIP installer setup.py.

Configuration

Configuration item preference order is:

Cli Options >> Environment Variables >> Local Config.yaml

Configuration items:

  • circleci_api_token: The 40 token characters to avoid throttling.

To test string length with shell, printf $STR | wc -c

Design

Data Models

Now CIstat has implemented XUnit model. Which is a widely used test report XML format. CIstat wrappered it with full functional Python type with +, +=, getter/setter with [key] and len() to simplify the further applications.

As a practice to prepare for future Cloud related statD thoughts (as DataDog service), XUnit is firstly picked up as XUnit has counter, scalar and ratio types of data. These data in various types can be accumulated or aggregated with +, += and speficied with [] operator.

The future step on data model is to expand to white-box parts as Lint scores and code coverage.

Requests

So far only circleci RESTful interfaces are supported. TravisCI and Jenkins are on following step while the preview version is under more tests.

References

[1] The README document with samples, https://github.com/maxwu/cistat

[2] My Gitbook chapter about Pip installation from Github, https://maxwu.gitbooks.io/python-tips/content/pip-supports-installation-from-github.html

Change Logs

May 10, 2017: Init post.