Portfolio and To date analysis

Member-level telemetry in portfolio page

Now when user click a sparkline in portfolio page, it will go to the associated member level telemetry analysis, which provides more detail about that sparkline. Actually, the analysis used to generate the sparkline is the member level telemetry. These analysis will return multiple streams, then the sparkline is generated by merging these streams. New attribute "merge" is added to portfolio definitions. Currently the system support "sum", "avg", "min" and "max" merge methods, but only "sum" is actually being used.

At the beginning of trying to implement this feature for portfolio page, I was trying to associate a telemetry analysis to the portfolio measure, and when user click the sparkline, it will invoke the associated telemetry analysis rather than the analysis used to generate the sparkline. But after some coding I found that, the associated telemetry is most likely to have different parameters from the original analysis. That means either the associated analysis has to use default parameters, or we need to add parameter configuration for the associated telemetry. Neither of these choices satisfy us, so we discard this idea and use merging.

To Date Analysis

Portfolio page now is mostly done at current level, so it is good time to start to date analysis page now. Surprisingly, to-date page is pretty much the same as portfolio:

• It has few input: projects only.
• Each project we generate some analysis and list them in a table.
• It has a configuration panel to configure these analysis.

So after copying code from portfolio, I just need to do some small change and it will work.

There are two major issue we need to address:

1. Which analysis should use cumulative value and which should use latest value.
2. When should the accumulation start.

The first one is easy because it can be tell from the cumulative parameter in telemetry definition. If a telemetry has cumulative parameter, that means accumulation make sense to it, therefore it should use cumulative value, and the others should use latest value.

The second one is more difficult to answer. A quick idea is to start from the start date of the project, but that will cause the serious performance issue, especially when requesting the Default project, which start date is set to 2000-01-01 by default. The other problem is, even if start from start date of the project is doable, is it make sense to do so? May be it will be better to let user select the start date? This is the first research question for to-date analysis.

Progress in project browser

Ant task improvement:

After tolerate long verify time for quite a long, I finally start to fix this. When running verify.build.xml, Unit test and EMMA are both invoked. The unit test is to ensure all unit tests are passed, and the EMMA is to get a coverage of the current system. But both of them need to run junit to execute all unit tests in the system. Exactly same tests are executed twice. It is surely a waste of time, especially when these tests takes several minutes to complete and I have to wait until it finish then I can commit the code. So the idea it to combine these two task together. In the EMMA task, it requires to run the junit and track it to get coverage data. But in our old version, the EMMA task just get the coverage data and discard the test result. So I added code to let it monitor the test result as well and generate junit reports in build directory, which can be used to generate junit results and sensor data. Then I add a task in emma for verify, which invoke emma.tool, junit.report, junit.sensor, emma.report, emma.sensor, emma.echo to complete all task in Unit test and EMMA. Now, the verify.build.xml runs almost twice as fast as before.

Telemetry Improvement:

The color issue is fixed now. The random colors are selected in a way that colors have maximum difference from each other. This is done under the color wheel conception. If we need to pick n random color. The first color will be selected randomly, the other colors will have 360/n degree away from the preceding one, so that they will be distributed on the color wheel with maximum distance.

Moreover, the stream color mechanism is improve to take more use of coloring. The new rules are:

• If there are more than 1 unit axis, streams will be colored according to the axes.
• If there is only one axis, but have multiple stream names, streams will be colored according to the stream name.
• If there is only one stream name as well. All stream will be colored separately.

So when users viewing member level telemetry or one telemetry over several projects, they will see a chart will stream of the same color. There will almost always be streams will different colors.

Portfolio Improvement:

Trend interpretation has been improved. Before, only the monotonous trends will be considered as stable, increasing or decreasing. All others will be considered as unstable. This is not the way people interpret trends. People accept a certain amount of vibrate in trends and still consider stable, increasing or decreasing. So I change the system this way. If a point has difference from the preceding point with certain amount, which is calculated as 5% of the average of the stream, will be considered same as its preceding. Otherwise if the point is higher, then it is increasing, if lower, then it is decreasing. Then the system go through the stream. If there is no increasing or decreasing points, the stream is stable. If it has increasing but decreasing points, it is increasing. If it has decreasing but increasing points, it is decreasing. If it has both increasing and decreasing points, it is unstable. In this way the system can accept trends with small vibrations and still classify them as how people usually do. Also the acceptable difference is discussable.

What's more, I added a classifier class to do this job. That means user can write their own classifier to interpret the stream trends, may be using some sophisticated mathematics methods as well.

Thesis outline

The final tech report of 699 will be the draft of my thesis. The semester is half way through, it is time to think about it again to put something new to it.

Development on project portfolio page goes well. But my literature review is far behind schedule. I need to catch up with it now.

Here is the outline of my thesis:

1. Introduction

2. Related work

• 2.1 Software Product and Process Metrics

2.1.1 Product Metrics (Coverage, Complexity)

2.1.2 Process Metrics (LOC, Churn, DevTime)

• 2.2 Software Project Portfolios

2.2.1 Risk Based Software Project Portfolio

2.2.2 Some other Portfolios

3. Research Questions

• 3.1 How useful is this approach?
• 3.2 Does the system successful achieve its goal?
  

4. System Description

• 4.1 Overview
• 4.2 Portfolio Table

4.2.1 Sparklines

4.2.2 Evaluation color rules

• 4.3 User Configurations
• 4.4 Portfolio Definitions
  

5. Experiment Design and Analysis

• 5.1 Research Approach
  
• 5.2 Classroom Evaluation
• 5.3 Industry Approach
  

6. Future directions

Struggling on HTML stylesheet

We lately found that, as the width of project selection field is limited and the multiple selection is not horizontal scrollable, it is no way to tell projects with long name such as hackystat-sensor-ant, hackystat-sensor-eclipse and hackystat-sensor-emacs, so we have to change it. And I found this interesting page Select Multiple Form Fields. It make me think more about the solution. But for Wicket, the multiple selection check box is supported natively. So that will be the easiest and acceptable solution. In order to scale the selection field properly into the input table, I got struggling on HTML stylesheet again. But I am going better with it now. There are too many properties in stylesheet that are overwhelming to me at the beginning. But finally I found this page http://www.w3.org/TR/CSS2/propidx.html to be very helpful. It is a full list of stylesheet properties and quite easy to find what you need.

Also, the user configuration persistence using UriCache and initial portfolio definition XML are implemented. Now the portfolio become more user friendly especially you used to customize your portfolio configuration. But currently there is no working sample for the XML definition, so I want to move some of the hard-wired measures into the sample XML, may be those uncolored process analyses such as Build, Commit and UnitTest.

As students in ICS 413 will use Hackystat soon, the logger for user usage will have the higher implementation priority. Hopefully we will obtain a useful set of evaluation data.