When measuring boxes with the world-facing Intel® RealSense™ camera DS4, I discovered that I needed the ability to automate the capturing of the box image and size data. This would allow a camera mounted over a scale to auto-capture the image and then send a known accurate value back to the system. This type of automation enables the design of a kiosk where placing a box on the scale triggers the image capture and automates the image process, so the clerk doesn’t have to press a button to facilitate the transaction. With the weight and size data calculated, the data can be entered as measured into a mail system programmatically.
Thinking about how to automate the image capture when the images are composed of streaming data presents new problems. The idea presented in this code is to determine when the image has stabilized to a point where we have a high confidence in the data that is being seen. The basis used in this class is that we can use statistics to determine when we have a stable image and thus when to automatically capture the image. To do this, we use the standard deviation model.
The standard deviation model or bell curve is normally represented using this type of graph.
https://en.wikipedia.org/wiki/Standard_deviation
Another way to look at this data is to represent how much of the data is within a range of standard deviations from the mean.
https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule
As shown in the graph, when you are within 1 standard deviation of the mean, you have a 68-percent confidence in the data. By using 2 standard deviations as the cutoff, you know you have a 95-percent confidence level in the accuracy of the data that is coming across the stream from the camera.
Now that I’ve proposed the idea of using the standard deviation as the method for auto-capturing an image, determining the standard deviation for a set of streaming data becomes another issue. Searching for solutions, I came upon a refresher from college days using Knuth. On page 232 of Donald Knuth's The Art of Computer Programming (volume 2, third edition) is a formula for determining variance around the standard deviation for streaming data. The code in this class implements this formula and is documented in the source code header files, exactly which variables are used for which parts of the formula in the code.
This line of formulas are described in this article:
http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
With this class, I now have the ability to determine how many standard deviations away from the mean a data stream is. Here’s how I exposed this knowledge in the UI. In my sample, the Intel RealSense camera was used to measure a 3D box and then draw a bounding box around the item in the picture. When the standard deviation was less than 1, the box was drawn with a red border. When the standard deviation was less than 2, it was drawn as yellow, and anything greater than 2 was drawn as green. This UI implementation of the functionality allows immediate feedback to the applications’ user and helps them visualize the consistency of the data. Of course the stability is also viewed as the box lines drawn over the image also move about less and become more stable until the image freezes when captured.
With that preface, here’s the class just described implemented in C code.
For any questions or discussions, please email me
Dale Taylor Intel Corp
dale.t.taylor@intel.com
This is the source code from the StreamingStats.h file.
/******************************************************************************
Copyright 2015, Intel Corporation All Rights Reserved.
The source code, information and material("Material") contained herein is owned
by Intel Corporation or its suppliers or licensors, and title to such Material
remains with Intel Corporation or its suppliers or licensors. The Material
contains proprietary information of Intel or its suppliers and licensors.The
Material is protected by worldwide copyright laws and treaty provisions. No
part of the Material may be used, copied, reproduced, modified, published,
uploaded, posted, transmitted, distributed or disclosed in any way without
Intel's prior express written permission. No license under any patent,
copyright or other intellectual property rights in the Material is granted to
or conferred upon you, either expressly, by implication, inducement, estoppel
or otherwise. Any license under such intellectual property rights must be
express and approved by Intel in writing.
Unless otherwise agreed by Intel in writing, you may not remove or alter this
notice or any other notice embedded in Materials by Intel or Intel's suppliers
or licensors in any way.
******************************************************************************/
//
// http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
// This code implements a method for determining Std Dev from streaming data.
// Based on Donald Knuth's Art of Computer Programming vol 2, 3rd edition,
// page 232
//
// The basic algorithm follows (to show what the variable names represent)
// X is the current data item
// Mc is the running Mean (Mean current), Mc-1 is the mean for the previous
// item, Sc is the running Sum (Sum current), Sc-1 is the sum for the previous
// item, c is the count or current item
// init M1 = X1 and S1 = 0 during the first pass and on reset
// Data is added to the cumulating values using this formula
// Mc = Mc-1 + (Xc - (Mc-1))/c
// Sc = Sc-1 + (Xc - (Mc-1))*(Xc - Mc)
// for 2<= c <= n the cth estimate of variance is s*2 = Sc/(c-1)
//
#include "math.h"
class StreamingStats {
private:
unsigned int count = 0;
unsigned int index = 0;
double ss_Mean, ss_PrevMean, ss_Sum;
double* ss_Data;
unsigned int ss_Size = 1;
// Internal functions defined here
public:
StreamingStats(unsigned int windowSize); // Constructor, defines window size
~StreamingStats(void) { delete [] ss_Data; }; // destructor for data
int DataCount(); // return # items are in this data set
int DataReset(); // reset the data to empty state
int NewData(double x); // add a data item
double Mean(); // return Mean of the current data
double Variance(); // return Variance of the current data
double StandardDeviation(); // return Std Deviation of the current data
};
Comments on the class and code defined in the H file.
Because the variable sized data structure (ss_Data) is defined via the new command in the constructor, a destructor was defined to assure that delete is called.
This is the source code from the StreamingStats.cpp file.
/******************************************************************************
Copyright 2015, Intel Corporation All Rights Reserved.
The source code, information and material("Material") contained herein is owned
by Intel Corporation or its suppliers or licensors, and title to such Material
remains with Intel Corporation or its suppliers or licensors. The Material
contains proprietary information of Intel or its suppliers and licensors.The
Material is protected by worldwide copyright laws and treaty provisions. No
part of the Material may be used, copied, reproduced, modified, published,
uploaded, posted, transmitted, distributed or disclosed in any way without
Intel's prior express written permission. No license under any patent,
copyright or other intellectual property rights in the Material is granted to
or conferred upon you, either expressly, by implication, inducement, estoppel
or otherwise. Any license under such intellectual property rights must be
express and approved by Intel in writing.
Unless otherwise agreed by Intel in writing, you may not remove or alter this
notice or any other notice embedded in Materials by Intel or Intel's suppliers
or licensors in any way.
******************************************************************************/
//
// http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
// This code implments a method for determining Std Dev from streaming data.
// Based on Donald Knuth's Art of Computer Programming vol 2, 3rd edition,
// page 232
//
// The basic algorithm follows (to show what the variable names represent)
// X is the current data item
// Mc is the running Mean (Mean current), Mc-1 is the mean for the previous
// item, Sc is the running Sum of Squares of Differences (Sum current),
// Sc-1 is the sum for the previous item, c is the count or current item.
// Init M1 = X1 and S1 = 0 during the first pass and on reset
// Data is added to the cumulating values using this formula
// Mc = Mc-1 + (Xc - (Mc-1))/c
// Sc = Sc-1 + (Xc - (Mc-1))*(Xc - Mc)
// for 2<= c <= n the cth estimate of variance is s*2 = Sc/(c-1)
//
#include "math.h"
#include "StreamingStats.h"
StreamingStats::StreamingStats(unsigned int windowSize)
{
if (windowSize > 0)
ss_Size = windowSize;
ss_Data = new double[ss_Size];
return;
}
// This is the only public routine, it returns the count of the # of items used
// to determine the current values in the object.
//
int StreamingStats::DataCount()
{
return count; // return the number of accumulated data items
}
int StreamingStats::DataReset()
{
// ss_PrevMean = ss_Mean = 0.0; // clear all data
// ss_PrevSum = 0.0;
count = 0;
index = 0;
return 0; // start empty, no elements
}
// this routine adds new data to the streaming stats totals
// returns the # of items added to the data set
int StreamingStats::NewData(double x)
{
ss_PrevMean = ss_Mean;
if (count >= ss_Size) { // We're rolling the window
// The oldest data point is the next point in a circular array
index++;
if (index >= ss_Size)
index = 0;
// Remove oldest data point from mean
ss_Mean = ss_Mean - (ss_Data[index] - ss_Mean) / ss_Size;
// Add new data point to mean
ss_Mean = ss_Mean + (x - ss_PrevMean) / ss_Size;
// Remove oldest data point from sum
ss_Sum = ss_Sum - (ss_Data[index] - ss_PrevMean) *
(ss_Data[index] - ss_Mean);
// Add new data point to sum
ss_Sum = ss_Sum + (x - ss_PrevMean) * (x - ss_Mean);
}
else { // We're still filling the window
count++;
if (count == 1) // initialize with the first data item only
{
ss_PrevMean = ss_Mean = x;
ss_Sum = 0.0;
}
else // we are adding a data item, follow the formula
{
ss_Mean = ss_PrevMean + (x - ss_PrevMean) / count;
ss_Sum = ss_Sum + (x - ss_PrevMean)*(x - ss_Mean);
}
index = count - 1;
}
// Store new data point - overwriting oldest in circular array
ss_Data[index] = x;
return count;
}
// if the count is positive, return the new mean
double StreamingStats::Mean()
{
return (count > 0) ? ss_Mean : 0.0;
}
// if the count is 2 or more, return a variance, otherwise zero
double StreamingStats::Variance()
{
return ((count > 1) ? ss_Sum / (count - 1) : 0.0);
}
// calc the StdDev based using sqrt of the variance (standard method)
double StreamingStats::StandardDeviation()
{
return sqrt(Variance());
}
About the Author
Dale Taylor has worked for Intel since 1999 as a Software Engineer. He is currently in Arizona and focused on Atom Enabling, helping our software partners use Intel’s latest chips and hardware. He’s been a programmer for 25 years. Dale has a BS in Computer Science and a Business Management minor, and has always been a gadget freak. Dale is a pilot and spends time in the summer flying gliders over the Rocky Mountains. When not soaring he enjoys hiking, cycling, boating, and photography
Notices
Intel, the Intel logo, and Intel RealSense are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© 2016 Intel Corporation.
This sample source code is released under the Intel Sample Source Code License Agreement.